未经批准:背景自COVID-19大流行以来,亚洲血统的个人(我们指的是北美流行的单词的口语用法,其中亚洲人用来指来自东亚的人,尤其是中国)在离线和在线社区中一直是污名和仇恨言论的主题。遇到这种不公平攻击的主要场所之一是Twitter等社交网络。随着研究界试图理解,分析和实施检测技术,高质量的数据集正变得非常重要。目的在本研究中,我们引入了一个手动标记的Tweets数据集,该数据集具有反亚洲污名化内容。数据集和方法我们对2020年1月至2020年7月在Twitter上发布的668M条推文进行了采样,并使用了迭代数据构建方法,该方法包括三个不同阶段的算法驱动数据选择,最后,我们有志愿者手动手动注释推文,以达到11,263条带有主要标签的推文(未知/无关,不污名化,污名化-低,污名化-媒介,污名化-高)和Tweet子主题(例如,潮湿的市场和饮食习惯,COVID-19病例,生物武器,等。).此外,我们从该数据集中选择了5,000条推文,并由第二个注释器标记它们,然后第三个注释器解决了第一个和第二个注释器之间的标签冲突。我们将最终的数据集作为高质量的Twitter数据集,介绍了COVID-19大流行期间对中国人的污名。数据集和标签说明可以在Github存储库中查看[46]。我们实施了一些最先进的模型来检测污名化的推文,为我们的数据集设置初始基准。我们的结果表明,当使用传统模型(如支持向量机以73%的精度执行)检测看不见的数据时,来自变压器的双向编码器表示(BERT)模型达到了79%的最高精度。结论我们的数据集可以用作围绕该问题进行进一步定性和定量研究和分析的基准。它首先重申了对全球亚洲人口的普遍歧视和污名的存在和意义。此外,我们的数据集和随后的论点应该有助于来自各个领域的其他研究人员,包括心理学家,公共政策当局,和社会学家,分析复杂的经济,政治,历史,以及反亚洲污名化和仇恨行为的文化根源。手动注释的数据集对于开发可用于检测污名或有问题的文本的算法至关重要。尤其是在社交媒体上。我们相信这一贡献将有助于预测,随后设计干预措施,将大大有助于减少污名,恨,以及在像COVID-19这样的未来危机中对边缘化人群的歧视。
BACKGROUND: Since the advent of the COVID-19 pandemic, individuals of Asian descent (colloquial usage prevalent in North America, where \"Asian\" is used to refer to people from East Asia, particularly China) have been the subject of stigma and hate speech in both offline and online communities. One of the major venues for encountering such unfair attacks is social networks, such as Twitter. As the research community seeks to understand, analyze, and implement detection techniques, high-quality data sets are becoming immensely important.
OBJECTIVE: In this
study, we introduce a manually labeled data set of tweets containing anti-Asian stigmatizing content.
METHODS: We sampled over 668 million tweets posted on Twitter from January to July 2020 and used an iterative data construction approach that included 3 different stages of algorithm-driven data selection. Finally, we found volunteers who manually annotated the tweets by hand to arrive at a high-quality data set of tweets and a second, more sampled data set with higher-quality labels from multiple annotators. We presented this final high-quality Twitter data set on stigma toward Chinese people during the COVID-19 pandemic. The data set and instructions for labeling can be viewed in the Github repository. Furthermore, we implemented some state-of-the-art models to detect stigmatizing tweets to set initial benchmarks for our data set.
RESULTS: Our primary contributions are labeled data sets. Data Set v3.0 contained 11,263 tweets with primary labels (unknown/irrelevant, not-stigmatizing, stigmatizing-low, stigmatizing-medium, stigmatizing-high) and tweet subtopics (eg, wet market and eating habits, COVID-19 cases, bioweapon). Data Set v3.1 contained 4998 (44.4%) tweets randomly sampled from Data Set v3.0, where a second annotator labeled them only on the primary labels and then a third annotator resolved conflicts between the first and second annotators. To demonstrate the usefulness of our data set, preliminary experiments on the data set showed that the Bidirectional Encoder Representations from Transformers (BERT) model achieved the highest accuracy of 79% when detecting stigma on unseen data with traditional models, such as a support vector machine (SVM) performing at 73% accuracy.
CONCLUSIONS: Our data set can be used as a benchmark for further qualitative and quantitative research and analysis around the issue. It first reaffirms the existence and significance of widespread discrimination and stigma toward the Asian population worldwide. Moreover, our data set and subsequent arguments should assist other researchers from various domains, including psychologists, public policy authorities, and sociologists, to analyze the complex economic, political, historical, and cultural underlying roots of anti-Asian stigmatization and hateful behaviors. A manually annotated data set is of paramount importance for developing algorithms that can be used to detect stigma or problematic text, particularly on social media. We believe this contribution will help predict and subsequently design interventions that will significantly help reduce stigma, hate, and discrimination against marginalized populations during future crises like COVID-19.