Title: X-FACT: A New Benchmark Dataset for Multilingual Fact Checking
Abstract: https://arxiv.org/abs/2106.09248
In this work, we introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims. The dataset contains short statements in 25 languages and is labeled for veracity by expert fact-checkers. The dataset includes a multilingual evaluation benchmark that measures both out-of-domain generalization, and zero-shot capabilities of the multilingual models. Using state-of-the-art multilingual transformer-based models, we develop several automated fact-checking models that, along with textual claims, make use of additional metadata and evidence from news stories retrieved using a search engine. Empirically, our best model attains an F-score of around 40%, suggesting that our dataset is a challenging benchmark for evaluation of multilingual fact-checking models.
Homepage: https://github.com/utahnlp/x-fact
@inproceedings{Gupta_2021,
title={X-Fact: A New Benchmark Dataset for Multilingual Fact Checking},
url={http://dx.doi.org/10.18653/v1/2021.acl-short.86},
DOI={10.18653/v1/2021.acl-short.86},
booktitle={Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
publisher={Association for Computational Linguistics},
author={Gupta, Ashim and Srikumar, Vivek},
year={2021} }