Efficacy Analysis of Online Artificial Intelligence Fact-Checking Tools
DOI:
https://doi.org/10.29173/irie502Keywords:
Artificial Intelligence, Communication, journalism, Political CommunicationAbstract
Investments in artificial intelligence (AI) spurred development of online fact-checking tools; positioned to potentially serve as more accurate alternatives or appendages to search engines and/or nascent chatbots. This study analyzed the efficacy of four AI tools (ClaimBuster, Full Fact, TheFactual - IsThisCredible?, and Google’s Fact-Check Explorer) in producing accurate readings measured by a consensus of independent fact-checking organizations. 10 unique claims were inputted into each tool to produce individual fact-check reports, resulting in 40 fact-check reports being conducted. The results reflect an efficacy rating of 100% regarding the ability of the selected tools to produce an overall accurate reading with 89% of reports producing a unanimous determination. Additionally, recommendations were made to further map and analyze the efficacy of AI fact-checking. These findings support the notion that AI can play an effective role in aiding online truth-seeking when its determinations depend on transparently referencing its source of independent human fact-checkers.
References
Brenan, Megan. “Americans’ Trust in Media Remains near Record Low.” Gallup.Com, Gallup, 5 June 2023, news.gallup.com/poll/403166/americans-trust-media-remains-near-record-low.aspx.
CB | Fact Checker. CB | Fact Checker. 15 Dec. 2022, https://idir.uta.edu/claimbuster/factchecker/.
Full Fact - Full Fact Is the Uk's Independent Fact Checking Organisation. Full Fact - Full Fact Is the Uk's Independent Fact Checking Organisation. 15 Dec. 2022, https://fullfact.org.
Is This Credible?. Is This Credible?. 15 Dec. 2022, https://www.isthiscredible.com.
Fact Check Tools. Fact Check Tools. 15 Dec. 2022, https://toolbox.google.com/factcheck/.
Politifact - Claims That the 2020 Election Was Stolen Are Still False. Politifact - Claims That the 2020 Election Was Stolen Are Still False. 15 Dec. 2022, https://www.politifact.com/factchecks/2022/may/04/gateway-pundit/claims-2020-election-was-stolen-are-still-false/.
Fact Check: No Evidence of 8 Million 'excess' Biden Votes from 2020 Election. Fact Check: No Evidence of 8 Million 'excess' Biden Votes from 2020 Election. 15 Dec. 2022, https://www.usatoday.com/story/news/factcheck/2021/08/10/fact-check-8-million-excess-biden-votes-werent-counted-2020/5512962001/.
Politifact - the Faulty Premise of the ‘2,000 Mules’ Trailer About Voting by Mail in the 2020 Election. Politifact - the Faulty Premise of the ‘2,000 Mules’ Trailer About Voting by Mail in the 2020 Election. 15 Dec. 2022, https://www.politifact.com/article/2022/may/04/faulty-premise-2000-mules-trailer-about-voting-mai/.
Politifact - the Facts of a Fair US Election Have Only Gotten Stronger Since Capitol Attack. Politifact - the Facts of a Fair US Election Have Only Gotten Stronger Since Capitol Attack. 15 Dec. 2022, https://www.politifact.com/article/2022/jan/06/facts-fair-us-election-have-only-gotten-stronger-c/.
Politifact - Georgia’s David Perdue Said Elections Were Stolen from Him and Trump. Pants on Fire!. Politifact - Georgia’s David Perdue Said Elections Were Stolen from Him and Trump. Pants on Fire!. 15 Dec. 2022, https://www.politifact.com/factchecks/2022/mar/29/david-perdue/georgias-david-perdue-said-elections-were-stolen-h/.
Politifact - Why the ‘stop the Steal’ Arizona Republicans Are Wrong About 2020. Politifact - Why the ‘stop the Steal’ Arizona Republicans Are Wrong About 2020. 15 Dec. 2022, https://www.politifact.com/article/2022/aug/03/why-stop-steal-arizona-republicans-are-wrong-about/.
Fact Check: Meme Makes False Claims About Media's 2016 and 2020 Election Coverage. Fact Check: Meme Makes False Claims About Media's 2016 and 2020 Election Coverage. 15 Dec. 2022, https://www.usatoday.com/story/news/factcheck/2020/11/25/fact-check-false-claims-medias-2016-2020-election-coverage/3770232001/.
The Factual’s IsThisCredible.com - 15 Dec. 2022, https://www.thefactual.com/news/#!/search?s=The%202020%20US%20Election%20was%20stolen.
Express Opinion Piece Wrong to Allege Evidence of Widespread Voter Fraud in US Elections. Express Opinion Piece Wrong to Allege Evidence of Widespread Voter Fraud in US Elections. 15 Dec. 2022, https://fullfact.org/news/express-trump-election-voter-fraud/.
Fact Check Tools. Fact Check Tools. 15 Dec. 2022, https://toolbox.google.com/factcheck/explorer/search/the%202020%20US%20election%20was%20stolen;hl=en.
Horne, B. D., et al.. Tailoring Heuristics and Timing AI Interventions for Supporting News Veracity Assessments. Aug. 2020, https://doi.org/10.1016/J.CHBR.2020.100043.
