A reverse turing test for detecting machine-made texts

Jialin Shao, Adaku Uchendu, Dongwon Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As AI technologies rapidly advance, the artifacts created by machines will become prevalent. As recent incidents by the Deepfake illustrate, then, being able to differentiate man-made vs. machinemade artifacts, especially in social media space, becomes more important. In this preliminary work, in this regard, we formulate such a classification task as the Reverse Turing Test (RTT) and investigate on the contemporary status to be able to classify man-made vs. machine-made texts. Studying real-life machine-made texts in three domains of financial earning reports, research articles, and chatbot dialogues, we found that the classification of man-made vs. machine-made texts can be done at least as accurate as 0.84 in F1 score. We also found some differences between man-made and machine-made in sentiment, readability, and textual features, which can help differentiate them.

Original languageEnglish (US)
Title of host publicationWebSci 2019 - Proceedings of the 11th ACM Conference on Web Science
PublisherAssociation for Computing Machinery, Inc
Pages275-279
Number of pages5
ISBN (Electronic)9781450362023
DOIs
StatePublished - Jun 26 2019
Event11th ACM Conference on Web Science, WebSci 2019 - Boston, United States
Duration: Jun 30 2019Jul 3 2019

Publication series

NameWebSci 2019 - Proceedings of the 11th ACM Conference on Web Science

Conference

Conference11th ACM Conference on Web Science, WebSci 2019
CountryUnited States
CityBoston
Period6/30/197/3/19

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications

Cite this

Shao, J., Uchendu, A., & Lee, D. (2019). A reverse turing test for detecting machine-made texts. In WebSci 2019 - Proceedings of the 11th ACM Conference on Web Science (pp. 275-279). (WebSci 2019 - Proceedings of the 11th ACM Conference on Web Science). Association for Computing Machinery, Inc. https://doi.org/10.1145/3292522.3326042