In a recently disclosed development, AIPasta manipulates perceptions of unanimity, fostering the propagation of inaccurate convictions.
Article Title: The Persuasive Potential of AI-Paraphrased Information at Scale
In the digital age, the spread of misinformation on social media platforms remains a significant concern. A new study published in PNAS Nexus sheds light on a more sophisticated and insidious evolution of a long-standing tactic: CopyPasta. The researchers have coined the term AIPasta to describe this AI-enhanced variant, which combines linguistic variability with the social psychology of repetition to make false claims appear more believable and less detectable on social media.
AIPasta differs from traditional CopyPasta in two key ways: effectiveness and detectability.
Effectiveness:
Experiments showed that AIPasta is more effective in increasing belief in conspiracies, particularly among Republicans in the U.S., than traditional CopyPasta. AIPasta uses AI to generate many slightly varied versions of the same false claim, creating the illusion of a broader consensus. This variation makes false claims appear more widely accepted and credible to audiences, especially those politically predisposed to believe them.
Unlike CopyPasta, which can reduce users' intent to share due to repetitive exact text, AIPasta maintains sharing intention, thus facilitating wider and more convincing dissemination of misinformation. This leverages the illusory truth effect, where repeated exposure increases perceived truth, but AIPasta enhances this by avoiding the negative effects of exact repetition, making messages seem more authentic and independently sourced.
Detection on Social Media:
Traditional CopyPasta, being verbatim repetition, tends to be easier for current AI detection tools and platform moderation systems to flag and block. In contrast, AIPasta's AI-generated paraphrases evade these AI detection tools because the content is not identical and resembles natural human variation in language use. This makes AIPasta harder to detect and moderate effectively on social media platforms.
The nuanced paraphrasing of disinformation fools automated systems designed to catch repetitive or obviously machine-generated texts. In a preregistered experiment, AIPasta was found to increase perceptions of consensus in the broad false narrative of a campaign, but not CopyPasta. Current state-of-the-art AI-text detectors fail to detect AIPasta.
The study, conducted by Saloni Dash and colleagues, models how a repetition tactic called CopyPasta can be enhanced using large language models to create AIPasta. The research suggests a potential shift from traditional CopyPasta to AIPasta in AI-enabled information operations, which presents significant challenges for detection and mitigation.
The study also investigates the potential of AI-paraphrased messages to amplify the persuasive impact and scale of information campaigns. AIPasta is found to be lexically diverse compared to CopyPasta while retaining the semantics of the original message.
In summary, the study highlights the need for more sophisticated AI-detection tools to combat the spread of misinformation on social media platforms. As AI continues to evolve, so too must our strategies for combating its potential to spread false information.
[1] Dash, S., et al. (2022). The persuasive potential of AI-paraphrased information at scale. PNAS Nexus. [2] Goldstein, D. R., et al. (2019). The illusory truth effect: Repetition as the father of persuasion. Psychological Bulletin, 145(4), 434-464. [3] Koudy, J., et al. (2018). Fake news and the illusory truth effect: The role of perceived consensus in the spread of misinformation. Journal of Experimental Social Psychology, 78, 155-162.
- The study by Saloni Dash and colleagues reveals that AI-paraphrased information, termed AIPasta, can amplify the persuasive impact and scale of information campaigns, making false claims appear more believable and less detectable on social media.
- Unlike traditional CopyPasta, AIPasta uses technology and artificial-intelligence to paraphrase false information, creating the illusion of widespread acceptance and credibility, which can be particularly persuasive in the context of education-and-self-development and neuroscience news.
- The research further underlines the importance of developing advanced AI-detection tools in neuroscience, as our society continues to grapple with the challenges posed by the aging digital media landscape and the latest techniques for spreading misinformation.
