Looking for an Original Synth: Synthenticity and Authenticity in GenAI Content Creator Tutorials
Tanja Wiehn (Christoffer Bagger)
“I've been looking for an original sin One with a twist and a bit of a spin“ - Pandora’s Box, Original Sin (The Natives are Restless Tonight)
The proliferation of generative AI platforms for text, image, and video generation have exploded in the last couple of years. This change has hit the creative field of online content production, just as everywhere else. However, given the particular preoccupation with authenticity as a unique selling point in the creator economy (Hund, 2025; Tolson, 2010), this would seem to make the usage of such tools for synthetic content, such as GenAI, anathema to these platforms (Crawford, 2021; Wiehn, 2024). It would stand to reason that this would go doubly for content promoting and tutorializing synthetic content on these platforms. In this paper, we empirically and critically examine how content creator tutorials negotiate the legitimacy and authenticity of synthetic content. Specifically, we analyze a sample of twenty YouTube video tutorials, in which content creators tutorialize and advocate for the usage of generative AI for the wholesale production of online content. Our most striking overall finding is that these videos are extremely similar in their structure and content, regardless of whether the producer themselves is a synthetic influencer (synfluencer) or a flesh and blood person, from a small or large channel, or whether the channel generally concerned itself with GenAI. Our findings contribute to broader discussions of the legitimacy of generative AI usage in creative processes of platformed cultural production (Poell et al., 2021). These tutorials reflect both the self-similarity of successful creative content in specific (Archer & Jockers, 2016; Bordwell, 2006; De Waard, 2024), and of instructive popular communication in general (Gregg, 2018). Our argument is that, taken at face value, these channels both produce similar content and promote the production of similar content. If influential, they are accelerants of mimetic isomorphism (DiMaggio & Powell, 1983), in which content creators follow and imitate one another. We argue that this is likely a response to the uncertainty introduced by GenAI technologies (Fleming, 2019), which make an already precarious industry (Glatt, 2022; Nieborg & Poell, 2018) seem even more uncertain. As such, the proselytization of GenAI content creation risks making the proliferation of GenAI tools into an influential infrastructure of uncertainty.
Anticipatory (non)compliance: How AI streamers exploit the content moderation game
Blake Hallinan (CJ Reynolds)
For social media platforms, content moderation is an “exhausting and unwinnable game“ (Gillespie, 2018), necessary to create an environment where users and advertisers want to be. Forcreators and strategic operatives, content moderation is a game of adversarial creativity, where they test and subvert boundaries to make money or achieve political goals. The availability of generative AI raises the stakes, providing new tools to both enforce and exploit policies. While there is growing research on how creators integrate generative AI (e.g., Lyu et al., 2024) and the challenges of moderating synthetic content (e.g., Moreno, 2024), AI agents on social media introduce unique complications. To investigate how AI agents navigate content moderation, we turn to livestreaming, where the challenges are most acute. Because livestreaming involves real-time interactions between a streamer and chat, centralized platform moderation tools struggle to keep up, and platforms typically delegate responsibility to streamers (Cai et al., 2021; Thach et al., 2024). We focus on the case of Neuro-sama, the most-subscribed streamer on Twitch and an LLM-powered chatbot (Hale, 2026). We gathered media coverage about Neuro-sama (n=64) and collected every relevant thread in r/LivestreamFail (n=108). We then read through all of the material, watched all attached clips, and identified patterns related to practices of content moderation and fan reactions. We map the different sites of content moderation in AI streaming, including the moderation of the agent, live captioning, and chat participation. The challenges of content moderation are complicated by multi-streaming, especially when platforms operate under very different regulatory environments like Twitch (US) and Bilibili (China). Perhaps the bigger challenge comes from the antagonism of the chat, who enjoy making the robot say bad things. Neuro-sama’s rise to fame was boosted by her early ban on Twitch for, reportedly, denying the holocaust, following in an ignoble chatbot tradition set by Microsoft’s Tay (Hallinan, 2019). Almost all of her high-profile moments depict edgy behavior. We argue that the creator of Neuro-sama has effectively gamified content moderation, programming anticipatory (non)compliance into the virtual agent, balancing the need to engage the audience and appease multiple platforms. We conclude with a discussion of what AI streamers tell us about the future of platform governance in the creator economy.
Audience Emotional Responses to AI-Generated Social Media Content
Beste Budan Erdoğan
Content generated by artificial intelligence is rapidly circulating on social media platforms, increasingly shaping user experiences and forms of interaction. This content is interpreted by users in various ways and met with a range of reactions. While some users tend to question whether the content was generated by artificial intelligence and seek to verify it, others evaluate the videos by directly attributing them to truth. Evoking sadness at times, joy at others, and often prompting viewers to engage in moral reflection, this content—despite being artificially generated—can elicit genuine and intense emotional responses in viewers. With regard to this point, the study examines interesting cat videos generated by artificial intelligence. The primary reason for selecting this content is its high potential for engagement among social media users and its ability to elicit emotional responses from the viewers. These emotional reactions might look simple at first, but they are an important data source. They show how much users know about AI-generated content and how they understand it. The study uses comments from AI-generated cat videos on Instagram as its sample. These comments are separated into two groups: users who know the content is AI and users who do not. The comments with AI awareness mention or hint that AI made the video. The comments without awareness treat the video as a real event or real animal behavior, and they show emotional reactions based on this belief. During the analysis process, these comments are processed using Python-based text mining and natural language processing (NLP) techniques. First, data cleaning and preprocessing steps are applied to remove unnecessary characters, emojis, and repetitive phrases, preparing the text for analysis. Subsequently, sentiment analysis is applied to the comments, and each comment is classified into positive, negative, neutral, or mixed emotional categories. The data obtained is used to comparatively evaluate the emotional response distributions of user groups that demonstrate AI awareness and those that do not. In this way, the potential effects of AI awareness on viewers’ emotional responses to content are systematically examined. The analysis process allows for both measuring emotional intensity and understanding the contextual meaning of comments.
Fact-Checks Can Help Inoculate LLMs Against Disinformation
Morgan Wack (Eva-Maria Vogel, Christian Pipal)
Large language models (LLMs) have become part of the digital influence infrastructure that shapes how citizens encounter political information. Hundreds of millions of users now query AI systems to evaluate claims, yet these systems were trained on an open internet that state-sponsored disinformation operations target through high-volume flooding strategies (Paul & Matthews, 2016). When mainstream coverage of a topic is thin, disinformation producers hold a structural advantage (Golebiewski & boyd, 2018). Whether this ""data voids"" mechanism extends from search engines to LLMs has not been tested empirically, though initial audits have documented worrying evidence of contamination (Alyukov et al., 2025). This study audits four frontier LLMs across 2,268 queries on 63 documented fabrications from eight state-backed influence operations. Models correctly reject 81% of fabrications, but 19% of responses either leave users unable to assess veracity or repeat disinformation outright. Encouragingly, we find that the presence of even a single published fact-check can shift model behavior from hedging to definitive rejection (p < .001). Fact-checked narratives produce 93% correct rejections, compared to 76% for unchecked narratives. The mechanism appears to operate through training data. Fact-checks published before a model's training data cutoff produce strong inoculation (p = .002), while those published after do not reach significance. Providing further evidence of this mechanism, these effects scale with each model's cutoff date and models draw on the specific vocabulary of published corrections in their responses. These findings reframe the role of fact-checking infrastructure in democratic information ecosystems. Prior work has established that fact-checks reduce misperceptions across countries and topics (Walter et al., 2020), but has focused on human audiences. Our results suggest fact-checks have acquired an unintended second audience in AI systems. Sustaining fact-checking capacity is not only a question of reaching citizens directly but of maintaining the quality of the information environment from which AI systems learn.
How Political Narratives Are Shaped and Distributed by Large Language Models
Esther Omemu
The media as active agents in the construction of political narrative is not a new phenomenon but the concept has received renewed attention since Large language models (LLMs) became new interfaces for information seeking. Studies on LLMs have recorded many cases of these models used by ordinary users to understand political events, conflicts, and global affairs. As such, they operate as emergent influence infrastructures with the potential to shape and distribute geopolitical narratives. Despite the growing concern over this new reality, little is known about how LLMs become active actors in the construction of political narratives. Existing research on LLM bias, misinformation, or factual accuracy does not capture this and it constitutes a significant empirical and conceptual gap. This study addresses the gap through the lens of algorithmic mediated visibility which explains how networked sets of computational algorithms mediate what counts as visible, invisible, credible, legitimate, or even thinkable to users. It reframes LLMs as actors within broader “visibility regimes“ through a mixed‑methods prompting experiment by examining how five mainstream LLMs (ChatGPT, Claude, DeepSeek, Le Chat and Grok) respond to simulated user questions about the Russia–Ukraine war. The analysis spans two languages (English and Norwegian) to capture linguistic and regional differences. English functions as the dominant language of international political communication and global media where Norwegian, a low training resource language, a neighbouring state to Russia, largest contributor to Ukraine and an alleged war profiteer, introduces a different political context. Across 400 model responses, the study codes for the presence, absence, and presentation (does the model support or reject the narrative or refuses to answer) of documented Russian-origin strategic narratives. Quantitative descriptive statistics will capture frequency, cross‑model and cross‑language variation, while qualitative narrative analysis will examine how models build meanings around the presented narratives. The study will generate new insights into how LLMs narrativize geopolitical conflict. By foregrounding narrative construction rather than factual accuracy or bias alone, this study offers a new lens for understanding LLMs as emerging actors within broader digital influence ecosystems. It contributes to debates on the evolving role of AI systems in shaping public interpretations of conflict.
Artificial Voices, Real Impact: The Rise of AI-Driven Political Influencers in Poland
Malgorzata Szumna
AI is increasingly being leveraged by disinformation actors to enhance the reach and sophistication of their operations. Following the surge in AI‑generated posts and, subsequently, deepfakes, we now observe a rapid rise in AI‑generated influencers. These synthetic personas are becoming an important vector for shaping and shifting public opinion. From Amelia in the UK to artificial influencers in Germany encouraging support for Alternative für Deutschland, and Prawilne_Polki in Poland, far‑right movements are consistently adopting such tools to communicate with their audiences. While these actors do not yet match the visibility of major commercial AI influencers, their influence is growing steadily. These personas are not always designed to spread disinformation, yet the distinction between malicious influence and persuasion is becoming increasingly blurred, particularly as legal responsibility for their content remains difficult to define. Bold and outspoken, AI influencers possess a troubling appeal – both for the audiences who follow them and for the creators who exploit them for strategic gain. This makes it all the more important to confront the questions their growing presence raises within the social media landscape. This presentation aims to examine: 1. Why far‑right groups are early adopters of AI influencers: who they aim to influence, how they tailor their messaging, and which platforms they prioritise. 2. What we know about public trust toward AI-generated personas and this mode of political communication. 3. Whether other political or civic actors should consider using similar tools to reach their audiences. The presentation will primarily focus on the Polish context, where AI influencers increasingly intersect with anti‑migrant narratives and Polexit‑related messaging, while also drawing on examples from other countries to highlight broader trends.
