Years ago, orchestrating a campaign of disinformation required extensive resources, involving teams of cyber specialists creating fake news sites, social media profiles, and forgeries to spread false narratives. The rise of artificial intelligence (AI) has dramatically simplified this process.
A recent case by Anthropic has highlighted the potential misuse of AI in political manipulation. The company disclosed that its AI model, Claude, was employed in a political influence operation in Malaysia. This initiative utilized a network of around 1,000 accounts on X (formerly Twitter), operated a fabricated news outlet named “Malaysia Pulse,” and produced a series of counterfeit documents.
This operation aimed to sway voters during the recent state elections by leveraging census and electoral data to exploit divisive social issues such as race, religion, and royalty across Malaysia’s 222 parliamentary constituencies.
Fortunately, the initiative was ultimately disrupted when Anthropic intervened, but it raised serious concerns about how AI might be weaponized in future elections throughout South-East Asia, a region already susceptible to disinformation campaigns.
Katie Harbath, former public policy lead for global elections at Meta, noted that this instance exemplifies how AI enhances the sophistication and accessibility of disinformation tactics. “AI can supercharge it in terms of content creation. It allows for faster, more personalized, and cost-effective generation of false narratives,” she stated. However, she emphasized that despite this efficiency, a distribution channel such as social media platforms remains essential.
According to Anthropic’s recent threat assessment, published in September, the operation in Malaysia showcases how perpetrators can use Claude to establish networks of deceptive social media accounts and news outlets aimed at audiences on multiple continents. The report revealed that these actors included not only governments and state-aligned media but also private firms specializing in influence operations.
Disinformation in South-East Asia
Disinformation is not new to South-East Asia. Journalist Maria Ressa, a Nobel laureate from the Philippines, has characterized her country as a “petri dish” for disinformation strategies that have been refined locally and then exported globally.
However, the advent of AI simplifies and accelerates these tactics. Rossine Fallorina, from the Philippine think tank Sigla, remarked that AI operates much like a “troll factory,” significantly expediting narrative creation. “You don’t need a team anymore; with AI, a single prompt can generate content that spreads rapidly,” he added.
In an effort to curb AI misuse, last year OpenAI banned accounts that utilized AI to produce politically charged comments in the Philippines, many of which supported President Ferdinand Marcos Jnr..
The increasing influence of AI in South-East Asia, a region with largely young and digitally connected populations, has some experts worried about the potential for further division and polarization in public discourse.
Bridget Welsh from the University of Nottingham Asia Research Institute commented, “It feeds political polarization and capitalizes on existing divisions—this is particularly concerning in fragmented contexts like Malaysia.”
The next general election in Malaysia is expected to be called for late 2027, with both the Philippines and Cambodia set for elections in 2028, followed by Indonesia in 2029.
AI has already demonstrated its influence in the region. In Indonesia, a cartoon image of now-President Prabowo Subianto, generated by AI to soften his authoritarian image, gained significant traction during the 2024 election.
AI has also been implicated in inciting tension, particularly against marginalized groups like the Rohingya refugees in Malaysia, as noted by Dr. Ben Loh, a media lecturer at Monash University Malaysia. “Deep-seated prejudices are amplified when people encounter such AI-generated content, reinforcing harmful beliefs,” he explained.
The Anthropic report highlighted that the fake news outlet engaged in significant plagiarism, scraping genuine reports and reworking them under false bylines, while also utilizing articles from state-aligned foreign media without proper attribution.
The company stated that in each instance of identified political influence, they disrupted the activities and used the insights gained to bolster their protective measures, sharing intelligence with relevant authorities.
Looking ahead, Harbath warns that AI might increasingly be used to “flood the zone”—a tactic aimed at overwhelming information streams to manipulate the large language models on which AI relies for content creation. “In areas with limited independent news, the financial investment needed to flood the zone is minimal,” she cautioned.
Editor’s Take
The emergence of AI-driven disinformation poses a significant threat to the integrity of democratic processes, particularly in vulnerable regions like South-East Asia. As the technology becomes more accessible, the potential for misuse will increase, impacting public opinion and societal cohesion. Stakeholders must prioritize safeguards while fostering a discerning digital environment to combat the proliferation of fake news.
Source: www.theguardian.com
