Deepfake democracy: AI manipulation of India’s 2024 election

Do you want a rival to run for office on your behalf? Confound voters by showing them a video that is both real and fake? As India gets ready to hold the largest elections in the world, political parties are looking to artificial intelligence for innovative and perhaps risky ideas.

Delhi, India — New Delhi At the same time as voters were waiting in line early in the morning on November 30 of the previous year to cast their ballots in legislative elections to select the future administration of the state of Telangana in southern India, a seven-second video clip began to spread over social media platforms.

It was posted on X by the Congress party, which is in opposition on a national level and was present in the state at the time. The image displayed KT Rama Rao, a leader of the Bharat Rashtra Samiti, which was the party that was running the state at the time, urging people to vote in favor of the Congress.

According to a top leader who asked to remain anonymous, the Congress party disseminated it extensively throughout a variety of WhatsApp groups that were “operated unofficially” by the Democratic Party. After some time, it was uploaded to the official X account of the party, where it was viewed more than half a million times.

That was a hoax.
In an interview with Al Jazeera, the leader of the Congress party stated, “Of course, it was AI-generated—even though it appears to be completely real.” “However, a typical voter would not be able to differentiate between the two; voting had already begun [when the video was uploaded], and there was no time for [the opposition campaign] to control the damage.”

It was a marker of the flood of AI-produced, or manipulated, media that tarnished a series of elections in India’s states in recent months, and that is now threatening to fundamentally impact the country’s upcoming general elections. The timely deepfake was a marker of the flood of media that was generated by artificial intelligence.

In the largest elections in the history of the globe, which will take place between March and May, over one billion voters in India will choose their representative for the next national administration. When phony sexually explicit photographs of the musician Taylor Swift appeared on social media platforms in January, the world’s attention was drawn to the dangers that are posed by media that is generated by artificial intelligence that is capable of deception.

Deepfake democracy: AI manipulation of India’s 2024 election

During the month of November, Ashwini Vaishnaw, the Minister of Information Technology in India, referred to deepfakes as a “threat to democracy.” Prime Minister Narendra Modi has also expressed similarities to these concerns.

Nevertheless, teams from all of India’s political parties, including Modi’s Bhartiya Janata Party and the Congress, are using deepfakes to sway voters, according to managers of over forty previous campaigns who spoke with Al Jazeera. This is because the availability of useful artificial intelligence technologies has expanded. A number of the artificial intelligence technologies that are used to make deepfakes are free of charge, but others can be obtained through a subscription for as little as ten cents per movie.

Establishing one’s perception
When it comes to campaigning, the Bhartiya Janata Party (BJP), which is often considered to be India’s most technologically advanced party, has been in the forefront of using illusions. Even as far back as 2012, the party made use of 3D hologram projections of Modi so that he could simultaneously “campaign” in dozens of different locations at the same time. A significant amount of the tactic was utilized during the general elections that took place in 2014 and propelled Modi to power.

Even though there was not much deceit involved, in February of 2020, Manoj Tiwari, a member of parliament for the Bhartiya Janata Party (BJP), became one of the first people in the world to utilize deepfakes for campaigning purposes. Tiwari addressed voters in Delhi in three videos, one each in Hindi, Haryanvi, and English, in advance of the legislative assembly elections in the capital city.

This allowed him to communicate with three separate audiences in the city that is known for its multiculturalism. Only the Hindi video might be considered genuine: They were both deepfakes, which means that artificial intelligence was used to construct his voice and words, as well as to alter his facial expressions and lip movement. This was done in order to make it nearly difficult to tell that they were not authentic simply by looking at them.

In recent months, the Dravida Munnetra Kazhagam (DMK), which is the ruling party in the southern state of Tamil Nadu, has utilized artificial intelligence to bring back to life its legendary leader M. Karunanidhi. This was accomplished by using recordings of the former movie writer and veteran politician attending campaign rallies that were extremely lifelike.

Now, strategists and campaign managers are predicting that the elections in 2024 could be the catalyst for an even greater increase in the usage of deep fakes.

“Politics is about creating perception; with AI tools [of voice and video modulation] and a click, you can turn the perception on its head in a minute,” said Arun Reddy, the national coordinator for social media for the Congress, who handled the party’s tech-savvy Telangana poll. Reddy was in charge of the election. In addition, he stated that the team was brimming with ideas to include artificial intelligence into campaigns, but that they did not have the “trained people” to carry out all of the ideas.

Reddy is working to enhance his team, and other parties are doing the same.

According to what Reddy said to Al Jazeera, “AI will have a resounding effect in having the narrative created.” There will be a multiplication of the content that is manipulated by artificial intelligence in the political sphere, which will be significantly more than it has ever been.

Campaigns are becoming increasingly bizarre.
The Indian Deepfaker is an artificial intelligence startup that is led by Divyendra Singh Jadoun, who is thirty years old and hails from the desert town of Pushkar in western India. His company, which was established in October 2020, copied the voice of Ashok Gehlot, the Congress contender for the position of chief ministerial candidate in Rajasthan state.

This allowed his team to send personalized messages on WhatsApp, addressing each voter by their name, during the assembly elections that took place in November. The Indian Deepfaker is currently collaborating with the team of Prem Singh Tamang, the Chief Minister of Sikkim, to create holograms that will be used during upcoming promotional efforts. Perched on the Himalayas between India, Bhutan, and China, Sikkim is one of the smallest states in India. It is located in the northeastern region of the country.

He stated that this is the work that is clean and official. Furthermore, over the course of the past several months, he has been inundated with what he refers to as “unethical requests” from political campaigns. In an interview with Al Jazeera conducted over the phone, Jadoun stated that political parties communicate with one another in a roundabout way by using international numbers on WhatsApp, burner handles on Instagram, or connect on Telegram.

According to him, his company turned down more than fifty requests of this nature during the election in November. These requests were made by prospective customers who wanted recordings and audio to be manipulated in order to target political opponents, even with pornography.

Jadoun stated that his company is taking extra precautions to avoid getting into any legal difficulties because it is a startup. It is a really unethical use of artificial intelligence, he added. However, I am aware of a great number of individuals that are doing it for really inexpensive fees and are freely available at this time.

During the election campaigns for the state legislatures of Madhya Pradesh in central India and Rajasthan in the west of India in November of last year, the police registered multiple cases for deepfake videos targeting senior politicians such as Modi, Shivraj Singh Chauhan, Kailash Vijayvargia (all of whom are members of the BJP), and Kamal Nath (Congress).

In many cases, the production of deepfake content is contracted out to private consulting firms. These firms rely on social media networks for dissemination, with WhatsApp serving as the primary distribution platform.

Al Jazeera was informed by a political consultant who requested anonymity that a large number of ordinary persons who do not have a public profile are registered on WhatsApp and used for the campaigns. This is done to make it more difficult for anybody to immediately connect them back to parties, candidates, consultants, and entities that specialize in artificial intelligence.

Over the course of the previous year’s assembly elections, this consultant managed six campaigns for both the BJP and the Congress. According to what they said, “Where deepfakes were primarily circulated, we were using the phone numbers of construction laborers to run our network on WhatsApp in the state of Rajasthan.”

A consultant, whose own candidate was targeted with one of these recordings, stated that AI-manipulated audios are particularly useful tools in smaller constituencies. These tools are used to “target candidates with forged call recordings about arranging ‘black money’ for elections or threatening someone to buy votes,” the consultant said. For the purpose of presenting the recordings as evidence of corruption, the candidates’ voices are typically masked on the recordings.

The use of artificial intelligence to influence voters is not regarded as a wrongdoing by any political party, they noted. “This is merely a component of the overall campaign strategy.”