UK’s enemies could use AI deepfakes to try to rig election, says James Cleverly | Artificial intelligence (AI)

Criminals and “malign actors” working on behalf of malicious states could use AI-generated “deepfakes” to hijack the general election, the home secretary has said.

James Cleverly was speaking before meetings with social media bosses and said the rapid advancement of technology could pose a serious threat to elections across the globe.

He warned that people working on behalf of states such as Russia and Iran could generate thousands of deepfakes – highly realistic hoax images and videos – to manipulate the democratic process in countries such as the UK.

He told the Times that “increasingly today the battle of ideas and policies takes place in the ever-changing and expanding digital sphere”, adding: “The era of deepfake and AI-generated content to mislead and disrupt is already in play.

“The landscape it is inserted into needs its rules, transparency and safeguards for its users. The questions asked about digital content and the sources of digital content are no less relevant than those asked about the content and sources at dispatch boxes, newsrooms or billboard ads.”

The home secretary will use meetings with Silicon Valley bosses at Google, Meta, Apple, YouTube and others to urge collective action to protect democracy.

It is estimated that 2 billion people around the world will vote in national elections throughout 2024, including in the UK, US, India and 60 other countries.

A number of deepfake audios imitating Keir Starmer, the Labour leader, and the mayor of London, Sadiq Khan, were shared online last year. There have been cases of deepfake BBC News videos purporting to examine Rishi Sunak’s finances.

It comes as major technology companies signed a pact earlier this month to voluntarily adopt “reasonable precautions” to prevent artificial intelligence tools from being used to disrupt democratic elections around the world.

Executives from Adobe, Amazon, Google, IBM, Meta, Microsoft, OpenAI and TikTok gathered at the Munich Security Conference to announce a new framework for how they will respond to AI-generated deepfakes that deliberately trick voters. Twelve other companies – including Elon Musk’s X – are signing on to the accord.

“Everybody recognises that no one tech company, no one government, no one civil society organisation is able to deal with the advent of this technology and its possible nefarious use on their own,” said Nick Clegg, president of global affairs for Meta, the parent company of Facebook and Instagram, in an interview before the summit.