A new AI detection tool from Meta, which the tech company previewed this week alongside the launch of its image-generation model, Muse Image, failed to identify some of its own AI-generated images once they were cropped, according to a Reuters analysis.
The finding highlights the challenges of verifying AI-generated images after common alterations, a limitation that could make it harder to identify deepfakes online during a busy election year that includes the U.S. midterms.
In an analysis of 40 images generated using Muse Image, Reuters found the detection tool verified all of the original AI-generated images but failed to verify 55% of the same images after they were cropped to approximately one-third to one-half of their original size.
On its website, Meta says the preview detection tool can identify its own AI-generated images, even if they are cropped, through an invisible watermarking system called Content Seal, which is embedded in every image generated by Muse Image and designed to help users verify whether it was created by Meta's AI models.
When asked about the results of the Reuters analysis of the detection tool, Meta noted that the tool was a preview. The company said the watermark is designed to remain intact after common edits, but that the signal may be lost if an image is heavily cropped.
Rival tech companies Google and OpenAI have cautioned that their own detection tools are not foolproof against image-alteration techniques.
In March, Meta's Oversight Board, a body of experts that makes binding decisions and issues recommendations on content issues across the company's social media platforms, called on the company, opens new tab to do more to address the "proliferation of deceptive AI-generated content" on its platforms and invest in stronger detection tools.
Siwei Lyu, a computer science professor at the State University of New York at Buffalo who researches AI image forensics, said he had not evaluated Meta's tool but that watermark-based systems have limitations.
"Watermark-based methods can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal — such as cropping, resizing, heavy compression, or editing — may reduce their effectiveness, depending on how the watermark is designed," Lyu said.
Sarah Barrington, an AI researcher and Ph.D. candidate at the UC Berkeley School of Information, said watermarking holds promise for the future of AI-generated content, but could only do so much.
“Like many preventive cybersecurity or physical security measures, it may not be fully watertight, but even if we catch only 90% of cases, that’s still a great leap from 0,” she said.
-Reuters
The finding highlights the challenges of verifying AI-generated images after common alterations, a limitation that could make it harder to identify deepfakes online during a busy election year that includes the U.S. midterms.
In an analysis of 40 images generated using Muse Image, Reuters found the detection tool verified all of the original AI-generated images but failed to verify 55% of the same images after they were cropped to approximately one-third to one-half of their original size.
On its website, Meta says the preview detection tool can identify its own AI-generated images, even if they are cropped, through an invisible watermarking system called Content Seal, which is embedded in every image generated by Muse Image and designed to help users verify whether it was created by Meta's AI models.
When asked about the results of the Reuters analysis of the detection tool, Meta noted that the tool was a preview. The company said the watermark is designed to remain intact after common edits, but that the signal may be lost if an image is heavily cropped.
Rival tech companies Google and OpenAI have cautioned that their own detection tools are not foolproof against image-alteration techniques.
In March, Meta's Oversight Board, a body of experts that makes binding decisions and issues recommendations on content issues across the company's social media platforms, called on the company, opens new tab to do more to address the "proliferation of deceptive AI-generated content" on its platforms and invest in stronger detection tools.
Siwei Lyu, a computer science professor at the State University of New York at Buffalo who researches AI image forensics, said he had not evaluated Meta's tool but that watermark-based systems have limitations.
"Watermark-based methods can be highly effective when the watermark remains intact, but any modification that removes or weakens the embedded signal — such as cropping, resizing, heavy compression, or editing — may reduce their effectiveness, depending on how the watermark is designed," Lyu said.
Sarah Barrington, an AI researcher and Ph.D. candidate at the UC Berkeley School of Information, said watermarking holds promise for the future of AI-generated content, but could only do so much.
“Like many preventive cybersecurity or physical security measures, it may not be fully watertight, but even if we catch only 90% of cases, that’s still a great leap from 0,” she said.
-Reuters
