:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/72337694/GettyImages_532629665.0.jpg)
On May 22, a fake photo of an explosion at the Pentagon caused chaos online.
Within a matter of minutes of being posted, the realistic-looking image spread on Twitter and other social media networks after being retweeted by some popular accounts. Reporters asked government officials all the way up to the White House press office what was going on.
The photo was quickly determined to be a hoax, likely generated by AI. But in the short amount of time it circulated, the fake image had a real impact and even briefly moved financial markets.
This isn’t an entirely new problem. Online misinformation has existed since the dawn of the internet, and crudely photoshopped images fooled people long before generative AI became mainstream. But recently, tools like ChatGPT, DALL-E, Midjourney, and even new AI feature updates to Photoshop have supercharged the issue by making it easier and cheaper to create hyperrealistic fake images, video, and text, at scale. Experts say we can expect to see more fake images like the Pentagon one, especially when they can cause political disruption.
One report by Europol, the European Union’s law enforcement agency, predicted that as much as 90 percent of content on the internet could be created or edited by AI by 2026. Already, spammy news sites seemingly generated entirely by AI are popping up. The anti-misinformation platform NewsGuard started tracking such sites and found nearly three times as many as they did a few weeks prior.
“We already saw what happened in 2016 when we had the first election with a flooding of disinformation,” said Joshua Tucker, a professor and co-director of NYU’s Center for Social Media and Politics. “Now we’re going to see the other end of this equation.”
So what, if anything, should the tech companies that are rapidly developing AI be doing to prevent their tools from being used to bombard the internet with hyperrealistic misinformation?
One novel approach — that some experts say could actually work — is to use metadata, watermarks, and other technical systems to distinguish fake from real. Companies like Google, Adobe, and Microsoft are all supporting some form of labeling of AI in their products. Google, for example, said at its recent I/O conference that, in the coming months, it will attach a written disclosure, similar to a copyright notice, underneath AI-generated results on Google Images. OpenAI’s popular image generation technology DALL-E already adds a colorful stripe watermark to the bottom of all images it creates.
“We all have a fundamental right to establish a common objective reality,” said Andy Parsons, senior director of Adobe’s content authenticity initiative group. “And that starts with knowing what something is and, in cases where it makes sense, who made it or where it came from.”
In order to reduce confusion between fake and real images, the content authenticity initiative group developed a tool Adobe is now using called content credentials that tracks when images are edited by AI. The company describes it as a nutrition label: information for digital content that stays with the file wherever it’s published or stored. For example, Photoshop’s latest feature, Generative Fill, uses AI to quickly create new content in an existing image, and content credentials can keep track of those changes.
AI-labeling tools like Adobe’s are still in their early stages, and by no means should they be considered a silver bullet to the problem of misinformation. It’s technically possible to manipulate a watermark or metadata. Plus, not every AI generation system will want to disclose that it’s made that way. And as we’ve learned with the rise of online conspiracy theories in recent years, people will often ignore facts in favor of believing falsehoods that confirm their personal beliefs. But if implemented well — and especially if these labels are seen as more neutral than traditional social media fact-checking — AI disclosures could be one of our only hopes for navigating the increasingly blurry distinction between fake and real media online.
Here is how some of these early AI markup systems could work, what the limitations are, and what users can do to navigate our confusing post-truth internet reality in the meantime.
The devil is in the metadata
When you look at an image on social media or a search engine today, odds are you don’t know where the photo came from — let alone if it was created by AI. But underneath the hood, there’s often a form of metadata, or information associated with the digital image file, that tells you basic details, like when and where the photo was taken. Some tech companies are now starting to add specific metadata about AI to their products at the moment of creation, and they’re making that information more public in an effort to help users determine the authenticity of what they’re looking at.
Google recently said it will start marking up images made by its own new AI systems in the original image files. And when you see an image in Google Search that’s made by Google’s AI systems, it will say something like “AI-generated with Google” underneath the image. Going a step further, the company announced it’s partnering with publishers like Midjourney and stock photography site Shutterstock to let them self-tag their images as AI-generated in Google Search. This way, if you come across a Midjourney image in Google Search, it will say something like “Image self-labeled as AI-generated”
Google Search public liaison Danny Sullivan said that this kind of AI labeling is part of a broader effort to give people more context about images they’re seeing.
”If we can show you a helpful label, we’re going to want to do that,” said Sullivan, “but we’re also going to want to try to give you background information that we can determine independent of the label.”
:format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/24697608/AI_Label.jpg)
That’s why Google is also adding an “About this image” feature next to image search results — whether they are AI labeled or not — that you can click and see when the image was first indexed by Google, where it may have first appeared, and where else it’s been seen online. The idea is, if you searched for, say, “Pentagon explosion” and saw a bunch of images in the results, you would be able to see a fact-checked news article debunking the piece.
“These tools are really designed to help people understand information literacy more and bake it into the search product itself,” said Sullivan.
Other major industry players have also been working on the issue of how to label AI-generated content. In 2021, a group of major companies including Microsoft, Adobe, the BBC, and Intel created a coalition called the C2PA. The group is tasked with helping to create an interoperable open standard for companies to share the provenance, or history of ownership, of a piece of media. C2PA created its first open standard last January, and since then, Adobe and Microsoft have released features using that standard.
For example, if you’re a photographer at a news outlet, you can mark when a specific picture was taken, who took it, and have that be digitally signed by your publisher. Later, your editor could make changes to the photo, signing it again with a seal of authenticity that it’s been verified by the C2PA standard. This way, you know that the photo was taken by a person — not generated by AI— and know who has made edits to it and when. The system uses cryptography to preserve the privacy of sensitive information.
“Now you can read the entire lineage of the history of a piece of digital content,” said Mounir Ibrahim, EVP of public affairs and impact at Truepic, a visual authenticity app that is a member of C2PA. “The purpose of us is to help content consumers ... decipher the difference between synthetic and authentic.”
Knowing the history and provenance of an image could potentially help users verify the legitimacy of anything from a headshot on a dating app to a breaking news photo. But for this to work, companies need to adopt the standard.
Right now, it’s up to companies to adopt the C2PA standard and label verified content as they wish. The organization is also discussing potentially standardizing the look of the C2PA content credential when it shows up on images, Ibrahim said. In the future, the C2PA credential could be similar to the little padlock icon next to the URL in your browser window that signifies your connection is secure. When you see the proposed C2PA icon, you would know that the image you’re seeing has had its origins verified.
So far, two big C2PA members, Adobe and Microsoft, have announced tools that integrate C2PA standards into their products to mark up AI-generated content. Microsoft is labeling all AI-generated content in Bing Image Generator and Microsoft Designer, and Adobe is using C2PA standards in its new AI Firefly product’s content credentials.
“The biggest challenge is we need more platforms to adopt this,” said Ibrahim.
While the C2PA-style metadata labels work behind the scenes, another approach is for AI systems to add visible watermarks, as OpenAI has done with the rainbow bar at the bottom of DALL-E images. The company says it’s also working on a version of watermarking for its text app, ChatGPT. The challenge with watermarks, though, is that they can be removed. A quick Google search turns up forms of people discussing how to circumvent the imprint.
Another imperfect option is technology that can detect AI-generated content after the fact. In January, OpenAI released a tool that lets you cross-check a block of text to determine whether it’s likely written by AI. The problem, though, is that by OpenAI’s own assessment, the tool is not fully reliable. It correctly identified only 26 percent of AI-written texts in OpenAI’s evaluations, although it’s notably more accurate with longer than shorter text.
“We don’t want any of our models to be used for misleading purposes anywhere,” said a spokesperson for OpenAI in a statement. “Our usage policies also require automated systems, including conversational AI and chatbots, to disclose to users that they are interacting with our models.”
At the end of the day, even if these early AI flagging and identification systems are flawed, they’re a first step.
What comes next
It’s still early days for tech platforms trying to automate the identification of AI-generated content. Until they identify a dependable solution, however, fact-checkers are left manually filling in the gaps, debunking images like the Pope in a puffy jacket or fake audio of politicians.
Sam Gregory, executive director of human rights and civic journalism network Witness, who works with fact-checkers largely outside of the US, said that while he thinks technical solutions to AI identification like watermarking are promising, many fact-checkers are worried about the onslaught of misinformation that could come their way with AI in the meantime. Already, many professional fact-checkers are dealing with far more content to check than humanly possible.
“Is an individual going to be blamed because they couldn’t identify an AI-generated image? Or is a fact-checker going to be the one to take the strain because they’re overwhelmed by this volume?” said Gregory. The responsibility to address AI misinformation “needs to lie on the people who are designing these tools, building these models, and distributing them,” he added.
In many cases, Gregory says, it’s unclear exactly what social media platforms’ rules are about allowing AI-generated content.
TikTok has one of the more updated policies around “synthetic media,” or media that is created or manipulated by AI. The policy, which was revised in March 2023, allows synthetic media but requires that, if it shows realistic scenes, the image must be clearly disclosed with a caption, sticker, or otherwise. The company also doesn’t allow synthetic media that contains the likeness of any private figure or anyone under 18. TikTok says it worked with outside partners like the industry nonprofit Partnership on AI for feedback on adhering to a framework for responsible AI practices.
“While we are excited by the creative opportunities that AI opens up for creators, we are also firmly committed to developing guardrails, such as policies, for its safe and transparent use,” a TikTok spokesperson said in a statement. “Like most of our industry, we continue to work with experts, monitor the progression of this technology, and evolve our approach.”
But many other platforms have policies that might need some updating. Meta, which owns Facebook and Instagram, and YouTube both have general rules against manipulated media that misleads users, but those could be clarified regarding what uses are acceptable or not, according to Gregory. Meta’s fact-checking policies state that manipulated media containing misinformation is eligible for fact-checking by its third-party partners, as it did with the fake Pentagon AI explosion claims.
“AI is bigger than any single person, company, or country, and requires cooperation between all relevant stakeholders,” Meta said in a statement. “We are actively monitoring new trends and working to be purposeful and evidence-based in our approach to AI-generated content.”
Technological solutions to help people fact-check content themselves, like AI detection systems and watermarks, couldn’t come sooner.
But NYU’s Tucker says we need to test these solutions to see whether they’re effective in changing people’s minds when they encounter misleading AI content, and what the disclosures need to look to be impactful. For example, if the disclosures that an image or video is AI-generated are too subtle, people could miss it entirely. And sometimes, labels don’t work as expected. For example, Tucker co-authored a study last year showing that high- or low-quality news credibility labels had limited effects on people’s news consumption habits and failed to change people’s perceptions.
Still, there’s hope that if AI disclosures are seen not as politicized fact-checks but as neutral context about the origins of an image, they could be more effective. To know whether these labels are resonating with people and changing their minds will require more research.
There is an urgency to figure out these problems as AI-generated content floods the internet. In the past, tech companies had time to debate the hypothetical risks of AI misinformation because mainstream generative AI products weren’t yet out in the wild. But those threats are now very real.
These new tools that label AI-generated content, while far from perfect, could help mitigate some of that risk. Let’s hope tech companies move forward with the necessary speed to fix problems that come with AI as quickly as they’re being created.
