Watermarking has emerged as one of the most promising ways to combat the escalating AI misinformation problem online by Big Tech. But so far, the results don't seem promising, according to experts and a survey of misinformation conducted by NBC News.
Dana Rao, Adobe's general counsel and fiduciary officer, said in a February blog post that Adobe's digital watermarking standard, C2PA, to which Meta and other Big Tech companies are signatories, misleads the public about AI. I wrote that it would be useful for educating people.
“With more than 2 billion voters expected to participate in elections around the world this year, advancing C2PA's mission is more important than ever,” Rao wrote.
Although these technologies are still in their infancy and adoption is limited, watermarks have already proven to be easily circumvented.
Many modern watermarking technologies aimed at identifying AI-generated media use two components: an invisible tag contained in the image's metadata and a visible label overlaid on the image.
However, both invisible watermarks and visible labels, which take the form of microscopic pixels and metadata, can sometimes be removed using basic methods such as screenshots and cropping.
So far, major social media and technology companies have not strictly mandated or enforced labeling of AI-generated or AI-edited content.
The watermark vulnerability was revealed on Wednesday when Meta CEO Mark Zuckerberg updated his Facebook cover photo with an AI-generated image of a llama standing on a computer. became. It was created with Meta's AI image generator, Imagine, released in December. This generator is supposed to produce an image with an embedded label, which will appear as a small symbol in the bottom left corner of the image, similar to Zuckerberg's llama.
Mark Zuckerberg's AI-generated cover photo has a meta watermark cropped out.Facebook
However, Zuckerberg's AI-generated image of the llama did not display the label to users who logged out of Facebook. It also didn't show up unless you clicked on Zuckerberg's cover photo to open it. When NBC News used Imagine to create his AI-generated llama images, they could easily remove the labels by screenshotting the part of the image that didn't include them. According to Meta, the invisible watermark will also carry over to screenshots.
In February, Meta announced that it would begin using watermarking technology to identify and label AI-generated content on Facebook, Instagram, and Threads. The watermark used by Meta is included in the metadata. Metadata is invisible data that can only be viewed with technology built to extract it. In its announcement, Meta acknowledged that watermarks are not completely effective and can be removed or manipulated by malicious actors.
The company also said it would require users to disclose whether the content they post was generated by AI, and that failure to do so “may result in penalties.” Mehta said these standards will be developed in the coming months.
AI watermarks can be removed even if you do not intend to. When you upload photos online, metadata may be removed in the process.
Visible labels associated with watermarks pose further problems.
“It takes about two seconds to remove that kind of watermark,” says Sophie Tula, who works at Control AI, a British technology lobbying and advocacy firm founded in October 2023. She tends to be flat. ”
The original AI-generated image on the left has a watermark, but I was able to easily crop it to create the image on the right. Generated by Meta's Imagine
Senior technologists at the Electronic Frontier Foundation, a nonprofit organization that advocates for digital civil liberties, wrote that even the most robust and sophisticated watermarks can be removed by someone with the skill and desire to manipulate the files themselves. Masu.
In addition to being removed, watermarks can also be duplicated, creating the potential for false positives that suggest unedited, authentic media was actually generated by AI.
Companies working on collaborative watermarking standards include major companies such as Meta, Google, OpenAI, Microsoft, Adobe, and Midjourney. However, thousands of AI models that are not bound by watermarking standards are available for download and use on app stores such as Google Play and websites such as Microsoft's GitHub.
Adobe's C2PA standard, adopted by Google, Microsoft, Meta, OpenAI, major news organizations, and major camera companies, automatically inserts a watermark into images combined with a visible label called a “content credential.” It is designed to be
The label is a small symbol consisting of the letters “CR” in the corner of the image, similar to Meta's Imagine label. These invisible watermarks are contained in metadata located in pixels in visually significant parts of images, Adobe's Rao told NBC News in February. Both visual labels and metadata include information such as whether the image was generated by AI or edited with an AI tool.
“This is a good intention and a step in the right direction. For example, I don't think we should rely on deepfakes as a solution to all the problems with deepfakes,” Tula said.
Deepfakes are misleading images, videos, or audio that are edited or generated by AI. These are frequently used to target people (overwhelmingly women and girls) with nude or sexually explicit images or videos depicting that person's face or likeness without their consent. . In 2023, many of these deepfakes were posted online in his 2023 and 2024 as well. Earlier this month, NBC News revealed that Mehta had hosted hundreds of ads for a deepfake app since September that offered the ability to “take off your clothes.” ” Photos — The 11 ads featured a blurry nude photo of actress Jenna Ortega taken when she was 16 years old, “undressed.” Mehta, who had previously suspended dozens of ads, only suspended the company behind the ads after being contacted by NBC News.
Deepfakes are also increasingly being used for fraud and political disinformation, including regarding the 2024 election.
In January, a deepfake robocall called thousands of New Hampshire Democrats, using AI to imitate Joe Biden's voice and telling them not to vote in the primary. NBC News reported that a Democratic consultant with ties to rival campaigns paid a magician to create the audio, which was produced using Eleven Labs' AI software.
Eleven Lab embeds a watermark that is inaudible to the human ear into audio files created using its own software. Anyone can upload a sample to the free “audio classifier” and scan it for watermarks.
However, using deepfake audio for malicious purposes in the real world could alter the audio files and remove their watermarks. When NBC News uploaded the magician's original file to the audio classifier, Eleven Labs said there was a 98% chance that its software created the sample. But when NBC News uploaded a recording of a fake Biden call that was identical to the one recorded from the voicemail of the New Hampshire resident who received the call (a process that added some distortion to the audio file), the classifier , said that the proportion is only 2%. Eleven Labs software may be involved.
Social media platforms and search engines are already full of deepfakes, and app stores are full of services promoting deepfake creations. Some of these posts and ads include deepfake nude images of children's faces and sexually explicit images.
Rao was realistic about the impact of Adobe's own watermarking efforts. First, he said, the public needs to recognize the labels that indicate AI-generated content. To be widely effective, the public needs to learn how to verify visual media before trusting them. This would be a huge feat. Mr. Rao compared potential changes to expectations and perceptions of content credentials in visual media to public awareness of online phishing campaigns. Meanwhile, online phishing campaigns have skyrocketed with the rise of ChatGPT.
“You don't have to believe everything is right,” he said in an interview with NBC News in February. “It's really important that we need to work harder to believe that it's true.”
