There’s been a lot of discussion about the ethics of generative AI recently. Not only are many people worried about the possibility that they might soon be replaced by AI, but many creators are also unhappy that their work often appears in the vast datasets that are used to train some of the leading AI tools. Given the lack of regulation, this has given rise to a form of digital vigilantism called data poisoning where creators seek to sabotage generative AIs by feeding them tainted material. 

What is AI training?

Generative AI works by making predictions. Let’s say you give it a prompt like “create an image of a horse.” On its own, the AI has no idea what that should look like. But because you asked for a ‘horse,’ it can assume that you want to see a large four-legged animal. And since horses are normally associated with outdoor scenes, the AI shouldn’t put the animal in the middle of the ocean. Of course, the AI needs to be able to make good predictions. It needs to realize that not all four-legged animals are the same, and so it shouldn’t put horns on the horse. 

As Michael Chen of Oracle has pointed out, training AI can be similar to teaching children. If you want your child to understand the distinction between cats and dogs, you might start out by showing them images. Then, you might provide further context by telling them that cats meow while dogs bark. The more information you provide, the easier it becomes for the child to distinguish between the two animals. 

AI training often involves works created by humans

However, developers often train AI using works created by humans. For example, in a recent court case, a judge found that the LAION dataset used to train generative AI tools for Stable Diffusion, Midjourney, and Deviant Art included copyrighted works taken from across the Internet. Moreover, he held that the “way the product operates necessarily invokes copies or protected elements of those works.” Corporate copyright owners also have these developers in their sights with image libraries, music companies, and even newspapers filing suits alleging that their copyright has been infringed by AI training. 

Creators take matters into their own hands: data poisoning

One of the problems with going after AI developers in court is that it can take forever to resolve the dispute. For example, even though Getty Images first sued Stability AI in early 2023, the case is still ongoing due to protracted wrangling over jurisdiction and discovery. 

As a result, some creators are fighting back on their own. One of the ways they’re doing this is by ‘data poisoning.’ The idea is that, by making subtle changes at the pixel level, images can end up having a negative effect on AIs that are trained on them. This can mess up the algorithm, leading the AI to produce highly distorted images that are unlikely to be useful. As T. J. Thomson explains, data poisoning can lead to outcomes like images of balloons that look more like eggs. It can also exacerbate existing problems, such as the difficulty that many generative AIs experience when creating images of human hands, or introduce new problems like dogs with extra legs. This can have a cascading effect. Thomson notes that, if an AI is given a poisoned image of a Ferrari, prompt results for other car brands can be distorted as well. Sympathetic researchers have even created tools like Nightshade that can be used for data poisoning. 

One of the ways to counter data poisoning is by relying on tightly curated collections of training material instead of indiscriminately scraping content from the Internet. This may suggest a workable modus vivendi since it would incentivize developers to exclude content from creators who don’t wish to have their work used for AI training purposes. 

The darker side of data poisoning

It’s hard not to feel sympathetic towards artists who feel they have no choice but to resort to data poisoning in order to prevent their work from non-consensual exploitation. But not everyone has lofty motives. 

In 2016, Microsoft introduced a chatbot called Tay that was supposed to learn from its interactions with humans. But when people bombarded it with a deluge of hateful and vulgar language, it began to mimic their bile. Within 16 hours, Microsoft took Tay offline. 

In 2023, researchers revealed how a tainted AI model called PoisonGPT could be used to spread false material. They did this by modifying a large language model (LLM) to include doctored facts (such as claiming the Eiffel Tower was in Rome). If the compromised model was then uploaded to platforms providing LLM models to developers, the end result could be a cascade of disinformation.

We need rules for the ethical use of AI

There would be less need for creators to turn vigilantes if there were actual rules and regulations governing the development and use of generative AI. As one of Pope Francis’s advisers noted, we need “human-centric AI.” Transparency, inclusion, accountability, impartiality, reliability, security,  and privacy are the tentpoles of this concept, and they are the best way to ensure that AI serves all of humanity in a constructive manner.

Harmony between creators and developers is possible

The ethical implications of generative AI are complex, but one thing remains clear: we need to find a system that doesn’t make people feel like they’re being taken advantage of. Without meaningful regulations, the struggle between creators and AI developers will likely persist, leading to further tension and potential harm. Establishing a framework that emphasizes transparency, accountability, and respect for intellectual property could pave the way for peaceful coexistence instead of constant strife and a world where AI is a tool that augments human creativity rather than undermines it. 

Looking for more posts about AI? Check these out!

“Teaching machines to create: The hidden hazards of AI training”

“Transparency is the key to building trust with AI-generated content”