Data poisoning, or how to poison artificial intelligence in social networks
Digital ecosystems rely on artificial intelligence. It was logical to see tools emerge to manipulate it, notably data poisoning.
You are part of the 1025 subscribers reading this newsletter every Tuesday evening. A huge THANK YOU!
Artificial intelligence is a component of social networks. Machine learning techniques have been used early on to better understand user behaviors (and enhance their experience on the networks). And of course, for advertising, to offer more personalized content, through concatenation processes.
A deep-seated trend is beginning to emerge against artificial intelligences: data poisoning (or 'empoisonnement des données' in proper French).
Data poisoning: What is it?
The French Data Protection Authority (CNIL) provides a fascinating definition of data poisoning:
Poisoning attacks aim to modify the behavior of the Artificial Intelligence system by introducing corrupted data during the training phase (or learning). They assume that the attacker is capable of submitting data to be used during the training of the AI system.
CNIL
The semantics are close to those of the human body, which is another signal of the fusion of tangible and digital worlds.
Tricking platforms is not new. During the heyday of blogs, activists coordinated an attack on a brand's reputation by linking its name to URLs pointing to pages establishing unflattering facts about these organizations. Systems have since become much more sophisticated and are not immune to various manipulations.
OpenAI's announcements couldn't be clearer: the company aims to keep track of all user interactions to continue training its tools. Certainly, this is to optimize the experience with various robots, but it raises the question of controlling our humanities. If colossal amounts of data are already stored on servers, OpenAI opens the door to a conversational layer linked to our most intimate, implicit, and profound memories, which could not only influence our memories but also how they resurface. Historicity - already complicated on the Internet - could be on the path to privatization by companies like OpenAI.
Hypomnema, I hear your name!
In 2007, Christian Fauré proposed a distinction between explicit and implicit social networks.
Explicit social networks serve "a personal project: finding friends, networking, becoming known, etc. It is formed in service of a project and purpose. It is there as a means to an end."
On the contrary, implicit social networks "are formed based on our real activity (...) by mapping our data and activities, these social networks (...) are automatically deduced from our digital activity." OpenAI snatches our hypomnema, that is, all the supports of our memory, and positions itself as a toll between self-writing and articulation within a collective. This appropriation of the space between the implicit and the explicit through this temporal conversational layer will have significant implications. More and more humans believe in pseudo-realities. And as we use ChatGPT & co, we place a certain trust in these tools, giving them even more weight in how we construct ourselves and position ourselves in the world. Especially since ChatGPT acts as a "response engine" rather than a search engine as explained by Meta-Media.
Nightshade: the artists' revenge
Researchers at the University of Chicago have launched a tool called Nightshade. It aims to fight against AI companies that use artists' work to train their models without the creators' permission.
By "poisoning" the training data, Nightshade alters the future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, making some of their outputs useless – dogs become cats, elephants become scooters, etc.
“Moving forward, it is possible poison attacks may have potential value as tools to encourage model trainers and content owners to negotiate a path towards licensed procurement of training data for future models.”
Shawn Shan, Wenxin Ding, Josephine Passananti, Haitao Zheng, Ben Y. Zhao Department of Computer Science, University of Chicago
A real counter-power still in its infancy and not always so virtuous if it falls into the wrong hands. Extremist groups often have more time and desire to manipulate algorithms because their purposes, their objectives are clear, unlike the ordinary citizen who is not necessarily such a committed activist.
The hashtag of the week: #SlowedSong
In a world that sometimes moves too quickly, communities on TikTok are creating more and more content tagged #SlowedSong (songs that have been slowed down, which can give them an interesting and captivating cinematic dimension).
Amazing links
Are we now Technosexuals? Read the New York Times
The Cyber Feminism Index offers exciting resources to address technological and societal topics through an engaged lens (spotted in Mathilde Saliou's excellent newsletter Technoculture)
Have a great week! And feel free to share this newsletter, like, comment, or continue to send me emails: these notifications bring joy.