Researchers have found that training successive generations of generative artificial intelligence models on synthetic data gives rise to self-consuming feedback loops.
You know, this model autophagy disorder may explain what happens to human brains on Internet too. Our brains both poop in the net and eat from it, so regular users go increasingly mad.
The subtitle "Training #AI systems on synthetic data could include negative consequences" is quite the understatement if one looks at the left image. 🤪
Left image shows #LLM deterioration when trained with artificial data.
Right image shows the same when trained with prepared artificial data (see link for details).
Even there 20% of the info gets distorted quickly.
Rice researchers have found that training successive generations of generative artificial intelligence models on synthetic data gives rise to self-consuming feedback loops.
Yes. But this only applies to basic LLM strategies. It doesn’t apply for Self-Taught-Reasoning systems, STaR, which are behind the latest models that have other models first create synthetic higher quality data for training the final models.
Spot on! There has been one piece of “AI”-written BS posted about me each day this year, this is pretty much the present. #Google, naturally, spiders it all, and serves it back out, and itʼs #GIGO.
a great conclusion for those of us who are already readers, or writers, or publishers. TBH los of my time on the internet is sharing things I've read or listened to or want to. So many great online book groups! Supplementary / complementary to the real treasures of books and music! 📚 🎶 🎧
Academy Award-nominated animator/filmmaker Don Hertzfeldt (http://www.bitterfilms.com) answers the question of what it would look like if The Simpsons went f...
This is exactly the scenario I was posting about a few months ago, and I'm glad to see this validated by a cartoonist, usually the most savvy and informed political commentators in our society.
qp @jensorensen Je propose quelque chose : les business angels se mettent à financer très généreusement des artistes pour qu'illes puissent créer du contenu pour entraîner les IA. Et puisque pour avoir de bons modèles, il faut de bonnes données, illes chercheraient vraiment à avoir des artistes originaux et de qualité.
Dans un objectif d'amélioration globale de la filière, et aussi pour être un peu show-off, illes feraient aussi un peu d'effort pour mettre en ligne les artistes en question et favoriser la visibilité de ce travail.
Je suis sûr qu'on doit pouvoir trouver un nom pour ce modèle.
This, exactly. I've been comparing it to photocopying photocopies of photocopies of photocopies, etc. or cloning clones of clones of clones of clones, etc. I like the mad cow disease comparison though. It fits.
Jen Sorensen
in reply to Jen Sorensen • • •Breaking MAD: Generative AI could break the internet, researchers find
ScienceDailyFritz Adalis
in reply to Jen Sorensen • • •Phosphenes
in reply to Jen Sorensen • • •You know, this model autophagy disorder may explain what happens to human brains on Internet too. Our brains both poop in the net and eat from it, so regular users go increasingly mad.
Love the extreme mutant cow in panel 3.
Sepia Fan
in reply to Jen Sorensen • • •Thank you.
I ended up at the University page:
"Self-Consuming Generative Models Go MAD"
news.rice.edu/news/2024/breaki…
The subtitle "Training #AI systems on synthetic data could include negative consequences" is quite the understatement if one looks at the left image. 🤪
Left image shows #LLM deterioration when trained with artificial data.
Right image shows the same when trained with prepared artificial data (see link for details).
Even there 20% of the info gets distorted quickly.
Breaking MAD: Generative AI could break the internet
Rice News | News and Media Relations | Rice Universitym0xEE
in reply to Jen Sorensen • • •Roy Brander🇨🇦
in reply to Jen Sorensen • • •AI6YR Ben
in reply to Jen Sorensen • • •acb
in reply to Jen Sorensen • • •mikeTesteLinux
in reply to Jen Sorensen • • •Meercat ✅
in reply to Jen Sorensen • • •Lazarou Monkey Terror 🚀💙🌈
in reply to Jen Sorensen • • •DJ [REDACTED] 🇨🇦🇪🇺🇺🇦
in reply to Jen Sorensen • • •Jukka Niiranen
in reply to Jen Sorensen • • •Jen Sorensen
in reply to Jukka Niiranen • • •Jukka Niiranen
in reply to Jen Sorensen • • •Jen Sorensen
Unknown parent • • •justsethallcaps
in reply to Jen Sorensen • • •Toni Aittoniemi
in reply to Jen Sorensen • • •Steve
in reply to Jen Sorensen • • •Joacim Jacobsson
in reply to Jen Sorensen • • •Jack Yan (甄爵恩)
in reply to Jen Sorensen • • •Dźwiedziu
in reply to Jen Sorensen • • •Jen Sorensen
Unknown parent • • •patcanfield
in reply to Jen Sorensen • • •Kay
in reply to Jen Sorensen • • •📚 🎶 🎧
Dzso
in reply to Jen Sorensen • • •Brent Guernsey, Artrocity Studio
in reply to Jen Sorensen • • •Emery125
in reply to Jen Sorensen • • •Wade McGillis
in reply to Jen Sorensen • • •big cow in frame 3 reminds me of Don Hertzfeldt's Simpsons intro
youtube.com/watch?v=m78gYyTrG7…
The Simpsons | Travel Into The Future Couch Gag
YouTubeThe One
in reply to Jen Sorensen • • •Jen Sorensen
in reply to The One • • •Luna chan
in reply to Jen Sorensen • • •Will
in reply to Jen Sorensen • • •Jean Sans Peur
in reply to Jen Sorensen • • •qp @jensorensen Je propose quelque chose : les business angels se mettent à financer très généreusement des artistes pour qu'illes puissent créer du contenu pour entraîner les IA. Et puisque pour avoir de bons modèles, il faut de bonnes données, illes chercheraient vraiment à avoir des artistes originaux et de qualité.
Dans un objectif d'amélioration globale de la filière, et aussi pour être un peu show-off, illes feraient aussi un peu d'effort pour mettre en ligne les artistes en question et favoriser la visibilité de ce travail.
Je suis sûr qu'on doit pouvoir trouver un nom pour ce modèle.
Rainne
in reply to Jen Sorensen • • •Tichodrome Colvert
in reply to Jen Sorensen • • •vxo
in reply to Jen Sorensen • • •Court Cantrell prefers not to
in reply to Jen Sorensen • • •