I don’t know if this is an acceptable format for a submission here, but here it goes anyway:
Wikimedia Foundation has been developing an LLM that would produce simplified Wikipedia article summaries, as described here: https://www.mediawiki.org/wiki/Reading/Web/Content_Discovery_Experiments/Simple_Article_Summaries
We would like to provide article summaries, which would simplify the content of the articles. This will make content more readable and accessible, and thus easier to discover and learn from. This part of the project focuses only on displaying the summaries. A future experiment will study ways of editing and adjusting this content.
Currently, much of the encyclopedic quality content is long-form and thus difficult to parse quickly. In addition, it is written at a reading level much higher than that of the average adult. Projects that simplify content, such as Simple English Wikipedia or Basque Txikipedia, are designed to address some of these issues. They do this by having editors manually create simpler versions of articles. However, these projects have so far had very limited success - they are only available in a few languages and have been difficult to scale. In addition, they ask editors to rewrite content that they have already written. This can feel very repetitive.
In our previous research (Content Simplification), we have identified two needs:
- The need for readers to quickly get an overview of a given article or page
- The need for this overview to be written in language the reader can understand
Etc., you should check the full text yourself. There’s a brief video showing how it might look: https://www.youtube.com/watch?v=DC8JB7q7SZc
This hasn’t been met with warm reactions, the comments on the respective talk page have questioned the purposefulness of the tool (shouldn’t the introductory paragraphs do the same job already?), and some other complaints have been provided as well:
Taking a quote from the page for the usability study:
“Most readers in the US can comfortably read at a grade 5 level,[CN] yet most Wikipedia articles are written in language that requires a grade 9 or higher reading level.”
Also stated on the same page, the study only had 8 participants, most of which did not speak English as their first language. AI skepticism was low among them, with one even mentioning they ‘use AI for everything’. I sincerely doubt this is a representative sample and the fact this project is still going while being based on such shoddy data is shocking to me. Especially considering that the current Qualtrics survey seems to be more about how to best implement such a feature as opposed to the question of whether or not it should be implemented in the first place. I don’t think AI-generated content has a place on Wikipedia. The Morrison Man (talk) 23:19, 3 June 2025 (UTC)
The survey the user mentions is this one: https://wikimedia.qualtrics.com/jfe/form/SV_1XiNLmcNJxPeMqq and true enough it pretty much takes for granted that the summaries will be added, there’s no judgment of their actual quality, and they’re only asking for people’s feedback on how they should be presented. I filled it out and couldn’t even find the space to say that e.g. the summary they show is written almost insultingly, like it’s meant for particularly dumb children, and I couldn’t even tell whether it is accurate because they just scroll around in the video.
Very extensive discussion is going on at the Village Pump (en.wiki).
The comments are also overwhelmingly negative, some of them pointing out that the summary doesn’t summarise the article properly (“Perhaps the AI is hallucinating, or perhaps it’s drawing from other sources like any widespread llm. What it definitely doesn’t seem to be doing is taking existing article text and simplifying it.” - user CMD). A few comments acknowlegde potential benefits of the summaries, though with a significantly different approach to using them:
I’m glad that WMF is thinking about a solution of a key problem on Wikipedia: most of our technical articles are way too difficult. My experience with AI summaries on Wikiwand is that it is useful, but too often produces misinformation not present in the article it “summarises”. Any information shown to readers should be greenlit by editors in advance, for each individual article. Maybe we can use it as inspiration for writing articles appropriate for our broad audience. —Femke 🐦 (talk) 16:30, 3 June 2025 (UTC)
One of the reasons many prefer chatGPT to Wikipedia is that too large a share of our technical articles are way way too difficult for the intended audience. And we need those readers, so they can become future editors. Ideally, we would fix this ourselves, but my impression is that we usually make articles more difficult, not easier, when they go through GAN and FAC. As a second-best solution, we might try this as long as we have good safeguards in place. —Femke 🐦 (talk) 18:32, 3 June 2025 (UTC)
Finally, some comments are problematising the whole situation with WMF working behind the actual wikis’ backs:
This is a prime reason I tried to formulate my statement on WP:VPWMF#Statement proposed by berchanhimez requesting that we be informed “early and often” of new developments. We shouldn’t be finding out about this a week or two before a test, and we should have the opportunity to inform the WMF if we would approve such a test before they put their effort into making one happen. I think this is a clear example of needing to make a statement like that to the WMF that we do not approve of things being developed in virtual secret (having to go to Meta or MediaWikiWiki to find out about them) and we want to be informed sooner rather than later. I invite anyone who shares concerns over the timeline of this to review my (and others’) statements there and contribute to them if they feel so inclined. I know the wording of mine is quite long and probably less than ideal - I have no problem if others make edits to the wording or flow of it to improve it.
Oh, and to be blunt, I do not support testing this publicly without significantly more editor input from the local wikis involved - whether that’s an opt-in logged-in test for people who want it, or what. Regards, -bɜ:ʳkənhɪmez | me | talk to me! 22:55, 3 June 2025 (UTC)
Again, I recommend reading the whole discussion yourself.
EDIT: WMF has announced they’re putting this on hold after the negative reaction from the editors’ community. (“we’ll pause the launch of the experiment so that we can focus on this discussion first and determine next steps together”)
If people use AI to summarize passages of written words to be simpler for those with poor reading skills to be able to more easily comprehend the words, then how are those readers going to improve their poor reading skills?
Dumbing things down with AI isn’t going to make people smarter I bet. This seems like accelerating into Idiocracy
[…] then how are those readers going to improve their poor reading skills?
By becoming interested in improving their poor reading skills. You won’t make people become interested in that by having everything available only in complex language, it’s just going to make them skip over your content. Otherwise there shouldn’t be people with poor reading skills, since complex language is already everywhere in life.
Nope. Reading skills are improved by being challenged by complex language, and the effort required to learn new words to comprehend it. If the reader is interested in the content, they aren’t going to skip it. Dumbing things down only leads to dumbing things down.
For example, look at all the iPad kids who can’t use a computer for shit. Kids who grew up with computers HAD to learn the more complex interface of computers to be able to do the cool things they wanted to do on the computer. Now they don’t because they don’t have to. Therefore if you get everything dumbed down to 5th Grade reading level, that’s where the common denominator will settle. Overcoming that apathy requires a challenge to be a barrier to entry.
If the reader is interested in the content, they aren’t going to skip it.
But they aren’t interested in the content because of the complexity. You may wish that humans work like you describe, but we literally see that they don’t.
What you can do is provide a simplified summary to make people interested, so they’re willing to engage with the more complex language to get deeper knowledge around the topic.
For example, look at all the iPad kids who can’t use a computer for shit. Kids who grew up with computers HAD to learn the more complex interface of computers to be able to do the cool things they wanted to do on the computer.
You’re underestimating how many people before the iPad generation also can’t use computers because they never developed an interest to engage with the complexity.
Wikipedia is not made to teach people how to read, it is meant to share knowledge. For me, they could even make Wikipedia version with hieroglyphics if that would make understanding content easier
Novels are also not made to teach people how to read, but reading them does help the reader practice their reading skills. Beside that point, Wikipedia is not hard to understand in the first place.
Sorry, but that’s absolutely wrong - the complexity of articles can vary wildly. Many are easily understandable, while many others are not understandable without a lot of prerequisite knowledge in the domain (e.g. mathematics stuff).
I am not a native speaker, but my knowledge of the english language is better than most people i know, having no issues reading scientific papers and similar complex documents. Some wikipedia article intros, especially in the mathematics, are not comprehensible for anyone but mathematicians, and therefore fail the objective to give the average person an overview of the material.
It’s fine for me if i am not able to grasp the details of the article because of missing prerequisite knowledge (and i know how to work with integrals and complex numbers!), but the intro should at least not leave me wondering what the article is about.
Do you give toddlers post-grad books to read too? This is such an idiotic slippery slope fallacy that it just reeks of white people privilege.
TIL: Wikipedia uses complex language.
It might just be me, but I find articles written on Wikipedia much more easier to read than shit sometimes people write or speak to me. Sometimes it is incomprehensible garbage, or without much sense.
You’ve clearly never tried to use Wikipedia to help with your math homework
I never did any homework unless absolutely necessary.
Now I understand that I should have done it, because I am not good at learning shit in classrooms where there is bunch of people who distract me and I don’t learn anything that way. Only many years later I found out that for most things it’s best for me to study alone.
That said, you are most probably right, because I have opened some math-related Wikipedia articles at some point, and they were pretty incomprehensible to me.
I’m from a country where English isn’t the primary language, people tend to find many aspects of English complex
I am also from a country that English is not widely spoken, in fact most people are not able to make a simple conversation (they will tell you they know ““basic English”” though).
I still find it easier to read Wikipedia articles in English, than than understand some relatives, because they never precisely say what the fuck they want from me. One person even say such incomprehensible shit, that I am thinking their brain is barely functional.
These LLM-page-summaries need to be contained and linked, completely separately, in something like llm.wikipedia.org or ai.wikipedia.org.
In a possible future case, that a few LLM hallucinations have been uncovered in these summaries, it would cast doubts about the accuracy of all page content in the project.
Keep the generated-summaries visibly distinct from user created content.
Is this the same WiliMedia Foundation who was complaining about AI scrapers in April?
IIRC, they weren’t trying to stop them—they were trying to get the scrapers to pull the content in a more efficient format that would reduce the overhead on their web servers.
You can literally just download all of Wikipedia in one go from one URL. They would rather people just do that instead of crawling their entire website because that puts a huge load on their servers.
I’m ok with auto generated content, but only if it is clearly separated from human generated content, can be disabled at any time and writing main articles with AI is forbidden
Wikipedia articles are already quite simplified down overviews for most topics. I really don’t like the direction of the world where people are reading summaries of summaries and mistaking that for knowledge. The only time I have ever found AI summaries useful is for complex legal documents and low-importance articles where it is clear the author’s main goal was SEO rather than concise and clear information transfer.
Never thought I’d cancel my recurring donation for them, but just sent the email. I hope they change their mind on this, but as I told them, I will not support this.
I do have concerns about this but it’s really all about the usage, not the AI itself. Would the AI version be the only version allowed? Would the summaries get created on the fly for every visitor? Would edits to an AI summary be allowed? Would this get applied to and alter existing summaries?
I’m totally fine with LLMs and AI as a stop-gap for missing info or a way to coach or critique a human-written summary, but generally I haven’t seen good results when AI is allowed to do its thing without a human reviewing or guiding the outputs.
It’s kind of indirectly related, but adding a query parameter
udm=14to the url of your Google searches removes the AI summary at the top, and there are plugins for Firefox that do this for you. My hopes for this WM project are that similar plugins will be possible for Wikipedia.The annoying thing about these summaries is that even for someone who cares about the truth, and gathering actual information, rather than the fancy autocomplete word salad that LLMs generate, it is easy to “fall for it” and end up reading the LLM summary. Usually I catch myself, but I often end up wasting some time reading the summary. Recently the non-information was so egregiously wrong (it called a certain city in Israel non-apartheid), that I ended up installing the udm 14 plugin.
In general, I think the only use cases for fancy autocomplete are where you have a way to verify the answer. For example, if you need to write an email and can’t quite find the words, if an LLM generates something, you will be able to tell whether it conveys what you’re trying to say by reading it. Or in case of writing code, if you’ve written a bunch of tests beforehand expressing what the code needs to do, you can run those on the code the LLM generates and see if it works (if there’s a Dijkstra quote that comes to your mind reading this: high five, I’m thinking the same thing).
I think it can be argued that Wikipedia articles satisfy this criterion. All you need to do to verify the summary is read the article. Will people do this? I can only speak for myself, and I know that, despite my best intentions, sometimes I won’t. If that’s anything to go by, I think these summaries will make the world a worse place.
Honestly, I think it’s a good idea. As long as it’s clearly highlighted that “this is an AI generated summary”, it could be very useful. I feel like a lot of people here have never tried to e.g. read a maths article without having a PHD in mathematics. I would often find myself trying to remember what a term means or how it works in practice, only to be met by a giant article going into extreme technical detail that I for the life of me cannot understand, but if I were to ask ChatGPT to explain it I would immediately get it.
People will believe the AI summary without reading the article, and AI hallucinates constantly. Never trust an output from a LLM
Relax, this is not the doom and gloom some of y’all think this is and that is pretty telling.
Given the degree to which the modern day Wiki mods jump on to every edit and submission like a pack of starved lions, unleashing a computer to just pump out vaguely human-sounding word salad sounds like a bad enough idea on its face.
If the AI is being given priority over the editors and mods, it sounds even worse. All of that human labor, the endless back-and-forth in the Talk sections, arguing over the precise phrasing or the exact validity of sources or the relevancy of newly released information… and we’re going to occlude it with the half-wit remarks of a glorified chatbot?
Woof. Enshittification really coming for us all.
Yeah, the catastrophic comments do take it too far… WMF has already announced they’re putting it on hold, so at the very least there’s a lot of discussion with the editors and additional work that will have to happen before this launches - if it ever launches.
AI threads on lemmy are always such a disappointment.
Its ironic that people put so little thought into understanding this and complain about “ai slop”. The slop was in your heads all along.
To think that more accessibility for a project that is all about sharing information with people to whom information is least accessible is a bad thing is just an incredible lack of awareness.
Its literally the opposite of everything people might hate AI for:
- RAG is very good and accurate these days that doesn’t invent stuff. Especially for short content like wiki articles. I work with RAG almost every day and never seen it hallucinate with big models.
- it’s open and not run a “big scary tech”
- it’s free for all and would save millions of editor hours and allow more accuracy and complexity in the articles themselves.
And to top it all you know this is a lost fight even if you’re right so instead of contributing to steering this societal ship these people cover their ears and “bla bla bla we don’t want it”. It’s so disappointingly irresponsible.
The point is they should be fighting AI, not open the door even an inch to AI on their site. Like so many other endeavors, it only works because the contributors are human. Not corpos, not AI, not marketing. AI kills Wikipedia if they let that slip. Look at StackOverflow, look at Reddit, look at Google search, look at many corporate social media. Dead internet theory is all around us.
Wikipedia is trusted because it’s all human. No other reason
RAG is very good and accurate these days that doesn’t invent stuff.
In the OP I linked a comment showing how the summary presented in the showcase video is not actually very accurate and it definitely does invent some elements that are not present in the article that is being summarised.
And in general the “accessibility” that primarily seems to work by expressing things in imprecise, unscientific or emotionally charged terms could well be more harmful than less immediately accessible but accurate and unambiguous content. You appeal to Wikipedia being “a project that is all about sharing information with people to whom information is least accessible”, but I don’t think this ever was that much of a goal - otherwise the editors would have always worked harder on keeping the articles easily accessible and comprehensible to laymen (in fact I’d say traditional encyclopedias are typically superior to Wikipedia in this regard).
and would save millions of editor hours and allow more accuracy and complexity in the articles themselves.
Sorry but you’re making things up here, not even the developers of the summaries are promising such massive consequences. The summaries weren’t meant to replace any of the usual editing work, they weren’t meant to replace the normal introductory paragraphs or anything else. How would they save these supposed “millions of editor hours” then? In fact, they themselves would have to be managed by the editors as well, so all I see is a bit of additional work.
I don’t think the idea itself is awful, but everyone is so fed up with AI bullshit that any attempt to integrate even an iota of it will be received very poorly, so I’m not sure it’s worth it.
I don’t think it’s everyone either - just a very vocal minority.
ok, just so long as the articles themselves aren’t AI generated.
I’m pro AI but absolutely fucking not.
The use case for AI is to summarize Wiki as an external tool. If Wikipedia starts using AI, it becomes AI eating its own tail.
