To be fair, though, this experiment was stupid as all fuck. It was run on /r/changemyview to see if users would recognize that the comments were created by bots. The study’s authors conclude that the users didn’t recognize this. [EDIT: To clarify, the study was seeing if it could persuade the OP, but they did this in a subreddit where you aren’t allowed to call out AI. If an LLM bot gets called out as such, its persuasiveness inherently falls off a cliff.]
Except, you know, Rule 3 of commenting in that subreddit is: “Refrain from accusing OP or anyone else of being unwilling to change their view, of using ChatGPT or other AI to generate text, [emphasis not even mine] or of arguing in bad faith.”
It’s like creating a poll to find out if women in Afghanistan are okay with having their rights taken away but making sure participants have to fill it out under the supervision of Hibatullah Akhundzada. “Obviously these are all brainwashed sheep who love the regime”, happily concludes the dumbest pollster in history.
It’s like creating a poll to find out if women in Afghanistan are okay with having their rights taken away but making sure participants have to fill it out under the supervision of Hibatullah Akhundzada. “Obviously these are all brainwashed sheep who love the regime”, happily concludes the dumbest pollster in history.
I don’t particularly like this analogy, because /r/changemyview isn’t operating in a country where an occupying army was bombing weddings a few years earlier.
But this goes back to the problem at hand. People have their priors (my bots are so sick nasty that nobody can detect them / my liberal government was so woke and cool that nobody could possibly fail to love it) and then build their biases up around them like armor (any coordinated effort to expose my bots is cheating! / anyone who prefers the new government must be brainwashed!)
And the Bayesian Reasoning model fixates on the notion that there are only ever a discrete predefined series of choices and uniform biases that the participant must navigate within. No real room for nuance or relativism.
To be fair, though, this experiment was stupid as all fuck. It was run on /r/changemyview to see if users would recognize that the comments were created by bots. The study’s authors conclude that the users didn’t recognize this. [EDIT: To clarify, the study was seeing if it could persuade the OP, but they did this in a subreddit where you aren’t allowed to call out AI. If an LLM bot gets called out as such, its persuasiveness inherently falls off a cliff.]
Except, you know, Rule 3 of commenting in that subreddit is: “Refrain from accusing OP or anyone else of being unwilling to change their view, of using ChatGPT or other AI to generate text, [emphasis not even mine] or of arguing in bad faith.”
It’s like creating a poll to find out if women in Afghanistan are okay with having their rights taken away but making sure participants have to fill it out under the supervision of Hibatullah Akhundzada. “Obviously these are all brainwashed sheep who love the regime”, happily concludes the dumbest pollster in history.
I don’t particularly like this analogy, because /r/changemyview isn’t operating in a country where an occupying army was bombing weddings a few years earlier.
But this goes back to the problem at hand. People have their priors (my bots are so sick nasty that nobody can detect them / my liberal government was so woke and cool that nobody could possibly fail to love it) and then build their biases up around them like armor (any coordinated effort to expose my bots is cheating! / anyone who prefers the new government must be brainwashed!)
And the Bayesian Reasoning model fixates on the notion that there are only ever a discrete predefined series of choices and uniform biases that the participant must navigate within. No real room for nuance or relativism.