This is a good overview of recent AI-assisted persuasion research. It's notable that even when people are talking to robots, learning new facts seems more persuasive than exposure to various framing or priming strategies.
I read these a bit ago just after reading a survey paper about AI generating responses to particular types of contentious requests. It’s an interesting emerging space and I agree the factuality piece was encouraging. It also means that politicization and ref-working is going to accelerate with a vengeance because what the model decides is “evidence” is informed in part by what it discerns as “reliable sources”.
Yes, interesting to try and work out what the lines of conflicts will in the epistemic infrastructure., the stuff behind the scenes which informs the models. Wikipedia edits? Astroturf websites? Billionaires buying newspapers (still)?
I agree with most of your argument but I'm skeptical of your optimistic take on people learning to mistrust bad sources. Although I think there's are very real gradual trust learning and rapid trust unlearning effects I suggest the trust unlearning is often counteracted by certain powerful kinds of motivated reasoning. For example when we learn to simplistically divide sources into good and bad and then fear the "bad" sources so much that we trust the "good" ones by default even when they say things we might with further reflection find implausible. I suspect that even a badly corrupted generative AI will continue to be persuasive if it seems to people to be defending the "good" and tilting against the "bad."
Would it be fair to say that the disagreement is about *how much* trust in bad models would be sustained, rather than whether it would or not? Like, I think it would go down significantly, you think it might not, so the remaining questions is just how we could get an estimate of how much unreliability would feed into loss of trust?
Yes I think that’s fair. I’m sure I’m skewed a bit by my experience in recent years watching people I consider reasonable and intelligent leaning into what seem to me to be less reasonable (e.g. delegitimizing) trajectories seemingly out of fear of the bad guys.
This is a good overview of recent AI-assisted persuasion research. It's notable that even when people are talking to robots, learning new facts seems more persuasive than exposure to various framing or priming strategies.
I read these a bit ago just after reading a survey paper about AI generating responses to particular types of contentious requests. It’s an interesting emerging space and I agree the factuality piece was encouraging. It also means that politicization and ref-working is going to accelerate with a vengeance because what the model decides is “evidence” is informed in part by what it discerns as “reliable sources”.
Yes, interesting to try and work out what the lines of conflicts will in the epistemic infrastructure., the stuff behind the scenes which informs the models. Wikipedia edits? Astroturf websites? Billionaires buying newspapers (still)?
I agree with most of your argument but I'm skeptical of your optimistic take on people learning to mistrust bad sources. Although I think there's are very real gradual trust learning and rapid trust unlearning effects I suggest the trust unlearning is often counteracted by certain powerful kinds of motivated reasoning. For example when we learn to simplistically divide sources into good and bad and then fear the "bad" sources so much that we trust the "good" ones by default even when they say things we might with further reflection find implausible. I suspect that even a badly corrupted generative AI will continue to be persuasive if it seems to people to be defending the "good" and tilting against the "bad."
Would it be fair to say that the disagreement is about *how much* trust in bad models would be sustained, rather than whether it would or not? Like, I think it would go down significantly, you think it might not, so the remaining questions is just how we could get an estimate of how much unreliability would feed into loss of trust?
Yes I think that’s fair. I’m sure I’m skewed a bit by my experience in recent years watching people I consider reasonable and intelligent leaning into what seem to me to be less reasonable (e.g. delegitimizing) trajectories seemingly out of fear of the bad guys.