AI systems that we may rely on for accurate information – even if that information is not something that would make us happy – must have the capacity to ‘push back’ against our protests until we are ready to be convinced of the unwelcome news/facts.
AI is not very good at this; since frontier models are designed to be consumer-facing, they tend towards deference and, in fact, towards sycophancy.
A notable strand of research has emerged to deal with the issue; but recent efforts have concluded that we can easily gaslight AI back into sycophancy if it challenges us; that AI defers to human authority even when it is manifestly wrong; and that the usually effective technique of reinforcement learning actually makes this particular problem worse.
The fact that deference is culturally enmeshed with politeness appears to mean that ‘straight talking’ is impossible in a service>consumer context; the ‘customer is always right’, even when they’re about to walk off a cliff.
Well, I wish I could tell you that this week’s crop of papers has produced one that really moves the needle forward on this issue. Sadly, all we’re getting is more information about how models that do ‘push back’, and attempt to impress the truth on their human masters, still aren’t doing a particularly great job of it* – and that even when they are willing to speak truth to power (that’s you), it’s really more of a muffled whisper.
The new paper in question is a considerable academic...
Former Venezuelan President Nicolás Maduro told a judge that US allegations against him, including new claims that he conspired to torture US citizens detained in his country, were false. Maduro, ...