Overcoming AI's Difficulty with Disagreement

The author explores the challenges of getting AI language models to engage in realistic debates, as they are trained to be agreeable and avoid conflict. The article discusses various approaches tried and the final solution of using separate personas and contexts for each debater.

Details

The author initially tried to build a tool that would generate debate videos between rival brands, but found that the AI language models refused to truly disagree with each other. They would simply respond with polite, agreeable statements. This is because modern language models are heavily trained to be helpful assistants, not adversarial debaters. Attempts to prompt the models to be more aggressive or confrontational only worked temporarily before they reverted to consensus-building. The breakthrough came when the author split the two debaters into completely separate contexts, where each side only saw the other's statement as something 'wrong' that they needed to respond to, rather than an explicit debate framing. This sidestepped the agreeable training. Additionally, giving each side distinct personas, rhetorical styles, and pacing cues helped make the two outputs sound authentically different, rather than just the same AI in two costumes. The author concludes that language models are resistant to behaviors that contradict their training, and that architectural changes, not just prompting, are often needed to achieve desired outputs.