Multi-Round Human-AI Collaboration with User-Specified Requirements
2026-02-19 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on making AI assistants in multi-step conversations more reliable for important decisions by following two main ideas: the AI should not harm human strengths (counterfactual harm) and should help where humans usually make mistakes (complementarity). They let users define exactly what these ideas mean for their tasks and create an algorithm that respects these rules over time. Testing this on AI and human tasks showed their method keeps the AI helpful without causing harm, and adjusting the rules changes how accurate decisions become. This shows their approach can guide better teamwork between humans and AI without needing to predict human behavior.
Conversational AICounterfactual HarmComplementarityMulti-round CollaborationOnline AlgorithmFinite Sample GuaranteesNonstationary DynamicsHuman-AI InteractionMedical DiagnosisPictorial Reasoning
Authors
Sima Noorani, Shayan Kiyani, Hamed Hassani, George Pappas
Abstract
As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.