Task-Aware Delegation Cues for LLM Agents
2026-03-11 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors focus on improving teamwork between humans and AI language agents by making the AI's strengths and uncertainties clear to users. They create a system that uses past evaluations of AI responses to identify tasks and show users which AI is best suited for each task, along with how confident the AI is. This system helps people choose when to rely on the AI, see explanations for AI choices, and keep privacy-safe records of interactions. Their tests show that recognizing task types helps predict AI performance better, making human-AI collaboration more transparent and trustworthy.
LLM agentshuman-agent collaborationtask taxonomypreference evaluationcapability profilescoordination-risk cuesdelegation protocolsemantic clusteringclosed-loop systemprivacy-preserving accountability
Authors
Xingrui Gu
Abstract
LLM agents increasingly present as conversational collaborators, yet human--agent teamwork remains brittle due to information asymmetry: users lack task-specific reliability cues, and agents rarely surface calibrated uncertainty or rationale. We propose a task-aware collaboration signaling layer that turns offline preference evaluations into online, user-facing primitives for delegation. Using Chatbot Arena pairwise comparisons, we induce an interpretable task taxonomy via semantic clustering, then derive (i) Capability Profiles as task-conditioned win-rate maps and (ii) Coordination-Risk Cues as task-conditioned disagreement (tie-rate) priors. These signals drive a closed-loop delegation protocol that supports common-ground verification, adaptive routing (primary vs.\ primary+auditor), explicit rationale disclosure, and privacy-preserving accountability logs. Two predictive probes validate that task typing carries actionable structure: cluster features improve winner prediction accuracy and reduce difficulty prediction error under stratified 5-fold cross-validation. Overall, our framework reframes delegation from an opaque system default into a visible, negotiable, and auditable collaborative decision, providing a principled design space for adaptive human--agent collaboration grounded in mutual awareness and shared accountability.