Be Fair! Can Machine Learning Engineering Agents Adhere to Fairness Constraints?
2026-06-03 • Machine Learning
Machine LearningDatabases
AI summaryⓘ
The authors study machine learning engineering (MLE) agents, which are tools that can build ML pipelines automatically from simple instructions. They highlight a problem where users may not see important design decisions, leading to issues with fairness and correctness, especially in sensitive areas like medicine. By testing these agents on melanoma detection with fairness across skin tones as a focus, they found the automated pipelines were less accurate and fair compared to those made by humans. The authors suggest improvements are needed to let people better guide and check these automated ML processes.
machine learning engineeringMLE agentspipeline automationfairness in MLmelanoma classificationpredictive qualityresponsibility in AIregulatory compliancehuman-in-the-loopbias mitigation
Authors
Anna Richter, Julia Stoyanovich, Sebastian Schelter
Abstract
Machine learning engineering (MLE) agents promise to automate end-to-end ML pipeline development from raw data and natural language instructions, potentially making ML accessible to non-technical domain experts. However, in sensitive and regulated domains, this abstraction creates a responsibility gap: end-users may lack visibility into design choices that affect correctness, robustness, fairness, and regulatory compliance. We argue that existing benchmarks are insufficient to assess whether MLE agents can be safely applied in such settings. We propose desiderata for a responsibility-centered evaluation framework and conduct an exploratory study on melanoma classification, focusing on fairness across skin tones as a responsibility constraint. When evaluating two recent MLE agents, we find that agent-generated pipelines show high variance and consistently underperform manually designed baselines in both predictive quality and fairness, despite fairness-oriented prompts. These preliminary results suggest that further research is needed towards redesigning MLE agents to allow humans to guide the search process and reliably assess the compliance and quality of the generated ML pipelines.