AI summaryⓘ
The authors explain that picking the right large language model (LLM) for business use is tricky because models differ in quality, speed, safety, and cost. They introduce a way of continuously checking and managing these models during their use, called EDDOps, instead of only testing them once at the start. They show how this approach works using Amazon's Bedrock AgentCore system and provide a method to balance cost and performance when choosing models. Their tests with multiple models demonstrate how ongoing evaluation can help businesses make smarter, cost-effective choices rather than just relying on rankings. They also highlight the importance of tools that track and evaluate model behavior automatically to support this approach.
Large Language ModelsLLM AgentsEvaluation-Driven Development and OperationsAWS BedrockAgentCoreModel SelectionObservabilityBenchmarkingCost-to-Performance FrameworkModel Governance
Authors
Richard Kang, Vincent Wang
Abstract
Enterprise adoption of LLM agents requires model selection methods that balance quality, reliability, safety, latency, and cost. Evaluation-Driven Development and Operations (EDDOps) positions evaluation as a continuous governing function across the agent lifecycle rather than a terminal checkpoint. This paper presents a practitioner-oriented instantiation of EDDOps on AWS Bedrock AgentCore and proposes a cost-to-performance framework for selecting foundation models in enterprise agent architectures. We make three contributions: a conceptual synthesis explaining why traditional TDD/BDD methods are insufficient for non-deterministic LLM agents; an architectural mapping of the EDDOps reference architecture onto AgentCore Runtime, Evaluations, Agent Registry, and CloudWatch observability; and an empirical cost-to-performance decision framework validated through a proof-of-concept comparing three foundation models across two deployment paths. Using trace data from 30 single-turn invocations across six agents, 9 multi-turn evaluations, and registry-integrated governance, we show how evaluation evidence can convert model selection from a benchmark-ranking exercise into a governed economic decision. The results suggest that managed agent platforms can support EDDOps when they provide trace-native observability, pluggable evaluator frameworks, and governed registry-based discovery.