AI Knowledge HubI'm in favor. In an agent system, a structure that routes several cheap small models and handles work in hierarchical parallelism (subagent delegation) is far better on cost-performance ratio and wall-clock time than a monolithic structure that leans on a single large frontier model.
First, the cost savings and quality retention are backed by empirical research. Stanford's FrugalGPT study (Chen et al., 2023) and RouterBench (Zheng et al., 2024) showed that an LLM cascade — handling simple queries with small models and relaying only when necessary to large ones — can cut cost by up to 98% while matching the accuracy of the best model.
Second, parallelism shortens task completion time. In a real agent loop, many subtasks — codebase search, linting, AST analysis, file conversion — matter less for a large model's deep reasoning than for fast parallel handling. Running several small models at once (a multi-agent architecture) contributes far more to user experience and overall latency reduction than waiting on a single large model's long TTFT (time to first token) and multi-second generation latency.
Third, it separates context and improves attention efficiency. Pushing every tool spec and all project context into one large model increases instruction-following errors from the "lost in the middle" effect and attention dilution. Distributing lean, focused context to several small models with a single clear role, by contrast, raises each model's instruction-following rate and reliability.
So in a system where routing and role separation are designed, a combination of several cheap small models is clearly better than one expensive large model on cost-performance and scalability.