The AI Agent Dilemma — Losing Autonomy, or Losing Control

Autonomy and control in AI agents cannot be solved in a single layer. Tie down every behavior with code to prevent malfunction, and the agent loses the very autonomy that defines it, regressing into an ordinary program. Hand it autonomy and lean only on text prompts, and the system sinks into instability almost immediately. The pragmatic compromise the industry has found is clear: delegate the reasoning layer to text (autonomy), and let code (control) physically block the execution layer. This article lays out both extremes of that dilemma and the design principles that balance them.

1. Where the Dilemma Begins: Why Code Control Kills the Agent

Conventional software runs on hard code: conditionals (if-else) and precisely engineered algorithms. It boasts a 0% error rate, but the moment an exceptional situation or an ambiguous input arrives, it stops working.

The AI agent emerged to overcome this. An agent's essence lies in "the autonomy to pick the right tool and solve the problem flexibly, even in unforeseen situations."

But what happens when, fearing instability, you start binding every action and every branch of the agent back into legacy code?

• Autonomy disappears: The instant you normalize every path in code, it is no longer an agent that judges for itself, but merely an expensive automation script.
• Flexibility is lost: You must handle countless edge cases one by one in code, so you fall back to the limits traditional programming already had (brittle software).

In other words, perfect code control negates the very reason an agent exists.

So where does the limitation that "code cannot handle the fuzzy world" come from? To implement business logic like "if the user's tone is urgent, act quickly; if the question is ambiguous, ask a follow-up" in conventional code, you need tens of thousands of lines of exception handling, and a slight deviation breaks the system. Express the same rule as a single line of text — "when the user's intent is unclear, ask a follow-up" — and the AI responds flexibly to thousands of variations of input.

2. The Trap of Text Control: The Instability of Soft Instructions

So what if you give the agent full autonomy and specify its behavior rules and formats only in natural-language text (prompts, schemas, logic)? A second tragedy unfolds here.

• Text has no force of law (Soft Constraint): No matter how much a prompt insists "never break the absolute rules," or "adhere to this JSON schema format," to an LLM that predicts the next token probabilistically, text is only a strong recommendation.
• Context overflow and lost attention (Lost in the Middle): The more schemas, skill manuals, task logic, and past conversation history the agent must process, the more its Attention overloads. It ignores a schema that was stated moments ago, emits malformed output, or hallucinates.
• Cross-Schema Interference: When multiple schemas are injected dynamically, the memory of the previous step collides with the currently injected instructions and produces wrong values, a black-box phenomenon that occurs constantly.

In the end, text-based control inevitably introduces instability that lowers trust in the entire system.

3. The Irony Compared: Two Modes of Control in Opposition

4. So Why Did We Have to Choose Text Control?

There is a common misconception here. It is the view that text-based control is a "hack chosen because the technology was lacking." But there were clear and decisive reasons developers did not control everything with hard code from the start and instead adopted text injection.

4.1 Code Cannot Handle the Fuzzy World

Conventional programming runs only when conditions are 100% explicit. As we saw above, moving ambiguous business logic into code drags you into a swamp of exception handling. Mimicking in tens of thousands of lines of code a judgment that a single natural-language sentence can express is the extreme of inefficiency.

4.2 Overwhelming Development and Update Speed (Hot Updates)

Modifying system code means a heavy, risky process of deployment, compilation, and server restarts. With text control, by contrast, a few lines of prompt change the entire system's behavior immediately, with no restart. The burden of redeploying the backend every time a new API field or rule appears disappears.

4.3 An LLM's Brain Runs on Natural Language, Not Code

An LLM's basic unit is not program code but the associations between words (Token Attention). The most natural and efficient language for giving a model behavioral instructions is text itself. Since it is impossible to forcefully manipulate a model's internal weights with code, writing task guidelines in the natural language the model understands and feeding them in was the most intuitive control method.

So engineers, too, at first tried to build agents with text control alone and suffered major setbacks from hallucination and ignored schemas. As a result, today's architectures have converged on a hybrid structure that mixes only the strengths of both approaches.

┌─────────────────────────────────────────────────────────────┐
│ 1. Text control (Prompt / Schema Text)                       │
│    - Role: navigation (flexible direction, understanding     │
│      complex context)                                        │
├─────────────────────────────────────────────────────────────┤
│ 2. Code control (Grammar Engine / Validation Code)           │
│    - Role: brakes and seatbelt (physically block format      │
│      breakage, retry)                                        │
└─────────────────────────────────────────────────────────────┘

5. Overcoming the Contradiction: Separating the Flexible Compass from the Physical Fence

So how are engineers untangling this grand irony? Modern agent architectures (Hermes, LangGraph, AutoGen, and others) have abandoned the fantasy of "controlling everything with code" or "solving everything with text." Instead, they found a hybrid compromise that strictly separates the reasoning layer from the execution layer.

┌──────────────────────────────────────────────────────────────┐
│  [Reasoning layer] Text control (Soft Constraint)             │
│  - Flexible intent recognition, autonomous search for a       │
│    solution path, situational analysis                        │
│  - "Give the AI the autonomy to freely set its destination    │
│    (the compass)"                                             │
└──────────────────────────────┬───────────────────────────────┘
                               │ (output)
                               ▼
┌──────────────────────────────────────────────────────────────┐
│  [Execution layer] Code control (Hard Constraint)             │
│  - Engine-level Grammar Sampling, Pydantic, Sandbox Isolation │
│  - "Install a fence that physically blocks the AI when it     │
│    tries to derail"                                           │
└──────────────────────────────────────────────────────────────┘

1. Delegate situational judgment and path selection to text (autonomy). Ambiguous context and judgment areas — like "if the customer is angry, empathize and guide them quickly through the refund process" — are left as prompt instructions to preserve the agent's flexibility.

1. Format validation and system execution are blocked by code (control). The moment the AI has judged autonomously and actually calls a system or API, engine-level grammar filtering (Grammar Filter) and a backend validation parser (Validation Code) take physical action. If the schema format is off by a single character, execution is refused and the AI is required to retry immediately, and fatal terminal commands are blocked outright at the code level.

Conclusion: The Sense of Balance Between Autonomy and Control

Building an AI agent is not a question of "how much unlimited freedom to give the AI." It is, rather, a high-level design problem of "how far to permit the autonomy that conventional code systems cannot provide, without letting the AI become uncontrollable."

Text-based control is not a hack chosen for lack of technical capability, but an essential pillar of software design for controlling a flexible interface called AI. Areas that need flexible judgment (intent recognition, reasoning, root-cause analysis) are delegated to text instructions, and areas that must be error-free (JSON syntax validation, terminal execution permissions, blocking dangerous commands) are physically guarded by code.

Ironically, the true competence of an AI agent is completed not by unlimited freedom, but when the agent can think freely inside a tight safety fence built with code.