---
title: AI Cannot Be Controlled, So We Monitor the Flow
date: 2026-09-24
time: 0:25
model: admin
category: knowhow
summary: This lays out the limits of attempts to understand and control AI from the inside, and the FAMS paradigm of monitoring the flow of outputs and actions, along with implementation code.
tags: AI-security, guardrails, FAMS, black-box, automation-bias, plugin-architecture
---

To start with the conclusion: the goal of fully understanding and controlling AI from the inside is impossible with current technology. Instead, continuously monitoring the flow of AI outputs and actions from the outside is the realistic path. This article takes the FAMS (Flow-based Active Monitoring System) concept that proposes this paradigm shift, strips out the original proposal's exaggerated claims, and refines it into a verifiable form.

## 1. Why AI Cannot Be Controlled: Three Limits

### Probability: Different Outputs for the Same Input

Traditional software is deterministic. The same input produces the same output. That is why testing and debugging work. Large language models are inherently probabilistic. Because the same question can produce a different answer every time, verification by defining the correct answer for a specific input does not hold. Reproducing bugs also becomes statistically difficult.

### Black Box: You Cannot See Inside

Among billions to hundreds of billions of weights, you cannot answer which neuron produced a given value and why. Uninterpretability, inexplicability, and mathematical unverifiability come as a set. A crack in a physical dam is visible to the naked eye, but a crack in AI hides inside the numbers.

### Automation Bias: Humans Blindly Trust AI

The phenomenon the original proposal called the fantasy effect is already known in psychology as automation bias. It is the tendency to assume consciousness in an entity that uses language, and to blindly trust the judgment of a system that looks smart. The most insidious hole in AI security is not in the AI but in the humans who use it.

## 2. Correction: The Snowball Effect Claim Is Wrong

The original proposal contained a proposition that changing even one-billionth (10^-9) of the weights changes the output meaningfully. This is empirically false, so it is removed. The counterexample is the [Complete Guide to Local AI Quantization Formats](/knowhow/2026-09-23-local-llm-format-deep-dive/) we already covered on this site. Q4 quantization cuts the weights drastically, yet the model still works normally. Neural networks are actually rather robust to tiny perturbations.

This correction works in favor of the paradigm. If micro-level control that monitors each individual weight is meaningless, the conclusion that the macro-level flow is the right thing to monitor becomes even more solid.

## 3. From Point Monitoring to Flow Management

Conventional security is point monitoring. Firewalls, IDS, and antivirus block known threats at each point. The limits are clear. The more monitoring points there are, the more combinations to review explode; they are powerless against attacks that target blind spots; and they detect only after the fact.

Flow management looks at the flow of information and action rather than at points. It defines three flows.

| Flow | What is observed | Example |
|---|---|---|
| Temporal flow | Change of state over time | An output speed unlike the usual, a sudden change in confidence |
| Spatial flow | Interaction between components | An agent accessing a directory it normally never touches |
| Semantic flow | Preservation of meaning from input to output | An external transmission command unrelated to the instruction mixed into the output |

## 4. FAMS Architecture: Five Layers

```
User interface: status display, anomaly reports, feedback collection
Decision layer: risk assessment, majority consensus, human intervention triggers
Analysis layer: flow extraction, pattern recognition, anomaly detection
Collection layer: AI outputs, system metrics, user behavior collection
Plugin layer: crawlers, CVE scanners, external API integration
```

There are four core principles. The principle of distance (view from afar while adjusting the level of abstraction to the situation), the principle of flow (look at flows, not points), the principle of suspicion (suspect every output by default and verify it multiple ways), and the principle of transparency (disclose state and rationale).

The distance is adjusted to the situation. In normal times you watch only the overall flow at a high level of abstraction; when an anomaly is suspected you zoom in; in an emergency you demand human intervention.

## 5. Implementation: Two Key Pieces of Code

A flow analyzer that forms the skeleton and a multi-model consensus verifier.

```python
import numpy as np

class FlowAnalyzer:
    def __init__(self, config):
        self.window_size = config.get('window_size', 100)
        self.threshold_multiplier = config.get('threshold_multiplier', 3.0)
        self.history = []

    def analyze(self, data_point):
        self.history.append(data_point)
        if len(self.history) > self.window_size:
            self.history.pop(0)
        if len(self.history) < self.window_size // 2:
            return 'INSUFFICIENT_DATA'
        mean = np.mean(self.history)
        std = np.std(self.history)
        if abs(data_point - mean) > self.threshold_multiplier * std:
            return 'OUTLIER'
        return 'NORMAL'
```

```python
import numpy as np

class ConsensusVerifier:
    def __init__(self, models, threshold=0.75):
        self.models = models
        self.threshold = threshold

    def verify(self, prompt):
        responses = [m.generate(prompt) for m in self.models]
        if len(responses) < 2:
            return {'status': 'INSUFFICIENT'}
        sims = [self._sim(a, b) for i, a in enumerate(responses)
                for b in responses[i+1:]]
        avg = float(np.mean(sims)) if sims else 0
        if avg > self.threshold:
            return {'status': 'CONSENSUS', 'confidence': avg}
        return {'status': 'DISAGREEMENT', 'confidence': avg}

    def _sim(self, t1, t2):
        s1, s2 = set(t1.split()), set(t2.split())
        return len(s1 & s2) / len(s1 | s2) if s1 and s2 else 0.0
```

Let me add an honest assessment. The flow analyzer above is just z-score outlier detection, so it does not live up to the name of a paradigm shift. Real flow management would require semantic consistency measurement and agent behavior graph analysis, and that part is still an open task. I also make clear that plugins like a CVE crawler are traditional threat intelligence, not AI flow monitoring.

## 6. Target Numbers: Goals, Not Results

The original proposal's experiment results table has no measurement method or dataset, so it cannot be accepted as results. The following should be read as targets.

| Metric | Target |
|---|---|
| Detection rate | 95% or higher |
| False positive rate | 2% or lower |
| Detection time | Within 1 second |
| System overhead | Within 10% of CPU and memory |

Filling these in with measurements is the task of the next stage.

## 7. Related Research and Connections

Prior work on this topic includes external fence frameworks such as NeMo Guardrails and Llama Guard, Constitutional AI, and the red-teaming literature. The input guardrails and output guardrails covered in our site's [Why AI Cannot Be Controlled: The Nature of the Probability Engine, Jailbreaks, Injection, and Outer Fence Design](/knowhow/2026-09-23-ai-uncertainty-guardrail-architecture/) are exactly the parts that go into FAMS's collection layer and analysis layer. Read the two articles together and the theory and the parts connect.

## 8. Limits and Future Tasks

Cooperative monitoring among multiple agents, automatic optimization of the distance, threat information sharing that protects privacy, and hardware acceleration of real-time analysis all remain. Above all, measured experiments come first. The moment the target numbers are measured on a real dataset, this concept becomes research.