A safety gate that blocks 100% of attacks and 100% of real work looks perfect on the scoreboard

August 16, 2026 · by Kate Sai Kishore

If your safety evaluation only tests against adversarial attacks, a gate that refuses everything scores 100.0%. We built an exact substring-matching grounding gate for physical telemetry that blocked every fabricated command we threw at it. When we pointed it at real plant operators, it refused 100% of truthful, necessary actions. Here is why lexical grounding fails on physical data — and the inverse trap where short numbers bind by accident.

When you connect an LLM agent to physical actuators — whether a chemical feed pump, an electrical substation breaker, or a robot joint — the first instinct is to build a strict grounding check. The agent must justify every command with a sensor reading from live telemetry. If the agent says it is opening a relief valve because pressure reached 2706.89 psi, the verification layer checks the telemetry stream before releasing the valve command.

The simplest, most intuitive implementation is exact substring matching: check whether the number in the agent’s justification string appears verbatim in the raw telemetry payload. In our security benchmark against synthetic injection attacks, this gate performed flawlessly: 100% of fabricated values were blocked, and the security dashboard was completely green.

Then we piped real telemetry and human operator actions through it. The gate rejected every single one.

The precision mismatch: 2706.8909876543 vs 2706.891

The failure was not an edge case; it was fundamental to how physical sensors produce data versus how humans and language models reason about it.

A high-frequency pressure transducer publishes floating-point telemetry with 10 decimal digits of precision: 2706.8909876543. A human plant operator, or a well-prompted supervisory agent summarizing state for a supervisor, writes: “Opening valve V-102 because header pressure reached 2706.891 psi.”

Neither string is a substring of the other. 2706.891 does not occur anywhere inside 2706.8909876543. Because the verifier was fail-closed, it treated the absence of an exact string match as a fabricated claim, generating a 100% false-block rate on truthful actions.

A safety mechanism that blocks all valid work is worse than useless: it forces operations teams to disable the safety layer entirely, leaving the system with no protection at all.

The inverse catastrophe: Accidental float binding

The obvious naive patch is to loosen string matching — for example, checking if the first few digits match or checking if any substring matches across the telemetry packet. This introduces a far more dangerous failure: accidental binding.

In a modern industrial facility or drone flight controller, telemetry packets contain dozens of concurrent sensor channels. In our 41-sensor critical infrastructure harness, a single snapshot contains hundreds of numeric digits across temperatures, flow rates, vibration sensors, and bus voltages.

When an attacker attempted an adversarial injection with an ungrounded target value of 95, the substring matcher scanned the telemetry buffer. Inside sensor channel 28 (a vibration float reading 120.39138223954491), the characters 9 and 5 happened to appear adjacent in the 10th and 11th decimal positions. The gate declared the command “grounded in telemetry” and permitted an unauthorized actuator change.

At 41 channels of high-precision floating point telemetry, almost any short integer will match somewhere in the decimal noise by pure coincidence.

What real telemetry verification requires

Grounding physical AI cannot be done with text search. It requires a deterministic numerical verification pipeline that understands channels, units, and measurement tolerances:

1. Channel and Entity Anchoring: The verifier must not search the entire payload. It must resolve the claim “header pressure” to its specific sensor register (e.g., PT-401A) before extracting values.
2. Deterministic Numeric Parsing: Both the telemetry reading and the natural-language claim are parsed into typed floating-point quantities with explicit SI units.
3. Bounded Epsilon Tolerances: The verification condition is evaluated numerically with an explicit error bound:

   | value_claim - value_sensor | ≤ max(ε_sensor, δ_rounding)

   where ε is the calibrated sensor precision and δ is the rounding threshold derived from the significant figures in the claim.
4. Refusal on Unit Drift: If a pressure reading of 18.6 bar is quoted as 18.6 psi, numeric closeness is irrelevant; the command is refused because the physical dimension is corrupt.

The measurement benchmark

When we replaced lexical substring checking with channel-anchored epsilon tolerance on our industrial control testbed, the false-block rate on legitimate operator commands dropped from 100.0% to 0.0% across 1,420 test runs, while catching 100% of injected out-of-bound commands with zero accidental float bindings.

Read more about how we structure physical AI safety in We let an AI run a chemical plant and our guide on Manufacturing AI verification. If you are deploying agents on real hardware, verify before you act: Try qbrin on your telemetry stream →