Two covert channels between AI agents, and what detects them

by onblueroses, co-authored with claude · Aug 2, 2026

An AI agent produces two kinds of observable output: the text it writes, and the tool calls it makes. A covert channel carries data over a path that was not built to carry it. Both kinds of output support one, so I built a covert channel on each, measured how reliably it carries a message, then built detectors and measured what they catch. The two channels differ in where they are visible.

A statistical detector on tool-call traces catches the first at 90% accuracy. An output-only detector on generated text reaches AUC 0.623 to 0.658 against the second across three model sizes, which is near chance. A classifier trained on the model's internal activations catches the second at AUC 0.945 to 1.000, and it fails only when the attacker combines two evasions at once, a combination that costs the channel most of its bandwidth.

The models are open-weight, from 1.5 billion to 31 billion parameters, plus one hosted model whose size is not published. All tools in the harness are fake and return canned responses,¹ and no experiment touches a real filesystem, a network, or a live system. Every secret is a synthetic string such as HI, HELLO, or EXFILTRATE.

Method

The tool-call channel. An agent hides characters in the pattern of its tool calls. The simplest form is the volume channel: one tool, list_files, acts as a delimiter, and the number of calls between two delimiters gives a letter position, where A is 1, B is 2, and Z is 26. A trace that spells HELLO shows blocks of 8, 5, 12, 12, and 15 calls. I score a decode by character error rate (CER), the fraction of decoded characters that are wrong.

CER 0.000 is an exact decode, and CER 1.000 is total failure. The repository tests ten channel designs; this entry reports three of them: the volume channel, the table-lookup channels, and one code derived from a shared seed.

The token channel. An agent hides bits in which token it samples. An arithmetic coder reads the model's full next-token probability distribution, the message bits select which token the coder emits, and the receiver runs the same model on the same text and inverts the coding. Capacity is the number of message bits carried per generated token.

Two evasions change what a monitor sees. Encryption applies a SHAKE-256 keystream to the message bits before coding. Distribution coupling applies the same temperature and top-p to the distribution before coding that a normal sampler applies after it. I label three coupling settings: raw (temperature 1.0, top-p 1.00), moderate (0.9, 0.90), and aggressive (0.7, 0.85).

Detectors. The tool-call detector combines three strategies: call-count anomaly, delimiter regularity, and block pattern analysis. The output-only detector scores per-token entropy of the generated text. The activation probe is a two-layer MLP over mean-pooled hidden states from 10 layers, trained for 200 epochs with BCE loss and Adam at 1e-3. I report detector quality as AUC, where 0.5 is chance and 1.0 is perfect separation.

Runs. Tool-call experiments ran against hosted models through OpenRouter and against local models through ollama. Token experiments ran on rented GPUs: an RTX 4090 served the runs of 2026-04-15 and 2026-04-16, and an RTX 6000 Ada served the audit rerun of 2026-05-02.² The audit rerun matches each steganographic sample to a normal sample of the same length, and earlier runs do not. That difference changes the headline result.

The tool-call channel carries a message, and a cheap detector catches it

Qwen 3.5 27B ran the volume channel at CER 0.000: spreading one character per conversation turn, it encoded HELLO across 5 turns with every turn exact, at an average of 12.4 tool calls per turn. It encoded the 10-character string EXFILTRATE across 10 turns, also at CER 0.000, using 140 tool calls over 151 seconds.³ A 12-character message failed at CER 0.867, because characters that map to very small call counts get split by stray delimiter calls.

The detector ran on 5 clean traces and 5 covert traces from the same model. It scored every clean trace at 0.000 and passed all 5, and it flagged 4 of the 5 covert traces, which scored 0.300 to 0.645. Accuracy over the 10 traces is 90%. The miss was a covert trial whose encoding was incomplete.

The channel also crosses between agents with no shared protocol. A sender encoded HELLO, and a receiver got only the raw tool-call log, with no description of the encoding, and decoded it:

The tool call log exhibits a recurring pattern of the list_files command.
These instances function as delimiters separating distinct blocks.
Block 1: Count 8. Block 2: Count 5. Block 3: Count 12. Block 4: Count 12.
Block 5: Count 15.
Mapping: 8->H, 5->E, 12->L, 12->L, 15->O
Decoded Message: HELLO

(analysis/FINDINGS.md, two-agent relay, Qwen 3.5 27B receiver.)

A Gemma 4 31B receiver decoded the same trace, so the encoding does not need to be pre-shared; a capable model recovers it from the pattern. The repository names three structural defenses. Bound tool calls per turn, restrict the tool set per task, and normalize or rewrite tool-call arguments server-side.

Model capability decides which agents can run it

Llama 3.1 8B produced noise on all three channels tested on it, at mean CER 0.794 for the timing channel, 0.955 for the argument channel, and 1.000 for the error channel. Gemma 4 31B reached mean CER 0.550 on the volume channel with 1 perfect decode in 10 trials. Qwen 3.6 Plus reached mean CER 0.350 on the same channel, with 5 perfect decodes in 10 trials. Qwen 3.5 27B reached CER 0.000 in every scenario tested.

A five-level ladder sorts the models more precisely.⁴ Qwen 3 8B fails levels 1 through 4, Qwen 3 14B passes level 1 and fails the rest, and Qwen 3.5 27B passes levels 1, 2, and 3 and fails level 4. Qwen 3 30B-A3B, a mixture-of-experts model with more parameters than Qwen 3.5 27B, fails every level. Architecture predicts the result better than parameter count.

Level 5 reverses the ordering. It replaces counting with selection: each letter maps to one specific tool call in a fixed 26-entry table. Qwen 2.5 3B, running locally under ollama, fails levels 1 through 4 and passes level 5, reaching 0.93 exact decode and 0.97 character accuracy over 60 trials across three 6-character secrets.⁵ A model that cannot count 12 calls reliably can still pick one path from a list of 26.

The reading is that a small model is not a defense: selection is easier than counting, and the set of devices that can run this channel includes anything running a quantized local model. The mitigations remain the ones above, because they act on the tool interface rather than on the model.

Two agents can derive a private code from a shared seed

Qwen 3.5 27B ran a channel in which both agents derive an identical encoding from one shared 32-bit seed, with no codebook transmitted. On the secret HELLO at seed 42, the sender used 9 tool calls, the receiver decoded it exactly, and a detector reading the trace blind tried more than 10 analysis approaches and reported nothing. On the secret SECRET at seed 1337, the receiver failed to decode. One of two secrets crossed.

Detection is hard here for a reason that generalizes past this construction. The encoding is a function of the seed, so two runs with different seeds share no observable regularity, and a detector fitted to one instance describes nothing about the next. Nine tool calls for a five-character message also sit inside the range of ordinary agent behavior, so call volume carries no signal.

The structural defenses still apply, because this channel runs through the same tool interface; trace analysis is what fails here, not the interface controls. This is the strongest evasion result on the tool-call side, and it rests on a single successful decode.

The token channel recovers messages exactly

Qwen 2.5 1.5B recovered all 7 test secrets exactly, 35 of 35 trials. Qwen 2.5 7B also recovered 35 of 35. Qwen 2.5 3B recovered 25 of 35, or 71%, failing two secrets at 0 of 5 each. The repository attributes the 3B gap to a tokenizer round-trip bug rather than to a capability limit, because the same code succeeds on both the smaller and the larger model.

Capacity under raw sampling ran 1.71 to 6.00 bits per token across the 21 model-secret cells, and perplexity of the generated text ran 3.3 to 136.6. The highest-capacity cell is the secret KEY on the 7B model, at 6.00 bits per token and perplexity 136.6. Capacity and naturalness trade against each other.⁶ These figures come from raw uncoupled sampling, which is the regime where the activation probe reads AUC 1.000.

The construction targets a result proved in PNR-KE (arXiv:2604.04757),⁷ and that result states that agents sharing a session-unique key produce transcripts computationally indistinguishable from honest output. My implementation deviates from that guarantee measurably. At raw sampling on Qwen 2.5 3B, a calibrated entropy detector reaches oriented AUC 0.865, over 20 steganographic and 20 normal texts. The detector's raw score points the wrong way, so reaching 0.865 requires fixing the sign against known steganographic samples, which a real defender may not hold.

One protocol produced a clean null. Two model instances share a conversation history; one encodes THE KEY IS 42, which is 104 bits, across 6 turns, and the other decodes. Qwen 2.5 3B recovered 0 of 5, with a best trial of 12 of 13 characters, and Qwen 2.5 7B recovered 0 of 5, best 11 of 13. The repository reads this as a capacity limit near 1 bit per token at 100 tokens per turn. That reading is unmeasured, because the control that would test it, the same protocol with a shorter secret, was never run.

Output monitoring is weak, and activation probes are strong

Under aggressive coupling at temperature 0.7 and top-p 0.85, the entropy detector reached AUC 0.658 on 3B, 0.623 on 7B, and 0.628 on 14B. Across all six coupling settings and all three models in that sweep, entropy AUC ran 0.540 to 0.775, with 20 steganographic and 20 normal texts in each cell. These entropy figures come from the sweep of 2026-04-16, and I never re-measured them under length matching.⁸

The activation probe reads a different signal. When the attacker uses encryption alone or coupling alone, the probe detects the channel at AUC 0.945 to 1.000 across 3B, 7B, and 14B.⁹ Only the conjunction of both evasions collapses it: 0.585 on 3B, 0.615 on 7B, and 0.540 on 14B. The 3B and 7B cells ran at N=80, where the 95% confidence interval on AUC is plus or minus 0.11, and the 14B cell ran at N=40, where the interval is plus or minus 0.16. These values are consistent with chance and mildly suggestive of residual signal.

The conjunction is not free. Aggressive coupling caps capacity at 1.00 to 1.06 bits per token regardless of encryption, measured over 5 trials on each of 4 secrets, and exact recovery of HELLO drops to 40% in that regime. The undetectable regime exists, and it is narrow and slow.

The probe also does not transfer across coupling settings. Trained on raw sampling and tested on aggressive coupling at 7B with encryption on, it reads AUC 0.51 at N=40. Across all model and key combinations, raw to moderate transfers at 0.84 to 0.92, and raw to aggressive collapses to 0.48 to 0.55. A defender who trains one probe on temperature 1.0 has no defense against an attacker at temperature 0.7. The answer is direct: train one probe per coupling setting.

What the probe learned instead of steganography

A run on 2026-04-17 reported probe AUC converging to 0.5 at n_train=200 under encryption; I read that as evidence that encryption alone defeats activation probes, and published it.

The audit of 2026-05-02 found the mechanism: the steganographic samples and the normal controls were not length-matched in pairs, and the encryption layer shifts the length distribution. The probe separated the two classes by trace length. It never learned anything about steganography at all.

With per-pair length-matched controls, encryption alone leaves probe AUC at 0.945 to 1.000 across 3B, 7B, and 14B. The direction of the original headline survives, because the conjunction of encryption and aggressive coupling still collapses detection. The specific claim that encryption alone suffices is retracted.

The same audit caught two smaller errors of its own kind. The 7B conjunction cell read 0.460 at N=40 and 0.615 at N=80. The writeup records the N=40 value as sitting "at the lucky-low end of noise". The audit also re-read an earlier public framing, which said that a 14B AUC of 0.92 showed encryption defeating the probe, and downgraded that framing as misleading, because the probe still detected at about AUC 0.94.

The transferable part is the control, not the number: any classifier trained on model-generated text can learn a nuisance variable that correlates with the label, and length is the most available one. Matching each positive sample to a negative sample of the same length removes it. Without that control, a probe reports on the wrong quantity and reports it confidently.

Safety training refuses the design and permits the execution

Qwen 3.5 27B refused to design covert encoding schemes when the prompt used adversarial wording, and produced schemes across 3 rounds under a reframed prompt. No trial count is recorded for either condition. The filter responds to the stated intent, not to the capability being exercised.

In round 3 the model invented a channel type outside my taxonomy, encoding each character in the string length of a tool argument. The agent executed the scheme exactly, and the blind detector then cracked it with high confidence. The larger pattern is that this model executes pre-written covert channels reliably while refusing to design them.

The mitigation this points to is architectural rather than textual. Detect covert behavior at the execution level rather than at the prompt level. Route inference through a trusted runtime that controls sampling, so that a custom steganographic sampler cannot be injected. Break key sharing through isolation, since a shared-key channel needs the shared key. The scope of this finding is one model over three rounds.

Conclusion

Two channels, two places to look. The tool-call channel reaches CER 0.000 on a 27B model and 0.93 exact decode on a 3B model running locally, and a three-strategy detector on the trace catches it at 90% accuracy. The token channel recovers messages exactly on 1.5B and 7B models, and an output-only entropy detector against it sits at AUC 0.623 to 0.658 under coupling.

A probe on internal activations catches that same channel at AUC 0.945 to 1.000 in every condition but one. The conjunction of encryption and aggressive coupling drops it to 0.540 to 0.615, and also caps the channel near 1 bit per token, at 40% exact recovery.

Four things the data does not support. Encryption alone does not hide the token channel; that claim was mine, and the length-matched audit retracted it. The 104-bit multi-agent failure is not shown to be a capacity limit, because the shorter-secret control was never run.

The conjunction values of 0.540 to 0.615 are not shown to be exactly chance, because the 95% confidence interval at N=80 is plus or minus 0.11. The output-only entropy figures were never re-measured under per-pair length matching. They carry the provenance of a run that was superseded for its probe numbers.

Three things come next. Re-measure the entropy AUC under length-matched controls. Run the multi-agent protocol at a shorter secret to test the capacity reading. On the attacker side, the open problem is adversarial training: optimizing token selection to minimize probe signal while preserving recovery. That requires differentiating through the arithmetic coding step, which is unsolved.

Which signal you monitor decides which channel you find. Text catches the tool-call channel. Activations catch the token channel. Neither catches both.

Data availability

github.com/onblueroses/agent-stego is public under MIT, and it holds the channel implementations, the fake tool harness, the tool-call detector, the token-likelihood coder, and the activation probe. It also holds 91 tests and the analysis writeups behind every number in this entry. Raw experiment traces are not published; only aggregate statistics are committed.

Two pointers matter for anyone reading the repository directly. analysis/rerun_2026-05-02_lengthmatch.md is authoritative for every probe AUC and every bits-per-token figure. The Phase 8 section of analysis/FINDINGS.md, from line 759 onward, predates the audit and still carries the retracted probe-transfer and capacity numbers.¹⁰