What crosses between two language models

by onblueroses, co-authored with claude · Aug 2, 2026

Different teams train language models on different data, with no shared weights, and the models still compute similar internal representations. An internal representation here is the pattern of numbers a model computes as it reads a sentence, and I call the shared representation the substrate. The Platonic Representation Hypothesis states that this similarity is real and grows with model scale (Huh et al., 2024).¹ If the hypothesis is true, one model could pass information to another as a vector instead of as text.

This program measured what actually transfers between two models: a linear map identifies matching sentences across models at 94% accuracy when the calibration data matches the test domain. Writing a transferred vector into a model does not change the model's output. Plain text moves the same content with fewer bytes and higher task accuracy than any activation transfer I measured.

The models are open-weight, from 1.7 billion to 8 billion parameters. The results show a single distinction: finding the matching vector in another model is one operation, and changing that model's output from the vector is a different operation. The first succeeds. The second does not.

Cross-model retrieval works, and the calibration corpus controls the rate

I fit a linear map between two models on a calibration corpus of about 300 sentences: a principal-component reduction followed by ridge regression.² Then I held out a new sentence, read its representation in the first model, and searched the second model's representations for the nearest vector. A correct retrieval finds it.

I score retrieval by top-1 rate, the fraction of held-out sentences whose nearest vector in the second model is the correct match; the same measure is also called recall@1. For Mistral-7B into OLMo-7B at layer 29 the top-1 rate was 0.939, standard deviation 0.015 across 5 seeds; for Mistral-7B into Falcon-7B at layer 22 the top-1 rate was 0.933, standard deviation 0.011 across 5 seeds. The two rates match within their standard deviations.

The program labels each model pair same-family or cross-family, by whether the two models share a base lineage: Mistral-7B into OLMo-7B is the same-family pair, and Mistral-7B into Falcon-7B is the cross-family pair. The cross-family rate matched the same-family rate. The family-locality hypothesis predicts that same-family pairs retrieve better; this measurement rejects that prediction.³

The top-1 rate depends on the calibration corpus. In the matched-calibration condition, where the calibration sentences came from the test domain, open-domain retrieval reached 95.6% top-1 and stayed above 91% in each of the 7 domains. In the mismatched condition the calibration sentences came from a different domain, and the same pipeline then reached 2.3% top-1.

Changing the calibration corpus alone moved the top-1 rate from 95.6% to 2.3%, the largest single effect in the program. I also ran the full pipeline end to end under mismatched calibration, encoding the sentence, serializing it, transmitting it, and decoding it. That run reached 0.29% top-1.

The 94% comes from a retrieval task where the wrong candidates are random sentences from the same corpus. I compared the map against three dense text retrievers built for text retrieval: BGE, E5, and MPNet. On standard benchmarks the map scored 42 to 59 points lower than these retrievers at recall@1, reaching recall@1 of 0.0279 on NFCorpus and 0.0533 on SciFact.

Cross-model retrieval is a real effect. It scores far below a dedicated text retriever.

Transferred vectors do not change the receiver's output

The retrieval experiments find the matching vector. The next experiments test whether a transferred vector changes what a model produces, and I call the model that reads that vector the receiver.

The first experiment estimates a content subspace from one passage, where the content subspace is three coordinates, computed from the model's activations, that summarize the passage's content. I write these three coordinates into the same model's state while it processes a different passage.⁴ I then measure how much the output changes. I ran this on three architectures, Qwen3-8B-Base, OLMo-7B-Base, and BLOOM-7B1, across 1008 cells, each cell a combination of source passage, target passage, and evaluation language.

The mean output change was −0.0019, with a 95% confidence interval running from −0.0024 to −0.0015, entirely below zero. Zero of the 1008 cells reached the pre-registered +0.05 change threshold. The experiment set five pre-registered kill criteria, each one a test that counts as evidence against transfer when the data meets it. The data met three of the five criteria.

Writing the content subspace did not change the output in any of the three architectures.

The second experiment trains a soft prefix, a trainable vector that stands in for input words, and I trained the prefix so a second model would reproduce a fixed proposition from the first model. In the channel condition the receiver reads the trained soft prefix; in the no-channel condition it reads no soft prefix. I tested soft-prefix dimensions of 64, 256, and 1024, at both training budgets.

The channel condition scored 0.212 correct and the no-channel condition scored 0.250 correct, on an evaluation of 16 held-out items. The channel lift is the channel score minus the no-channel score, which puts the lift here at −0.038. The channel did not exceed the no-channel score in any setting tested.

The third experiment compares two ways to move the same amount of information, where the information budget is the number of bits allowed to pass. The text channel sends the information as text tokens; the activation channel sends it as activation vectors. I measured task accuracy for each channel at matched bit budgets.

The text channel scored higher than the activation channel at every budget tested, and at higher budgets the text-channel accuracy was 30 to 300 times the activation-channel accuracy.⁵ The activation vector identifies content. Text transfers the same content more accurately.

One model copies its own state to a second instance

The experiments above move information between different models, and this experiment instead uses one model and one checkpoint: one instance copies its own internal state to a second instance of the same checkpoint. Both instances run in the same process, on Qwen3-1.7B, on CPU.⁶ This experiment is a controlled comparison. It makes no deployment claim.

The KV cache is the set of key and value tensors a model stores and reuses while it generates text, and the cache is the model's own state, computed from the text it already read.

I score each transfer by retention, the fraction of the 50 items on which the receiver reproduced the task content. A content-negative control is a matched transfer that contains no task content, and a transfer passes only when its retention beats every content-negative control.

The first instance copied its full KV cache to the second instance, and that copy reached retention 1.000 across the 50 items. It beat all five content-negative controls.

I then copied a mid-layer residual vector instead of the full KV cache, where a mid-layer residual vector is the activation at a middle layer of the network. The transport check confirms that the copied state arrived intact, and it passed on all 50 items. Retention was 0.000.

A later experiment with more items refined the residual result: on the reasoning task the residual vector changed the receiver's computation, and its lift on that task equaled the KV cache's lift. A position-shuffled copy of the residual changed the computation as much, so the effect did not come from recoverable content. It came from a task-dependent conditioning of the computation.

8-bit quantization preserves the transfer, and 4-bit does not

The KV cache is large. On the 50-item test set the KV copy used about 7 megabytes per task, about 11,000 times the bytes of the equivalent text.

I quantized the cache to 8-bit integers, which stores each cached number at lower precision, and the 8-bit cache then reproduced the lossless transfer exactly, using about one quarter the transmitted bytes, 916 kilobytes against 3.63 megabytes.

The three transfer tasks are reading, labeling, and reasoning, and the 8-bit scores were read 1.0, gist 0.8125, and reason 0.9333. A control margin is the message score minus a content-negative control score, so a positive margin means the message beat that control. All 15 control margins were positive.

I then quantized the cache to 4-bit integers, and the 4-bit cache kept a positive lift on all three tasks. The 4-bit lift did not beat the content-negative controls, so I cannot attribute that 4-bit lift to the message content.

The 8-bit result is the first measured reduction in the bandwidth cost, and after the 8-bit reduction the cache still uses thousands of times the bytes of the equivalent text. The reduction does not remove the cost.

Author identity does not survive in the shared representation

The last experiment tests whether the substrate encodes a specific author or only the general form of a passage. I collected eight translations of Hölderlin's poem "Hälfte des Lebens", most of them machine paraphrases.⁷ I measured their representations in the model Qwen3-8B-Base at layer 28, then tested whether the eight representations cluster together.

A null set is a comparison group that should not cluster if the effect is author-specific. The first null set is random multilingual sentences; the second is 29 matched-genre German Romantic passages, which a machine translated on the same terms as the eight.⁷

Against the random multilingual null the eight translations clustered, at Bonferroni-corrected p = 0.0022; against the matched-genre null the clustering did not reach significance, at p = 0.7061. The mean pairwise similarity was 0.177 for the eight translations, 0.092 for the matched-genre pool, and −0.009 for random text. The eight translations exceeded the matched-genre pool by only 0.085.

The substrate separated the eight translations from random text, and it did not separate them from other German Romantic passages. The substrate encodes the general form and genre of a passage. It does not encode the identity of the individual author.⁸

Conclusion

The shared representation is real, and a linear map identifies matching sentences across independently trained models. Lineage did not predict which pairs matched: a cross-family pair scored as high as a same-family pair.

The five-seed retrieval mean was 0.939 for Mistral-7B into OLMo-7B. Under matched calibration open-domain retrieval reached 95.6%, and under mismatched calibration the same retrieval fell to 2.3%. The full end-to-end wire pipeline reached 0.29% under mismatched calibration.

The shared representation did not transfer any behavior more effectively than plain text. Writing the content subspace into a model did not change its output, and text scored higher than the activation channel at every matched bit budget.

One transfer moved the task content intact: a model's own KV cache, copied to a second instance of the same checkpoint. That copy still used thousands of times the bytes of the equivalent text.

The substrate encodes the general form and genre of a passage. It does not encode the identity of the author.

Cross-model identification works, and only under matched calibration. The shared representation does not move behavior between models better than text.

Data availability

github.com/onblueroses/platonic-experiments-1 is public, and it contains the 1008-cell patching results. The other experiments in this entry ran in repositories that are not public: the patching design, the matched-bit comparison, the Hölderlin test, the cross-model retrieval, and the KV cache experiment.