A replay test caught every simple attempt to smuggle a model's weights out through its answers within 6 to 14 responses, but missed stealthier ones

A paper listed on arXiv on 9 October, “Anytime-valid detection of LLM weight exfiltration” (arXiv 2610.11843, submitted 8 October), takes on a worry for labs that guard their model weights: an inference server that has been compromised could leak those weights a few bits at a time by hiding them in plausible choices of word. The authors are Ines Ortega-Fernandez (MATS Research), Mateusz Kowalczyk (NASK – National Research Institute, and MATS Research) and Keri Warr (Anthropic).

The defence is to replay prompts on a trusted server and compare. The difficulty is that ordinary numerical noise also causes mismatches: between 41.2% and 87.2% of harmless responses, depending on the model, had at least one mismatched token. The authors add up evidence across many responses and calibrate it on trusted harmless traffic, which keeps false alarms bounded however long the monitoring runs.

They tested four open models: Llama-3.2-3B-Instruct, Qwen3-32B, OLMoE-1B-7B-0125-Instruct and Qwen3-30B-A3B-Instruct. Against a simple attack that does not know the sampling seed, the monitor alarmed on 100% of attack streams, at a median of 6 to 14 responses depending on the model, and on none of 1,000 harmless streams per model. An earlier baseline method alarmed on every harmless stream.

Against a stealthier attack that hides bits only where two words are nearly tied, results were mixed. The paper says: “Some seed-aware attacks that transmit only a few hundredths of a bit per token are not detected within 250 responses.” The result matters to any lab or cloud that serves valuable weights and needs to show that its serving stack is not leaking them.

• Confirmed Against seed-blind attacks the monitor alarmed on 100% of attack streams at a median of 6 to 14 responses, and on none of 1,000 harmless streams per model. arXiv 2610.11843
• Confirmed Some seed-aware attacks leaking a few hundredths of a bit per token were not detected within 250 responses. arXiv 2610.11843