Locate-then-edit · key cosine · collateral damage
Closed-form key geometry predicts locate-then-edit collateral.
A rank-one ROME update writes a key–value pair. Pairwise key cosine, and the S×C product with residual-norm growth, rank the collateral a later edit inflicts on a held fact. The signal is edit-specific, not probe-marginal.
The object
The geometry score is the cosine of two edit keys. The magnitude score is the product of residual norms. Neither is a learned probe — both are read from the editor’s own closed form.
S×C uses residual-norm growth as S. On Llama-3.2-1B the within-probe ρ(S×C, damage) is 0.402 / 0.553 / 0.677 / 0.504 at L8 / L10 / L12 / L14, and beats raw key-cosine at every layer.
What is measured
- Within-probe partialled Spearman(key-cos, damage): L8 0.395 / L10 0.534 / L12 0.602 / L14 0.301; permutation p at the floor
- Survives double-centering and norm-growth partialling
- Norm-growth baseline overtakes key-cos at L14 (0.502 vs 0.301)
- AlphaEdit residual damage 0.07–0.13 vs ROME 2–5.4; ρ(key-cos, damage-removed) L8 0.397 / L10 0.532 / L12 0.597 / L14 0.302
Protocol
- Install one ROME edit; dump keys, residuals, and held-fact logits.
- Score pairwise key cosine and S×C against collateral logit drop, within probe.
- Causal arm: replace the update with an AlphaEdit projection and re-score damage removed.
Measurements
Confound-clean gate
The within-probe partial stays above 0.15 at all four layers. Probe-identity leakage would have collapsed it; it does not.
Causal removal
AlphaEdit removes most ROME collateral at every layer. Geometry still ranks the damage that was removed, including at L14 where the correlational ρ weakens.
Scope
Primary cells are 1B with selected mid-scale checks through 3B. This page states geometry and measurements only.