Locate-then-edit · key geometry · collateral interference
When rank-one edits share a weight matrix, geometry predicts the collision.
Locate-then-edit methods write a rank-one update into an MLP. Merging several such updates by plain addition produces collateral interference: one fact moves another. A closed-form key-cosine discriminant tracks that damage — until perturbation gain and merge size push the system into a second regime.
The object
A rank-one locate-then-edit (ROME and close variants) inserts a key–value pair (k, v) into a chosen layer. Two edits collide when their keys align. The geometry score is the cosine of those keys; a magnitude score is the product of residual norms. Both are read straight from the editor’s own closed form, so the map runs without a learned probe and without a held-out fit.
Interference is the change in a held fact’s target logit after a second edit is added. Negative drops are constructive (the other edit helps); positive drops are destructive. The map asks which cells stay predictable from key geometry, and at which group size that prediction fails.
What is measured
- Per-pair logit drop at merge sizes g ∈ 2, 3, 5, 10, 20, 50, 100
- Partial Spearman ρ(Icos, drop | Imag)
- Perturbation gain = median |drop| / relative dose
- Constructive fraction: share of drops < 0
Protocol sketch
Every ROME cell is a two-phase measurement. Phase 1 installs edits and dumps keys, values, and residuals. Phase 2 merges random groups and records collateral drops. Analysis scripts then rebuild the operating curve on CPU from the shipped arrays — no model weights required.
Two-regime bound
High gain (≥ 8)
13 cells. Interference is mostly destructive, and geometry stays informative at small g (2, 3, 5). Each cell carries its own geometry-valid window: g ≤ 5 at the reference cell, wider at six further qualifying cells. The per-cell windows ship in the archive.
Low gain (< 8)
9 cells, typically deeper layers. Constructive merges become common (Qwen2.5-14B: 81–88% of drops negative). Geometry keeps tracking the size of the cross-term across the crossover, and gain does the screening work: it ranks which regime a layer sits in.
Ordering
Across 22 cells, Spearman(gain, constructive fraction) = -0.82 (frozen object <= -0.7; the point estimate is not the frozen bound). Llama-2-13B is an addendum and is excluded from this n=22 freeze. Matched-dose response still spans 760× between the loudest and quietest cells.
Public figures and the 22-cell table · Reproduce from the archive · Source · Zenodo archive