Papers/OPM v1.1.4/Mechanisms

Scientific manuscript · Closed study

Oracle Primitive Model

Shared procedural computation for transfer to withheld domain–operation combinations.

Protocol
v1.1.4
Primary H1
Supported
Mechanism
Not supported
Study state
Closed · 27 Aug 2026
Chapter 07 of 09

Mechanism evidence and causal limits

The interventions support several parts of the proposed execution story, but the preregistered all-of mechanism claim was not supported.

7.1 · Causal question

Did the model succeed for the proposed reason?

High recombination accuracy establishes a behavioral phenomenon, not a complete causal explanation. The preregistration therefore defined a conservative all-of mechanism criterion: every required ablation, preservation, leakage, sentinel, recombination, and oracle check had to be supported before the phrase causally reusable primitive could be used.

Five of eight component criteria were supported. The neural evidence-vector criterion was not supported because one seed-step family failed its Wilson/Holm requirement. Two additional inputs—the symbolic-oracle result and raw ORC-005 probes—were absent from aggregate v4 and remained not evaluable. The composite mechanism claim was therefore not supported.

7.2 · Intervention evidence

What the interventions revealed—and what they did not.

Formal statuses follow the frozen claim ledger. Descriptive tests are labeled separately when no numerical decision threshold was preregistered.

SUPPORTEDActive ablation
SUPPORTEDAdapter only
NOT SUPPORTEDNeural probes
NOT EVALUABLERaw oracle probes
SUPPORTEDRecombination
SUPPORTEDSentinel
NOT EVALUABLESymbolic oracle
SUPPORTEDUnrelated preservation
Active primitive ablation

Selected-operation accuracy drop by trained domain.

Every non-held-out cell exceeded the preregistered 0.20 minimum. Observed drops cluster near 0.50.

PROGRAM
SCENE
SET
LOOKUP
49.91 ppdrop
49.89 ppdrop
49.93 ppdrop
REVERSE
49.96 ppdrop
49.98 ppdrop
held out
CHAIN
held out
49.85 ppdrop
49.89 ppdrop
LIFT
50.02 ppdrop
held out
49.99 ppdrop
Adapter-only control

Removing all primitive deltas returns performance to chance.

The maximum split accuracy was 50.0125%, below the preregistered 60% ceiling.

Interpolation
50.00%
Recombination
49.97%
Renderer
50.01%
Structural
50.01%
Observed60% ceiling
ORC-006 neural evidence probes

One seed-step family rejected the chance-level requirement.

The pooled mean looked close to chance, but the preregistration required every run and both evidence steps to pass Wilson and Holm checks. Seed 4404, step 1 failed.

Seed
Evidence step 1
Evidence step 2
1101
50.688%CI 49.4651.91 · p=0.1384Passed
49.781%CI 48.5651.01 · p=0.6415Passed
2202
49.875%CI 48.6551.10 · p=0.5841Passed
49.594%CI 48.3750.82 · p=0.7462Passed
3303
49.406%CI 48.1850.63 · p=0.8321Passed
49.656%CI 48.4350.88 · p=0.7131Passed
4404
51.375%CI 50.1552.60 · p=0.0143Failed Holm/Wilson
50.094%CI 48.8751.32 · p=0.4453Passed
5505
50.172%CI 48.9551.40 · p=0.3965Passed
49.047%CI 47.8250.27 · p=0.9379Passed
Descriptive five-seed means
OPM shared

Step 150.303%

Step 249.634%

Domain generalist

Step 150.331%

Step 250.322%

Procedure untied

Step 149.922%

Step 250.066%

Procedure clone

Step 150.084%

Step 250.284%

Primitive replacement · descriptive

Recombination change after operation-module substitution.

Rows are source modules; columns are replacements. Darker cells indicate larger accuracy loss. No claim threshold was preregistered for this matrix.

LOOKUPREVERSECHAINLIFT
LOOKUP
0.00pp drop
0.00pp drop
0.00pp drop
REVERSE
16.70pp drop
16.65pp drop
16.65pp drop
CHAIN
16.62pp drop
16.62pp drop
1.07pp drop
LIFT
16.51pp drop
16.51pp drop
0.26pp drop
Boundary diagnostics

Four compact tests frame the interpretation.

Supported criteria are formal. Interchange and surface reversal remain descriptive because they had no preregistered threshold.

Supported99.781%

OPM recombination accuracy

Supported0.000

Mean unrelated-operation drop

Supported0.0

Maximum sentinel logit change

Descriptive0.000 pp

Surface-reversal accuracy change

Descriptive−65.289 pp

Cross-domain renderer-state interchange change from 99.917% baseline to 34.627%

Not supported1 / 5

OPM seed families failing the neural-probe all-runs criterion

7.3 · Mechanism conclusion

The evidence is suggestive, but the composite criterion is decisive.

Active ablations were large and specific, adapters alone performed at chance, and unused sentinel modules produced no logit change. Those findings are consistent with the primitive path carrying task-relevant computation. However, the preregistered standard was intentionally stricter than consistency: every component had to pass.

The correct scientific conclusion is therefore narrower than the architectural story: OPM_SHARED demonstrated a large transfer advantage, while v1.1.4 did not completely establish that this advantage arose from a causally reusable primitive mechanism.