The numbers, and their limits.
Tournament t003, 749 intervention and placebo windows on synthetic waterfloods with planted truth (the revised physical counterfactual; earlier runs are in the ledger). The bar and the result are on the proof page.
Effect bias on planted truth
See the tables below.
Interval coverage
See the tables below.
Placebo false-positive rate
See the tables below.
Unidentifiable pairs correctly flagged
See the tables below.
Same cases, same rules
Scored against, untouched
- Naive before–after
- Decline-curve refit
- 2 × 2 difference-in-differences
- An open capacitance–resistance library, untouched
- An open staggered difference-in-differences library, untouched
- Synthetic control with placebos
- Bayesian structural time series
Thresholds set on validation data and frozen for every test, including the cases the engine never saw. The truth model differs from the engine's own.
The bar, set before the first run
Do not lose to the untouched open libraries out of distribution.
Out of distribution (worst wells first, producer interference, allocation error, hidden aquifer, pooled), RMS relative error of the physical counterfactual: 143 % against 119 % for the open CRM library (difference 90 % interval 5 to 44 points); 144 % against 84 % for the open staggered DiD library (difference 90 % interval 42 to 77 points). Worse than both beyond the interval.
First proof
Three to four weeks: a capacitance–resistance fit on a released field's daily data (licence first), then a synthetic field with selection, staggering, interference and allocation error; one bias and coverage table against the open libraries.
Producer interventions: barrels against the planted truth
| Median bias | RMSE | Inside 90 % | Placebo called significant | |
|---|---|---|---|---|
| Sweep: capacitance-resistance counterfactual | 50 % | 130 % | 86 % | 11 % |
| Open CRM library, untouched | -2 % | 96 % | — | — |
| Open staggered DiD library, untouched | -2 % | 72 % | 52 % | 53 % |
| Synthetic difference-in-differences | 1 % | 48 % | 81 % | 13 % |
| Synthetic control | 4 % | 47 % | 75 % | 25 % |
| The causal engine as shipped | 6 % | 52 % | 92 % | 11 % |
| Difference-in-differences, not-yet-treated controls | 6 % | 58 % | 39 % | 62 % |
| Difference-in-differences, 2×2 | 6 % | 58 % | 40 % | 62 % |
| Bayesian structural time series | 6 % | 49 % | 40 % | 64 % |
| The well's own decline, refitted | 25 % | 85 % | 33 % | 73 % |
| Before and after | 5 % | 71 % | 50 % | 59 % |
Workovers and pump-speed changes on the held-out synthetic fields; truth = the treated run minus the untreated run of the same field. Relative errors on uplifts of at least 2,000 bbl. Placebo: windows with no intervention whose 90 % interval excludes zero.
Out of distribution: RMSE of the relative error
| worst wells first | producer interference | allocation error | hidden aquifer | |
|---|---|---|---|---|
| Sweep: capacitance-resistance counterfactual | 136 % | 102 % | 149 % | 174 % |
| Open CRM library, untouched | 89 % | 114 % | 157 % | 97 % |
| Open staggered DiD library, untouched | 91 % | 77 % | 107 % | 52 % |
| Synthetic difference-in-differences | 50 % | 28 % | 74 % | 42 % |
| Synthetic control | 54 % | 34 % | 58 % | 33 % |
| The causal engine as shipped | 56 % | 34 % | 53 % | 37 % |
| Difference-in-differences, not-yet-treated controls | 65 % | 35 % | 54 % | 40 % |
| Difference-in-differences, 2×2 | 65 % | 35 % | 54 % | 40 % |
| Bayesian structural time series | 50 % | 39 % | 86 % | 38 % |
| The well's own decline, refitted | 92 % | 50 % | 94 % | 61 % |
| Before and after | 71 % | 47 % | 77 % | 45 % |
The bar's splits. The bar compares against the two open libraries, pooled over these splits on windows where both apply; the panel methods are shown for context.
Injection changes, where no panel method applies
| RMS relative error | |
|---|---|
| Sweep: capacitance-resistance counterfactual | 70 % |
| Open CRM library, untouched | 232 % |
One injector up or down; the uplift summed over every producer. Only the two capacitance–resistance models can score these.
What is proven. What is not.
Proven, in simulation
- Calls 11 % of no-intervention windows significant; before–after 59 %, 2 × 2 difference-in-differences 62 % (simulation, t003)
- Its 90 % interval covers 86 % of planted uplifts, a little under nominal
- Injection changes, where no panel method applies: 70 % RMS relative error, against 232 % for the open CRM library
Not proven
- Against the bar: out of distribution its error is 143 % against 119 % for the open CRM library and 84 % for the open staggered DiD library, worse than both beyond the interval
- On producer workovers and pump changes the panel methods win: 130 % RMS relative error and +50 % median bias, against 47–48 % and 1–4 % for synthetic control and synthetic difference-in-differences
- Injector–producer gains: every pair flagged unidentifiable from 36 months; rank agreement with the true gains 0.19
- No field data; public panels for placebo runs only
- The full-physics deck not run
- No reserves booking, ever
The first real number comes from your data.
A month of your exports, replayed offline, gives the figure that matters on real operations. We will show it to you whether it is good or bad.