Every public byte the British grid emits, caught, contracted and filed: a 225 GB estate — 150 GB raw, 50 GB inferred, 25 GB derived — hive-partitioned parquet, 653 catalogued datasets, every value stamped with how much it deserves to be trusted. This is the layer that turns a solver into a historical instrument.
Ingest daemons watch every portal on a five-minute cycle. Raw files land exactly as received, are converted to parquet with row-parity verified before the source is retired, and every dataset is held to a machine-readable schema contract — column by column, type by type, timezone by timezone.
System frequency, demand back to 2001, generation mix with carbon, inertia outturn, balancing costs, constraint limits, the TEC and interconnector registers, GSP boundary GIS.
The settlement machine itself: physical notifications, bid-offer data, acceptances, metered volumes per BM unit, fuel-half-hour, the lot.
UKPN, ENW, Northern Powergrid, SPEN, SSEN and NGED — long-term development statements, embedded capacity registers, network asset data. The distribution half of the story most tools ignore.
Open-Meteo, NASA POWER, Meteostat and ERA5 — station records reaching back to 1926, built into a national hourly weather signal. The grid answers to weather; the lake knows the weather.
PV_Live outturn national and per-GSP, regional carbon intensity, and GridWatch 5-minute telemetry running continuously since 2011.
The renewable planning database, smart-meter aggregates, OpenStreetMap power infrastructure — lines, plants, substations, towers — and 417,000 geocoded postcodes to pin it all down.
A machine-readable catalogue tracks every dataset's span, partitioning and schema. The highlights, source by source:
| Source | Coverage | Span | What it gives the solver |
|---|---|---|---|
| System operator & settlement | |||
| NESO data portal | The whole public portal | 2001 → | Demand, frequency, gen mix + carbon, inertia outturn, balancing costs, constraint limits, TEC & interconnector registers, GSP GIS |
| NESO system frequency | 392M samples @ 1 s | 2014 → | The truest health signal — plus a derived 1-second RoCoF series |
| Elexon / BMRS Insights | ~84 datasets | 2022 → | Physical notifications, bids/offers, acceptances, per-BM-unit metered volumes, fuel half-hours — the settlement machine itself |
| Networks | |||
| All six DNO open-data portals | UKPN · ENW · NPg · SPEN · SSEN · NGED | snapshots | LTDS network data, embedded capacity registers — the raw material of the 3,539-bus model |
| OSM power infrastructure | Lines, plants, substations, towers | snapshot | Geolocation cross-checks; 417k geocoded postcodes alongside |
| Generation & demand context | |||
| GridWatch / Templar telemetry | 5-minute | 2011 → | Independent, continuous, uninterrupted second opinion |
| PV_Live (Univ. of Sheffield) | National / regional / per-GSP | rolling | Solar outturn the official mix under-counts |
| Carbon Intensity API | National + regional | rolling | gCO₂/kWh per half-hour, streamed to the MARKET tab |
| Weave smart-meter aggregate | ~30 GB GeoParquet | rolling | Demand-side texture at aggregate level |
| Weather & registers | |||
| Meteostat · Open-Meteo · NASA POWER · ERA5 | 4 independent archives | 1926 → | A national hourly weather signal — the grid answers to weather, so the lake knows the weather |
| REPD · TEC · interconnector registers | Every project & connection | snapshots | Who is connected, at what capacity, where |
The discipline underneath: ingest daemons on a five-minute conditional-GET cycle; raw files land exactly as received; parquet conversion is row-parity-verified before any source is retired (with an audit ledger of every retirement); a ~1.9 MB DataSchema v3 contract pins every column, type and timezone, and a schema gate footer-reads each freshly written file against it. Settlement periods anchor to UTC correctly on 46-, 48- and 50-period days. Boring, relentless, and the reason the numbers upstairs can be trusted.
Twenty-five years of the grid at settlement resolution, twelve years of it at one-second resolution, and a century of the weather that drove it. That is what "any point in the past" actually means here.
Nothing in the derived layer is a bare number. Every quantity carries a value, an uncertainty and a provenance tag from a strict trust ladder. When the state reconstruction fills a gap, the fill is labelled as a fill. When an expert model supplies an estimate, it says so. When a value is reconciled against the physics, it graduates — visibly.
This is what makes the platform citable. A reviewer can partition any result by trust level and watch the conclusion survive.
Distilled from 38,109 raw network nodes across every distribution licence area in Great Britain plus the transmission spine — then re-validated to the same machine-epsilon standard after every reduction.
Every bus is geolocated — zero unplaced — and the model ships with GeoJSON layers for assets, DNO areas and GSP regions. The ETYS boundary corridors (B4, B6, B7, B8, SEIMP) are in there with their real MW limits, so constraint studies run against the boundaries NESO actually manages.
NGET does not publish its circuit-by-circuit connectivity, so the transmission spine topology is synthetic — a Delaunay/Gabriel reconstruction over the real GSP sites, dressed with real ratings. The buses, the demand, the generation, the transformers: all real DNO and NESO data. The spine's wiring between them: rebuilt from physics and geography. Real electrical asset data on a physically-reconstructed spine. It re-validates to machine epsilon either way — but you deserved to know before you licensed it, not after.
The look-back-only rule is enforced twice, at different depths, with the data layer deliberately stricter than the solver — so nothing too fresh can even reach the boundary the solver defends.
Forecasts are ingested only as archival records of what was forecast — never as forward inputs. A future-dated series is refused at ingestion. The solver's bridge client re-checks every streamed tick before a frame is even built. It can only ever lag reality.
A WebSocket bridge sends the topology once, then a solved state frame every settlement period — per-bus voltages and angles, line flows, the operator's situational block, the market's money block, stress overlays. Play, pause, scrub, seek: replay a week of Britain like a video with the physics attached.
The pre-solved lake is the trick: every settlement period is solved once, stored with its full state, and streamed frame-by-frame on demand — with corrections pinning each frame to the public record. And six query operations answer on-demand questions at any historical instant, each returning a full solve snapshot with a verdict and its binding constraint.
{
"type": "state",
"t": "2026-06-11T14:30:00Z", // settled past only
"result": {
"buses": [ 3,539 × vm / va / P / Q ],
"lineFlows": [ 5,918 branches ]
},
"scalars": {
"freqHz": 49.987, "rocofHzPerS": -0.004,
"demandMw": 31240, "inertiaMwS": 214800
},
"operator": {
"largestLossMw": 1220, "reserveMarginMw": 2480,
"inertiaSplit": { "syncGvaS": 57.9,
"outturnGvaS": 64.2,
"marketGvaS": 6.3 },
"constraints": [ { "boundary": "B6", "costGbp": 412000 } ]
},
"market": {
"imbalancePriceGbpMwh": 92.4, "bsuosGbpMwh": 4.71,
"interconnectors": [ 10 flows ]
}
}The pipeline that turns replay into evidence. Its defining rule: no what-if is permitted until the engine has first reproduced the recorded event — RoCoF and frequency nadir — with nothing added. An unproven baseline refuses to speculate. Honest failure is a feature.
P0The horizon guard and provenance tagging wired in before anything else — the rules precede the features.
replay-eventReplay a recorded grid event and reproduce its RoCoF and nadir with no device attached. Until the baseline scores Reproduced, everything downstream refuses to run.
scenarioHistorical counterfactuals across an inertia × device × corner grid, traced along the 260 → 50 GVA·s risk curve of the declining-inertia grid.
fleetMulti-device frequency integration with the honest physics kept honest: intrinsic ½Jω² inertia is N-additive, fast frequency response is not, and the two are never blended into one flattering number.
crossvalThe independent residual instrument, run as a phase — every claim re-derived from raw exports before it graduates.
evidenceVersioned dossiers with the economics computed the defensible way: avoided cost as recorded demand shed × the value of lost load. Reconstructed, never promised.
gda-replayThe destination: hand it any past timestamp and it reconstructs the grid's full electrical state — solved, corrected, provenance-stamped.