Household-Level Fuel Poverty Risk Scoring from Aggregate Statistics
Official fuel poverty statistics in the UK are published only at local authority level — aggregates covering populations of tens of thousands of households. That resolution is too coarse to enable targeted interventions: a local authority can be flagged as high-risk while the actual vulnerable households are concentrated in specific streets, tenures, or property types that the aggregate masks entirely.
This project asks whether household-level fuel poverty risk can be reliably estimated using only publicly available data — without access to any household survey, administrative record, or linked dataset that would require data access agreements. The answer, within tested bounds, is yes.
Data assembly
The project integrates six government data sources at property level:
- EPC Register — Energy Performance Certificate data including Current Energy Efficiency ratings, CO₂ emissions per m², energy consumption, insulation scores, built form, construction age band, wall and floor energy efficiency, and fuel category
- ONS Census — household composition, tenure, and deprivation indicators
- NOMIS Labour Market Statistics — local employment rates and income proxies
- Stat-Xplore (DWP) — benefit claimant rates by small area, used as an income vulnerability proxy
- Climate Data Portal — heating degree days and climate zone assignments by property location
- BEIS Energy Price Data — fuel cost indices by fuel type and region
After cross-source linkage, deduplication, and quality filtering, the working dataset comprises 2.6 million property records. All location joins were performed at LSOA or postcode level depending on the source's finest available geography.
Target generation: the core methodological challenge
The project's central technical problem is that household-level fuel poverty labels do not exist in any public dataset. The LILEE (Low Income Low Energy Efficiency) indicator, published by DESNZ, provides count-level fuel poverty estimates at local authority level only. These aggregate statistics contain genuine signal about the distribution of fuel poverty risk but cannot be applied directly to individual properties.
The solution is MCMC Bayesian inference. Using the LILEE supplementary tables as informative priors — encoding what is known about the aggregate distribution of fuel poverty risk within each local authority — a Stan/brms model generates posterior predictive distributions of fuel poverty probability for each property in the dataset. These synthetic household-level probabilities are not ground truth; they are principled probabilistic estimates with full uncertainty quantification. Credible intervals are retained throughout rather than collapsed to point estimates.
Feature engineering
Fifty-plus features were engineered from the raw variables, including:
- Energy consumption per m² and CO₂ per m² (normalised for property size)
- Fuel cost-to-income ratio (combining BEIS energy prices with NOMIS income proxies)
- Insulation composite score (combining wall, floor, and roof efficiency ratings)
- Floor area per room (a proxy for overcrowding and heating inefficiency)
- Climate zone interaction terms (energy cost exposure varies significantly by region)
Energy consumption, fuel costs, and income variables are heavily right-skewed with lognormal distributions. Mean-based outlier removal and parametric thresholds were avoided throughout; quantile-based capping and nonparametric treatment were applied consistently.
Validation design
Two parallel validation pipelines were run to bound model reliability:
- Pipeline A includes Current Energy Efficiency (CEE) rating as a feature — providing a performance upper bound but with the risk of signal leakage (CEE is a component of the LILEE definition itself)
- Pipeline B excludes CEE entirely — a conservative clean result that demonstrates the model's performance without any definitional overlap with the target
SMOTE oversampling was applied before model training to address class imbalance across the five ordinal risk tiers. All evaluation was performed on a held-out test set. Sensitivity analysis across multiple random subsamples of the 2.6M record dataset confirmed output stability.
Results
| Model | Pipeline | Accuracy | ROC AUC |
|---|---|---|---|
| Random Forest | A — full features | 71.8% | 0.91 |
| Random Forest | B — clean result | 68.5% | 0.88 |
| XGBoost | B — clean result | 65.5% | 0.87 |
| Deep Neural Net | B — clean result | 55.3% | 0.79 |
Pipeline B Random Forest is the headline result: 68.5% accuracy across five ordinal risk classes, 0.88 ROC AUC, with no definitional signal leakage from the target construction process. Pipeline A provides an interpretable performance ceiling.
Technical notes
The full pipeline runs in R. MCMC inference used brms with the Stan backend (4 chains, 2,000 iterations each, convergence confirmed by R-hat diagnostics). Data wrangling used dplyr and arrow with Parquet-format intermediate files for performance across 2.6M records. Machine learning validation used caret with ranger (Random Forest) and the xgboost package. Visualisation used ggplot2. A parallel Python pipeline (scikit-learn, pandas) was used for cross-validation of the Random Forest results.