Research Outputs: Small area estimation of fuel poverty in England, 2013 to 2017

This article summarises research undertaken by the Office for National Statistics (ONS) and aided by the Department for Business, Energy and Industrial Strategy (BEIS) to produce estimates of fuel poverty at local and small area levels with indicators of quality. Following a review by the ONS Methodology Advisory Service (MAS) in 2012, we have made use of additional data sources to produce alternative estimates that have confidence intervals as a measure of precision at the local authority level.

The LA level estimates of fuel poverty presented here are broadly comparable with existing statistics published by BEIS using the current methodology. For the year 2017, 86% of the local authority estimates published by BEIS are within the confidence interval range produced by the alternative ONS method. We are still evaluating differences in the alternative ONS estimates so no conclusions should be drawn about the accuracy of the existing statistics; they remain the best official estimates.

ONS estimates of the proportion of households in fuel poverty at local authority level range from 5% to 20% in 2017. There is some variability across local authorities regarding the size of the confidence intervals that are generated using the alternative approach. Confidence intervals for local authority fuel poverty estimates range from plus or minus 1.4% to plus or minus 6.4%. The average confidence interval across local authorities is plus or minus 2.7%. The availability of confidence intervals is an important aid for users to interpret the quality of fuel poverty statistics, which can vary across local authorities. For example:

• detecting genuine differences in fuel poverty between local areas (for areas where the two confidence intervals do not overlap)

• detect genuine changes in fuel poverty over time (again, when two confidence intervals do not overlap)

• understand the range of likely values of fuel poverty instead of just providing a point estimate, which can be misleading

At this stage we have only been able to produce alternative estimates of fuel poverty at the local authority level.

We continue to work with BEIS to build on this research to improve estimates for lower-level geographies, in particular, Lower layer Super Output Areas (LSOAs). This includes the potential use of new data sources that can supplement the English Housing Survey (EHS) as an indicator of fuel poverty, for example, the increasing availability of Energy Performance Certificates¹ (EPCs) . We will also continue research to understand the differences at local authority level between these alternative estimates and the current published estimates.

We anticipate that the alternative approach presented here will be successful at delivering fuel poverty estimates with quality indicators for other geographies for which fuel poverty estimates are currently published. These include Parliamentary Constituencies (PCs) and County level estimates.

As this is the first iteration of this research to develop alternative methods, we are asking users for feedback on these methods and how these research outputs could meet their user needs in comparison with the published estimates available.

Notes for: Main points

1. Energy performance certificates (EPCs) are a rating scheme to summarise the energy efficiency of buildings. So far, EPCs have been issued to approximately 50% of residential addresses in England, however, some property types will be under-represented in the data available.

These Research Outputs are not official statistics, nor are they used in the underlying methods or assumptions in the production of official statistics. Rather they are published as outputs from research into a methodology different to that currently used in the production of existing fuel poverty estimates published by the UK Department for Business, Energy and Industrial Strategy. These outputs should not be used for policy or decision-making.

The Office for National Statistics (ONS) is transforming the way it produces population and wider statistics to better meet the needs of its users. Working in partnership with the Government Statistical Service (GSS), we are progressing a programme of work to put administrative data at the heart of our statistics system.

The Department for Business, Energy and Industrial Strategy (BEIS) has a separate programme of work to develop fuel poverty statistics, but have worked jointly with the ONS to evaluate the benefits of increased use of administrative data in their production.

This project was a collaboration between the ONS and BEIS. These estimates of fuel poverty are for local authorities in England. Fuel poverty is a devolved issue, with each nation in the UK having its own fuel poverty definition, targets and policies to tackle the issue.

We welcome users providing feedback to us on their quality, value or the impact that using these figures would have if they were used in place of official statistics. Feedback should be sent by email to fuelpoverty@beis.gov.uk

Fuel poverty statistics are in high demand from central and local government, academics and researchers. Some local authorities use these statistics to target energy efficiency policies. The UK Department for Business, Energy and Industrial Strategy (BEIS) publishes an Annual Fuel Poverty Statistics Report badged as National Statistics. This provides a detailed view of fuel poverty trends and analysis in England, including annual estimates of the number of households living in fuel poverty.

Fuel poverty is a complex concept to measure, either from surveys or alternative sources. Since the publication of the Fuel Poverty Report led by John Hills, fuel poverty in England has been measured by the Low-Income High Costs (LIHC) indicator. Under this indicator a household is fuel poor if: they have required fuel costs that are above average (the national median level); and were they to spend that amount, they would be left with a residual income below the official poverty line. The three main factors that determine whether a household is fuel poor are:

• household income
• household energy requirements
• fuel prices

Along with the National Statistics on fuel poverty, BEIS also publishes annual estimates of subregional fuel poverty, badged as Experimental Statistics.

The geographical levels covered by the above fuel poverty statistics include:

• region (National Statistics)
• county
• local authority
• Parliamentary constituency
• Lower layer Super Output Area (LSOA)

The subregional fuel poverty statistics are derived using a regression modelling approach and use household-level data drawn from the English Housing Survey (EHS), along with other aggregated household-level information derived from the 2011 Census and Experian data. More details about the current methodology are available in the Fuel poverty subregional methodology.

In 2012, the ONS Methodology Advisory Service (MAS), on behalf of BEIS, carried out a review of the current methodology used to produce subregional statistics of fuel poverty and advised that small area estimation (SAE) modelling could be used to improve these statistics.

Since that time, the ONS has acquired more administrative data sources that can be used as auxiliary data to support the development of SAE methods for fuel poverty. This article is a summary of our early research testing the SAE approach at local authority level. Longer-term, we will be exploring alternative approaches to deliver modelled fuel poverty outputs for lower geographies.

One of the main recommendations in the MAS review was that users of fuel poverty statistics need information about the quality of estimates that are being produced at the small area level. Confidence intervals, which are a widely accepted measure of uncertainty (or precision) among data producers and users, should accompany the outputs to aid user interpretation. Wider confidence intervals are an indication of less precise estimates, which means that data users may need to be more cautious when using and interpreting the relevant statistics¹ .

We have produced alternative subregional estimates of fuel poverty, using household-level fuel poverty data (modelled using data from the EHS) combined with area-level data from administrative data sources and the census. A small area estimation (SAE) approach is used here and compared with the method currently used by BEIS to generate subregional fuel poverty statistics.

The approach used for the analysis presented here has been widely used by the ONS to produce local area-level estimates of income poverty in England and Wales, along with measures of uncertainty (see, for example, Small area technical report 2014 (PDF, 1.37MB), and accompanying estimates).

Notes for: Background information

1. Users of fuel poverty estimates produced from the method described in this article can be 95% certain that the local authority estimate lies within the confidence interval range.

Small area estimation (SAE) is a growing area of development for national statistical institutes and academics worldwide. The SAE approach suggested by the Office for National Statistics Methodology Advisory Service (MAS) and used for the analysis here aims to derive model-based estimates of a measure of interest from survey data combined with area-level auxiliary data (for example, census or administrative data). In the context of fuel poverty, the measure of interest is whether a household is fuel poor, which is derived from the English Housing Survey (EHS), as described in the previous section.

Estimates of the proportion of households in fuel poverty by area can be obtained as direct survey estimates. This is arrived at by calculating the ratio (typically weighted) of the number of sampled households that were identified as fuel poor to the number of sampled households in an area. However, direct estimates are unreliable for small areas, as the sample sizes are usually too small. A model-based approach is preferred in this case, and the type of model used here is based on the principle of “borrowing strength” between areas. Using all the data available in the sample, a relationship is established between the variable of interest and so-called auxiliary variables.

Auxiliary variables capture information that is available for the whole population (rather than a sample) and are likely to be related to, or predictive of fuel poverty. This relationship (expressed through the estimated model coefficients) is assumed to apply everywhere, allowing for more accurate estimates of the variable of interest in the smaller areas (that is, where the sample is small relative to areas with more sample or respondents).

Since the review conducted by the MAS in 2012, more auxiliary data sources have become available to the ONS. These include data from the Valuation Office Agency (VOA), the Department for Work and Pensions’ Benefits and Income data, and the ONS Admin-Based Population Estimates (ABPE).

The subregional estimates presented here use fuel poverty data from the EHS and auxiliary data from the 2011 Census and the most recent administrative data sources that were available at the ONS, or from other publicly available sources. The list of auxiliary data used for the analysis are in Annex A.

Fuel poverty data from the EHS were supplied by BEIS to the ONS for the years 2013 to 2017 for five levels of geography:

• county
• local authority
• Parliamentary constituency
• Middle layer Super Output Area
• Lower layer Super Output Area

These data contained aggregated counts of sampled households and the corresponding counts of households in fuel poverty per area and year. Each of the datasets covered the full national sample of approximately 12,000 sampled households. The data for one year, for example, 2017, refer to the combined EHS sample from the 2016 to 2017 and 2017 to 2018 financial years, that is, the data are based on fieldwork carried out between April 2016 and March 2018. For more information, please see page 4 of the most recent annual Fuel poverty statistics report 2019. The dataset used in the modelling process was created by combining fuel poverty data and various administrative data at the local authority level, including data that the ONS has access to (see Annex A).

Small area estimates (SAEs) of the proportions of households in fuel poverty per local authority were produced for the years from 2013 to 2017, along with 95% confidence intervals, for 325 local authorities. This includes all local authorities in England apart from the Isles of Scilly.

The SAEs reported in this section are compared with the fuel poverty estimates published by the Department for Business, Energy and Industrial Strategy (BEIS) in the Annual Fuel Poverty Statistics Reports.

In the remainder of this section our alternative local authority-level fuel poverty estimates will be referred to as “ONS estimates” and those currently published by BEIS will be referred to as “published estimates”.

Alternative local authority-level estimates of fuel poverty (ONS)

Figure 1: ONS estimates of fuel poverty in England from 2013 to 2017

The main messages from the maps are as follows:

• the South East region has notably lower levels of fuel poverty
• the majority (or more than half) of local authorities have an estimated proportion of households in fuel poverty of between 8% and 14%
• overall, the levels of fuel poverty remain broadly unchanged over time, however, some areas mainly in the North of England improved (reduced levels of fuel poverty from 2013 to 2017)

Comparison with published local authority-level fuel poverty estimates

Figure 2 reports summary statistics for the estimated proportions of households in fuel poverty per local authority, according to the ONS estimates compared with the published estimates for all the years of data, 2013 to 2017.

The local authority median average for the proportion of households in fuel poverty is very similar between the two methods. For the years 2013 to 2017, this remains broadly unchanged at either 10% or 11%.

The range of ONS estimates is slightly wider than that of the published estimates. This indicates that there are larger differences across local authorities for the estimated proportion of households in fuel poverty using the ONS method. This is most noticeable in 2015, where ONS estimates range from 3% to 22%, compared with a range of 5% to 18% for BEIS estimates. Over the years, ONS estimates consistently have a wider range when compared with the published estimates.

Figure 2: Summary statistics of ONS and published fuel poverty estimates

Figure 3 reports the difference between the ONS and the published estimates of the proportions of households in fuel poverty per local authority for 2017, the most recent year of data. The dots on the right-hand side of the vertical line (orange dots) refer to local authorities where the ONS estimate is higher than the published estimate, and the other way around for the dots on the left-hand side (purple dots).

We can see that for the majority of local authorities, the difference between the ONS estimates and the published estimate is less than 3 percentage points. However, among cases where the ONS estimate is higher than the published estimate, there is a tendency towards larger differences, with over 10 local authorities estimating between 4 to 8 percentage points higher. 

The same scatter plots for the previous years of data were also produced and showed a very similar pattern to the one shown for 2017 so they are not reported here.

Figure 3: Difference between ONS and published estimates of proportions of households in fuel poverty per local authority, England, 2017

Figure 4 shows a scatter plot of the 2017 published estimates against the ONS estimates of the proportions of households in fuel poverty per local authority. Highlighted in red are those local authorities where the published estimate falls outside of the 95% confidence interval of the ONS estimate.

We can see that for the vast majority of local authorities, the published estimates fall within the 95% confidence intervals of the ONS estimates. However, there are some local authorities where ONS estimates a higher proportion of households in fuel poverty than the published estimate, with the latter falling outside of the ONS estimated confidence interval.

On the other hand, there are some local authorities where ONS estimates lower levels of fuel poverty compared with the published estimates and with the latter falling outside of the ONS estimated confidence interval. The same graphs were produced for the years from 2013 to 2016, and again showed a similar pattern to that observed in 2017, so they are not reported here.

Figure 4: Scatter plot of published estimates compared with ONS estimates of proportions of households in fuel poverty per local authority, England, 2017

We will continue working with BEIS to evaluate the quality of the ONS estimates and understand reasons for the differences in local authority estimates of fuel poverty. We will extend this evaluation to explore the stability of local authority-level estimates over time, with particular focus on any local authorities with notable changes in fuel poverty estimates between years.

Small area estimation of fuel poverty for other geographies

The same small area estimation approach that was used with the local authority-level data was also applied to the corresponding Middle layer Super Output Area (MSOA)-level data that were supplied by BEIS. The same data at MSOA-level violated one of the main assumptions (survey estimates following a binomial distribution) to this SAE approach. This was due largely to the small sample sizes in each of the MSOAs.

MSOAs with very small sample sizes of one or two will have had zero as the number of households in fuel poverty, simply because the sample was too small. For this reason, the model parameter estimation at MSOA-level could not be relied upon and the results are not presented here.

Given that Lower layer Super Output Areas (LSOAs) are smaller than MSOAs, we did not attempt to apply the same small area estimation approach to the LSOA-level data that were supplied by BEIS, as the same issue would have occurred. Potential alternatives for MSOA and LSOA models are discussed in Section 7.

Consideration has also been given to using the small area estimation approach described in this article to generate alternative estimates of fuel poverty rates for Parliamentary constituencies – alternative relative to the current Parliamentary constituency-level estimates of fuel poverty published by BEIS.

This is achievable as the English Housing Survey (EHS) sample sizes for Parliamentary constituencies are bigger than those for MSOAs or LSOAs, and the methods for getting good small area estimates with levels of precision for Parliamentary constituencies have already been successfully applied by ONS in other contexts, such as the estimation of unemployment¹ . Equivalent estimates at the county level are also feasible based on the success at local authority level, although the methods to generate estimates of precision in this case may require further development.

Notes for: Analysis

1. See the following ONS publication: Modelled unemployment for local and unitary authorities

This research has investigated the ability to produce subregional estimates of fuel poverty in England using a different modelling approach than that currently used in the Annual Fuel Poverty Subregional Statistics Reports published by the Department for Business, Energy and Industrial Strategy (BEIS).

The research has shown that it is feasible to use this alternative approach to generate local authority-level estimates of fuel poverty that are statistically robust and generally aligned with the published estimates. The main findings from the analysis of these alternative local authority-level estimates of fuel poverty are detailed in this section.

The Office for National Statistics (ONS) estimates that the South East of England has notably lower levels of fuel poverty. These are similar patterns in parts to the BEIS estimates.

Overall, the levels of fuel poverty are broadly unchanged over time, within the period of interest. However, some improvements can be observed in the North East of England between 2013 and 2017.

In terms of the comparison with the published estimates, the ONS estimates are overall consistent with the published estimates, with the median average proportion of fuel poor households per local authority being estimated at 10% to 11% depending on the year of data and according to both sources. The range of the ONS estimates is slightly wider than that of the published estimates, indicating larger differences across local authorities for the estimated proportion of households in fuel poverty.

For the vast majority of local authorities, the published estimates fall within the 95% confidence intervals of the ONS estimates. The cases where the published estimates fall outside of the ONS estimated confidence intervals are clustered around either the highest levels of fuel poverty (where the ONS estimate tends to be higher) or the lowest levels of fuel poverty (where the ONS estimate tends to be lower).

The analysis shows some evidence that the small area estimation (SAE) approach used in this article may be a suitable alternative to the current method used by BEIS to generate local-authority-level estimates of fuel poverty. It also has the advantage of providing confidence intervals along with the estimates, which the current method does not provide.

User feedback will be sought to support future decisions around the most appropriate method to use in future. This will also need to consider the user demand for fuel poverty statistics at a lower geography level than local authority.

BEIS and the ONS are currently exploring the feasibility of alternative methods to produce subregional estimates of fuel poverty for Lower layer Super Output Areas (LSOAs). These include the use of machine learning techniques, which involve use of algorithms to predict the fuel poverty status of individual households, and/or use of alternative data sources such as data from the Energy Performance Certificate (EPC) database, which currently has a national coverage of 50% of all the addresses in England and Wales.

Using EPC data instead of data from the English Housing Survey (EHS), or in combination with EHS data, along with auxiliary data from other sources, may provide a better coverage at small area level, and potentially allow for more robust estimates of fuel poverty for smaller geographies. One of the challenges of using EPC data will be taking into account the under-representation of certain property types in data that are currently available. We will consider how to do this in our further research on estimating fuel poverty at the LSOA level.

We want your feedback

Your feedback is important. We want to hear what our users think about using small area estimation to estimate fuel poverty at the subregional level, such as:

Are the patterns of fuel poverty highlighted in this research output (both between local authorities and over time) plausible?

Do you find the additional information on the uncertainty (or quality) of fuel poverty statistics provided here useful?

As discussed in Section 6, small area estimation was unsuccessful for fuel poverty at the LSOA-level. However, the current method published by BEIS should not be used to identify trends over time within an LSOA, or to compare LSOAs with similar fuel poverty levels due to very small sample sizes and consequent instability in estimates at this level. Which is of most importance to you, improved local authority estimates with confidence intervals, or estimates at the LSOA-level that must be treated with caution?

Feedback relating to this research is welcomed and should be sent by email to fuelpoverty@beis.gov.uk.