Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

Major issues with UK employment data were brought to the attention of the head of the statistics agency after the sample size for one estimate “collapsed to only five individuals”, according to emails obtained by the Financial Times.

Richard Heys, deputy chief economist at the Office for National Statistics, warned in an email in October 2023 that there was “little and falling merit” in keeping the agency’s long-running labour force survey (LFS), adding that a data point for one industry had “move[d] by 30%” because of the sample size collapse.

The emails to senior figures at the ONS, released in response to a Freedom of Information request by the FT, came ahead of a briefing to ONS chief Sir Ian Diamond in October 2023, shortly before the agency pulled publication of key labour market statistics with just one day’s notice.

Diamond told the House of Commons Treasury select committee last month that the October 17 2023 briefing was the first time he became aware of the fragility and unreliability of the survey, which underpins critical UK economic statistics such as the unemployment rate.

In the 2023 email, Heys said a new survey under development by the ONS — the “transformed” labour force survey (TLFS) — was “presenting some challenging data” but still advised the existing LFS be decommissioned.

“There is little and falling merit in retaining the LFS and the key point isn’t when the TLFS is perfect, it is when it is clearly superior to LFS,” Heys wrote on October 12, 2023.

“As someone who had just seen one of their statistics move by 30% because the sample size for the industry in the LFS survey has collapsed to only five individuals . . . I think we are probably already there from my perspective,” he added.

The email did not specify which data was affected, but indicated that Heys was talking about one of the detailed industry statistics that is published alongside headline unemployment numbers.

Heys also described how officials at the ONS should present the issues to Diamond. “We need to be careful to sound positive but not Panglossian. Data quality and stakeholder feedback are non-trivial issues to hit SID [Sir Ian Diamond] with and it’s important we don’t sound like we [are] underplaying these,” he said in the email.

The correspondence provides an insight into decision-making at the beleaguered agency, which has come under fire from politicians for its “dodgy statistics” that have left policymakers with “major blind spots”.

The ONS’ LFS began to falter significantly during the Covid-19 pandemic when the ONS was forced to switch from in-person to telephone interviews. By the end of 2023, the response rate to the survey had plummeted to below 15 per cent.

Despite plans to switch to an online-first TLFS by March 2024, “systematic” under-investment and “recurring over-optimism” has significantly delayed the project, according to an ONS internal review.

Employment estimates are still based on the flawed original survey and the transition to the TLFS is not expected until at least 2026.

ONS presentation slides from October 2023 highlighted “nervousness” among key stakeholders about moving to the TLFS too soon, citing concerns about data quality.

The ONS did not comment.