Patients used NHS England (NHSE)’s 111 non-emergency phone service in record numbers during the Covid-19 outbreak.
Case study: 111 First
Faculty helps NHS England roll out ‘111 First’ safely with real-time testing
Get in touch
NHS England needed to understand if expanding its 111 telephone service would improve patient care, safely.
Summary
Operating under significant time pressure, NHSE needed to quickly understand the impact of proposed changes to the service. They partnered with Faculty, to create a new methodology for monitoring results in real-time. This customised system gave NHSE the confidence to roll out changes to 111 nationwide.
Future
Results
Execution
Ambition
To help maintain this uptake, and ease pressure on A&E departments nationwide, the team at NHSE launched 111 First – an initiative designed to establish 111 as the ‘first line of defence’ for the Urgent and Emergency Care (UEC) system.
NHSE chose a number of regions around the UK as trial sites for 111 First. Before rolling it out nationally, NHSE needed to measure the initiative's impact. But traditional evaluation methods - such as a six-month retrospective study or randomised control trial - would have been far too slow.
A faster, reliable form of measurement was needed. One that would have to account for the many variations in demographic, geographic, and temporal factors between the trial sites.
Faculty’s methodology showed the impact on patient care in real-time
NHSE turned to Faculty to quickly and definitively understand what the impact of increased use of 111 would have on patient care. The team combined the best of NHSE’s operational expertise with Faculty’s knowledge of cutting-edge evaluation techniques to define a new measurement methodology.
In typical trials, these changes would be assessed via a retrospective study, or by a randomised control test. For example, comparing the number of 111 calls in a trial region with those in a non-trial region. However, the many variations in demographic, geographic, and temporal factors between every 111 trial region made them unsuitable for these comparisons.
Faculty's solution was to embed the methodology into a custom dashboard. One that displayed results in real-time, giving policymakers immediate visibility and understanding of 111 First's impact.
The impact:
A pioneering methodology for robust and real-time results
By analysing patient attendance at A&Es, Faculty's solution proved that 111 First was correctly diverting low-risk patients and did not negatively impact patients in need of emergency care. Combined with the real-time monitoring dashboard, this gave NHSE the confidence to roll out 111 First beyond the trial sites.
This continuous evaluation method offers a route forward for other trials that would have previously been considered too risky. The success of Fauclty's methodology showed how public services can use cutting-edge evaluation techniques to better manage their performance.
Instead, we developed a multi-region ML model which adjusts for these geographical and temporal differences to select the most comparable control locations for any given metric in a trial site. Crucially, the model was not allowed to learn anything about the trial, allowing us to isolate the true effect of the trial. This was known as a “control prediction” or “synthetic control” - what would have happened if no trial had taken place.
Summary
Operating under significant time pressure, NHSE needed to quickly understand the impact of proposed changes to the service. They partnered with Faculty, to create a new methodology for monitoring results in real-time. This customised system gave NHSE the confidence to roll out changes to 111 nationwide.
Patients used NHS England (NHSE)’s 111 non-emergency phone service in record numbers during the Covid-19 outbreak.
NHSE chose a number of regions around the UK as trial sites for 111 First. Before rolling it out nationally, NHSE needed to measure the initiative's impact. But traditional evaluation methods - such as a six-month retrospective study or randomised control trial - would have been far too slow.
To help maintain this uptake, and ease pressure on A&E departments nationwide, the team at NHSE launched 111 First – an initiative designed to establish 111 as the ‘first line of defence’ for the Urgent and Emergency Care (UEC) system.
A faster, reliable form of measurement was needed. One that would have to account for the many variations in demographic, geographic, and temporal factors between the trial sites.
NHSE turned to Faculty to quickly and definitively understand what the impact of increased use of 111 would have on patient care. The team combined the best of NHSE’s operational expertise with Faculty’s knowledge of cutting-edge evaluation techniques to define a new measurement methodology.
NHS England needed to understand if expanding its 111 telephone service would improve patient care, safely.
Faculty’s methodology showed the impact on patient care in real-time
In typical trials, these changes would be assessed via a retrospective study, or by a randomised control test. For example, comparing the number of 111 calls in a trial region with those in a non-trial region. However, the many variations in demographic, geographic, and temporal factors between every 111 trial region made them unsuitable for these comparisons.
Instead, we developed a multi-region ML model which adjusts for these geographical and temporal differences to select the most comparable control locations for any given metric in a trial site. Crucially, the model was not allowed to learn anything about the trial, allowing us to isolate the true effect of the trial. This was known as a “control prediction” or “synthetic control” - what would have happened if no trial had taken place.
Faculty's solution was to embed the methodology into a custom dashboard. One that displayed results in real-time, giving policymakers immediate visibility and understanding of 111 First's impact.
The impact:
A pioneering methodology for robust and real-time results
By analysing patient attendance at A&Es, Faculty's solution proved that 111 First was correctly diverting low-risk patients and did not negatively impact patients in need of emergency care. Combined with the real-time monitoring dashboard, this gave NHSE the confidence to roll out 111 First beyond the trial sites.
This continuous evaluation method offers a route forward for other trials that would have previously been considered too risky. The success of Fauclty's methodology showed how public services can use cutting-edge evaluation techniques to better manage their performance.
How could your organisation benefit from real-time results measurement reliable enough for life-or-death decisions? Let’s start a conversation.
Talk to an expert
How could your organisation benefit from real-time results measurement reliable enough for life-or-death decisions? Let’s start a conversation.
Get in touch