It’s time to explore the limitations of traditional evaluation methods in digital healthcare says Professor Kathrin Cresswell
CREDIT: This is an edited version of an article that originally appeared in Digital Health
As the NHS navigates mounting pressures – rising demand, workforce shortages and funding limitations – digital technologies have emerged as a beacon of potential. From AI-assisted diagnostics to virtual consultations, these innovations promise greater efficiency and improved patient outcomes. Yet, despite their growing adoption, evidence of any real boost to system-wide productivity remains limited.
This disconnect suggests a fundamental issue: the tools and metrics we use to measure the impact of digital healthcare technologies may not be fit for purpose.
Why Traditional Evaluations Fall Short
The conventional approach to impact evaluation in healthcare tends to focus on easily measurable, short-term outputs, like time saved per appointment or reduction in paperwork. While these metrics offer a snapshot of immediate benefits, they miss the deeper, more meaningful transformations digital tools can bring.
True digital transformation isn’t just about automation – it’s about changing how people work, how systems interact and how services are delivered. These changes are often slow to emerge, difficult to quantify and highly context-dependent.
Beyond the Numbers: Capturing Transformational Benefits
The real power of digital innovation often lies in intangible or indirect improvements. These can include:
Reducing the physical space needed for records by digitising paper notes
Shifting clinical workflows in ways that free up staff time for more valuable tasks
Enabling cross-organisational collaboration that would be impossible with legacy systems
However, these benefits are typically gradual and can go unrecognised if we rely solely on traditional evaluation methods designed for linear, predictable outcomes.
A New Approach to Digital Evaluation
If we’re serious about digital transformation and not just digital substitution, then we need to rethink how we measure success. For example, measurements that are:
Contextual: Recognising that organisational structure, clinical practice and local culture all influence outcomes
Flexible: Designed to capture both anticipated and unanticipated benefits
Longitudinal: Tracking changes over years, not just months
Collaborative: Involving all stakeholders, from IT teams to frontline clinicians and patients
Crucially, new evaluation frameworks must acknowledge that the benefits of digital technologies are often emergent and systemic. They don’t always follow a neat cost-benefit timeline, and their full value may only become clear with time, use and adaptation.
To unlock the full potential of digital health tools, we must move beyond outdated evaluation models that prioritise speed over substance. Instead, we need smarter, more holistic methods that reflect the complex, dynamic nature of healthcare systems.
Be the first to comment