Setting up an ‘implementation and governance framework’ for Artificial Intelligence (AI) pilot studies taking place in an NHS Trust

This case study explores the implementation and governance process created by a Consultant Radiologist working in breast screening at an NHS Trust. The Consultant Radiologist was interested in the use of AI technology to help clinicians triage and prioritise breast and chest scans. However, before undertaking any AI projects to look into the feasibility of using AI technologies, the NHS Trust where she worked recognised there was a need for a formal ‘AI implementation & governance framework’. This was to make sure that the set up of any projects using or piloting AI was efficient and effective and align all relevant stakeholders.

The problem and possible solution

There is currently a national shortage of radiologists, and record waiting times for cancer treatments. AI technologies that are ‘trained’ to look at scans are developed to ‘assess’ whether abnormalities in the scans matter. These AI technologies have the potential to help radiologists by highlighting the more urgent cases to look at, so the radiologist can prioritise the patients in most need of care, to get them access to treatment more quickly. It is the responsibility of the adopter to determine whether digital technologies are the right solution for providing safe and effective care.

See our best practice guidance for understanding if digital technologies or AI are the right solution to a problem.

Setting up an implementation & governance framework for AI studies

Discovery and evaluation

The Consultant Radiologist knew of AI imaging companies whose digital healthcare technologies could potentially help but decided that more thought was needed around how to set up AI technologies in the NHS Trust. The proposed AI implementation and governance framework would prioritise setting up a well thought out structure for implementation, which was to be agreed and formalised before the Trust embarked on any AI project set ups. As part of the discovery and evaluation phase, the team identified clinical priorities which aligned to the Trust’s existing vision and strategy.

Read our best practice guidance for adopters on thinking about whether a medical device will meet your needs.

Building the team

Stakeholders were brought on board, including the following professionals and teams in the Trust:

• clinicians
• information governance
• digital
• IT
• PACS
• procurement
• legal
• contracts
• research and development
• Integrated Care System

This multidisciplinary team laid the groundwork for effective collaboration and stakeholder engagement. Recognising the need for ongoing governance and oversight, the Trust established both an AI working group and an AI project management group.

AI working group

This working group convened every two to three months at first. The AI Working Group evolved to address the need for more frequent discussions on AI projects. Stakeholders from various departments participated, emphasising the importance of collaboration internally. The group comprised of:

• clinicians
• research finance
• digital services
• contracts
• information governance
• research AI lead,
• research and development operations manager
• clinical AI lead
• principal investigator
• clinical safety for IT systems

AI project management group

Comprising technical and clinical experts, this group met monthly to oversee and manage all AI projects within the Trust. The group comprised of:

• AI clinical lead
• AI research and development lead
• research and development associate medical director
• head of information governance
• chief information officer
• IT architecture
• PACS team
• clinical safety officer

Identifying opportunities and value in the use of AI

AI technologies were identified for potential pilots within the Trust, and it was important that they were strategically aligned to address existing challenges and priorities in the Trust. This stage concentrated on identifying opportunities for AI implementation and assessing its potential value for the clinical pathways, where there were existing challenges. Importantly this meant asking for suppliers to show how their technologies addressed these challenges with evidence, including providing their intended use statements as per their regulatory clearance. At this time, it was also possible to think about the potential risks involved with the identified AI technologies, including data privacy risks and various biases, such as, gender, or patient demographics or ethnicity. Suppliers were asked to share their training data set at the time of procurement to make sure that AI technologies would be appropriate for the diverse population within the ICS that the trust serves.

Planning for future AI projects

Acknowledging the need for funding and domain expertise, work began with the National Consortium of Intelligent Medical Imaging (NCIMI) and funding was granted. This meant that the team had access to a big portfolio of national AI research and development studies, and over the next two years, the team built their knowledge around how to set up local pilot studies involving AI. The funding also allowed the AI working group to build the infrastructure, both technically and in expertise, over this period of time.

Once the expertise was built, the intention to pursue more local AI projects was formalised. From there project plans were developed, and potential pilot studies were chosen due to their alignment with the Trust’s strategy and needs. The Consultant Radiologist told us:

Setting up a project plan and a dashboard for governance of AI studies

A formal ‘AI study set-up pathway’ was established to simplify the process and make sure AI project set up in the Trust was consistent. The AI specific set up was very much in line with how research and development would set up their usual studies for research purposes. There were some important differences which strengthened the governance of the AI projects, such as including ethical approval where appropriate. To help with the AI study set up, the following was developed:

Programme dashboard

A high-level program dashboard provided an overview of the status of all AI projects, to help monitoring and management.

Project plan

A detailed project plan was created for each AI study, documenting every aspect. Documenting each AI study in this way made sure there was good project oversight.

Risk management

An integrated risk management section was included in the pathway to proactively identify and mitigate potential risks associated with AI projects.

Benefits realisation

Benefits realisation made sure the trust concentrated on high-quality implementation, and formal closure and archiving of completed studies.

Piloting AI – the implementation phase

Process mapping

The Consultant Radiologist told us their first step was to do a comprehensive process mapping of the entire clinical pathway. This step was crucial for understanding the current state of data collection, identifying existing pain points, and envisioning the desired evolution of the pathway.

Accuracy testing and usability assessment

The next step involved a retrospective accuracy test of the AI technology. This step was indispensable to validate the tool's performance against existing patient demographics and imaging equipment that the Trust was using.

Usability testing involved exposing radiologists to the AI tool in a parallel setting alongside their regular workflow, which ran in silent-mode The goal was to observe interactions before, during, and after AI usage. This phase served as a pilot implementation phase, allowing for adjustments based on real-world experiences and feedback from the radiologists, who would be the primary end-users. The Consultant Radiologist told us:

Read our best practice guidance for adopters on piloting digital technologies in a health or care service.

Summary

This case study highlights the importance of setting up a comprehensive implementation and governance framework before embarking on piloting AI technologies in the NHS. The experience from the Consultant Radiologist that we interviewed provides valuable insights for other NHS Trusts thinking about similar AI piloting and implementation.