Establishing Healthcare Workers’ Confidence in AI

Background

The report on developing healthcare workers’ confidence in AI from the NHS AI Lab and Health Education England states that, to develop confidence in AI among healthcare workers: ‘the challenge is to provide the right resources to the right people. This means providing a solid foundation for developing AI-related knowledge as well as personalised advanced educational elements to fit the needs of individuals in different roles and responsibilities’.

This case study demonstrates Articulate Pro’s approach. This project provided training and education in AI software use, and knowledge sharing and validation activities to various staff groups. These included pathologists, clinicians, statisticians, and IT deployment and software engineering teams. In terms of the archetypes set out in the report, the project’s multidisciplinary team included the following archetypes: Driver, Creator, Embedder and User.

Articulate Pro

Articulate Pro, a 30-month project, received funding from the NHS AI Lab through an AI in Health and Care Award. The project aimed to evaluate the deployment of AI in the prostate cancer pathway by using an AI-based clinical decision support software, to assist pathologists when reading prostate biopsies in live clinical workflows. There were 4 phases to the project, which essentially constitutes a ‘before and after’ evaluation at 3 clinical sites.

This case study includes examples of activities conducted during the project, but is not an exhaustive list.

Types of training

In considering the report’s ‘suggested education and training approach’, the project offered both ‘advanced’ training, to enable the development of skills and capabilities in the clinical sites, and product-specific training.

The project team produced an education and training plan and a one-page flyer to encourage staff buy-in from clinical, IT and pathology laboratory staff at the sites.

The project also engaged in activities that promote knowledge sharing and collaboration across the project sites, including:

• Discussions on Baseline measurements including Gleason benchmarking and consensus grading exercises for pathologists. We also did an audit of pathology reporting practices at the 3 sites.
• Statistics discussions including endpoints, discrepancies, statistics analysis.
• Knowledge-sharing meetings for technical deployment activities.
• A demonstration of the AI software providing greater insight into the system and a pre-cursor to formal training.
• Knowledge sharing on Data Protection Impact Assessments and governance requirements.

The project team hosted face-to-face workshops and meetings for all team members, recognising the inherent value of in-person learning. However, they also offered meetings with hybrid online capabilities to include the widest audience possible. This was particularly beneficial for clinicians in the team who, because of their clinical duties, were not able to step away from their work environment for lengthy periods of time and for Paige colleagues, most of whom are based in Europe and the US.

Examples of these events included two Health Economics (HE) Workshops, because HE modelling is an important deliverable of the project and may show cost savings or other cost benefits for the NHS. The team also held a Safety meeting investigating the safety aspects of using AI in the pathology workflow and an Away Day to discuss strategy, publication policy, and to plan actions for upcoming milestones and deliverables.

Product-specific training

Step 3 in the report’s suggested education and training approach is ‘provision of product-specific training for users of each AI technology during its deployment’. Our training was done in a step-wise approach as each site entered each of the 4 phases. Some training was not phase dependent and was offered to all sites at the same time.

Note from editor: It is worth noting that if user training is specified by the legal manufacturer of a medical device, then delivery of this training may fall under organisation-led product specific training. This makes sure the requirements set by the manufacturer are met.

Technical deployment and implementation is key for enabling use of the AI software in the digital pathology pathway. This is often complex and technically challenging and demands integration with existing IT and Laboratory Management Systems. Our approach was to support all relevant staff across the clinical sites by implementing monthly technical workstream meetings with team members from each site and the Paige technical team. Technical Implementation Charters were issued to each site. The pathologists were given access to a ‘Sandbox’ training and discovery environment for the AI software, together with specific training provided by the Paige team.

Site-Specific Training was provided to each site including AI product-specific training to pathologists, training on how to use the data-capture tool, a custom-built database, and training and guidance from the lead pathologist and Principal Investigator on data fields and data requirements. There were also guidance notes on system use and data fields available for each project phase.

Trainees

Histopathology trainees, a key part of the project, had different education and training needs from the more mature pathologists on the team. The project team set up a training environment specifically for pathology trainees together with a protocol for data capture. Trainees took part in all training and other activities, recognising that their views are hugely important in terms of skills development and avoiding the risk of deskilling as more pathology laboratories turn to digital pathology and the use of AI in the diagnostic pathway.

Site visits

The project team did face-to-face site initiation visits, with members of the team providing business intelligence and IT and technology support. Each visit provided opportunities for in-depth discussions and answering queries on the project, technical requirements and training. This led to a series of actions for follow up. These on-site visits were key in securing site buy-in and helping to move the project forward. The main issues which arose were around how to categorise data, how to use the software platform, safety and security, governance, and deployment and integration into existing laboratory systems.

Summary

The report states ‘a collaborative effort between industry innovators and NHS staff in health settings will enable product-specific training to better reflect the local workflows and clinical settings and meet NHS user needs’.

The information, advice and training provided within the project to the multidisciplinary team has enabled members to upskill as appropriate, and enabled the successful implementation of the AI software in the clinical pathway at 3 sites. The project team took a step-wise approach to training, sometimes driven by the need to answer product or workflow specific questions that entirely reflect the differing local workflows at each site.