Developers - Planning for evidence generation

Plan for evidence generation that proves your digital technology is safe, and clinically and cost effective.

Evidence generation to support your digital technology

As early on as possible, you should think about what evidence you will need to prove to assessors that your technology is useful.

Evidence is information on which a decision or guidance is based. You can gather evidence from a range of sources, usually through studies and research. This could include randomised controlled trials or observational studies.

Well-planned and implemented evidence generation will help you smoothly navigate regulations and commercial processes. Make sure you do this throughout the technology’s lifecycle.

Plan to get the most out of evidence generation

You should plan to generate evidence that supports and is consistent with the intended purpose statement and value proposition of your digital technology.

You will need evidence to prove compliance with medical device regulations. You will also need evidence to prove to commissioners and health technology assessors (such as NICE) that your technology would be useful and worth adopting.

Planning makes sure you get the most out of your evidence generation and avoids duplication of effort.

For example, some regulators and commissioners may have the same evidence requirements. Or 1 piece of evidence may answer more than 1 question. You may be able to demonstrate safe use with users while capturing time savings in the same study. This could support 2 aspects of your value proposition.

Lack of planning could result in:

• disorganised or unfeasible evidence generation plans
• generating evidence of limited applicability to the UK health and social care system

This could:

• delay market access
• result in your technology no longer being financially viable to develop
• limit interest among adopters to buy the technology

Also think about how evidence-generation activities will change throughout the technology’s lifecycle.

For example, to secure compliance before placing your technology on the health and social care market, it is key to have evidence that meets regulatory requirements for:

• safety
• performance
• usability
• interpretability, and
• interoperability

This also means planning to generate evidence on the most recent version of your digital technology.

How to plan for evidence generation

Identify all the specific evidence requirements from regulators, commissioners and health technology assessors. Plan how and when you need to answer these questions, and how much detail will be needed. You should also build in time to determine the most suitable study designs for evaluating your digital technology.

Remember you might need approvals for some steps of your evidence generation. This is particularly the case if you are doing research, including qualitative research. For example, you may need approval to do a clinical investigation of your medical device. This would be to meet clinical performance requirements.

For health technology assessors and commissioners, generate evidence in line with evidence standards and methods guides from NICE:

• NICE's health technology evaluations manual
• NICE's evidence standards framework for digital health technologies
• NICE’s real-world evidence framework

Think about using NICE Advice. You’ll gain expert feedback and actionable advice on your evidence generation plans, helping to demonstrate the impact of your product to NHS decision makers.

Review the Digital Technology Assessment Criteria tool. Adopters use this to determine whether evidence is sufficient to consider buying a technology. 

Think about the resource requirements for each study including:

• funding sources
• time requirements
• collaborative partners

The MHRA’s AI-Airlock will provide a regulator-monitored virtual area for developers to generate robust evidence for their advanced technologies. This will be launched in April 2024.