We also identified similar challenges in the health and education sectors. From this we’re able to learn from what has (and hasn’t) worked in the health sector.
1. Trust and privacy
Current forms of AI demand huge amounts of data. Finding a balance between reaping the benefits of AI and ensuring trust and privacy is tricky.
This issue was illustrated in healthcare, when the Royal Free NHS Foundation Trust signed a 5-year deal with DeepMind. The collaboration required transfer of real patients data and an ICO investigation found that the Trust did not comply with Data Protection Act.
What surfaced from the DeepMind fiasco is that ultimately users (in this case, patients) need to know about how their data is going to be used.
There needs to be a tailored approach to trust and privacy concerns in education, to avoid a blunder, similar to DeepMind. For example, AI in the education sector involves the data of pupils who are minors, hence it would be important to come up with a robust structure for responsible data sharing.
So, what can we learn from the health sector context that might be useful for schools?
The Lords’ AI report recommends that NHS Digital and the National Data Guardian for Health and Care construct a framework for data sharing. Such a framework could ensure a higher level of public trust, if it clearly sets out mechanisms that allows patients to understand how their data is going to be used. A similar, robust framework of data sharing and processing, perhaps overseen by an accountable public body, could possibly be implemented in the education sector.
2. Public value and competitiveness
NHS data is extremely valuable. It is a unique source of value for the nation, with data going back decades. This creates a tension. If data can be shared safely with companies leading to improved health outcomes, there are obvious benefits. Equally, if this data is only shared with a small number of companies and has the ability to generate eye-watering profits for them (since 2015, VC investment in AI for healthcare has soared, reaching almost $1.3 billion across 103 deals in 2017) rather than the public, there are obvious downsides.
Sensitive health data should not be shared lightly and should be handled in a way that ensures value is recouped by the public. Small and medium-sized enterprises (SMEs) should also have access to health data, to avoid a monopoly status of big tech corporations and ensure competition within the Healthcare AI space.