Around the globe, artificial intelligence is increasingly reshaping human systems of all kinds, and healthcare is no exception. How we manage such change will determine whether we are able to fully exploit these new capabilities to improve the health system or allow them to remake the system in undesirable ways.
This question lies at the heart of a new article by PCHSS investigator Professor Enrico Coiera. Published this week in The Lancet, “The cognitive health system” addresses the interactions of human and machine actors in emerging “cyber-social” health systems. He writes: “Humans populate our sociotechnical system; we create technology and in turn are shaped by it. Technical systems have social consequences, and social systems have technical consequences.”
Professor Coiera cites the functioning of social media platforms as examples of cyber-social systems that are created, administered, and driven by human beings, but which can nevertheless behave in unpredictable ways. He discusses how vulnerabilities within these systems can result—both unintentionally and due to malicious actors—in negative outcomes, such as the amplification of anti-vaccination messages in “algorithmically targeted ways” within online networks. In this sense, an “online contagion of false beliefs leads to real world contagion of infectious disease”.
Further, he addresses the subject of a “learning health system”, in which data that is captured through routine clinical care as a matter of course is used to generate new medical evidence and fed back into the system to improve treatment. While this aspirational vision holds great promise, there are also potential pitfalls. As Professor Coiera explains:
“Managing changes in the health-care system has long been a challenge of juggling scarce resources, sunk costs, and vested interests. Faster shifts to new best practices, enabled by machine learning, also mean faster obsolescence of past practices, challenging our ability to plan for and manage change. Stockpiles of suddenly out-of-favour medicines could become unwanted, resulting not just in waste, but economic cost for manufacturers. Shortages would arise if there was no capacity to increase production to meet unexpected demand. Professionals might find that the demand for their skills disappears or quickly becomes greater than they can manage. People with the most to lose are likely to resist change, as we see with some industry responses to climate change. The pressure to accelerate adoption of new practices might see traditional safety checks set aside.”
Nevertheless, if thoughtfully and robustly administered, the cognitive health system could dramatically improve healthcare, and in this connection, Professor Coiera outlines a variety of strategies to manage these new capabilities. To be sure, by employing machines to gather quantities of medical data and take on tasks that would be beyond the capacity of human beings, future health systems have the capacity to dramatically “accelerate the discovery of new treatments, workflows, and personalisation of care”.
Coiera E. The cognitive health system, The Lancet, 2020.