Predictive data and experiential knowledge in the neonatal intensive care unit

by Claire Maiers

The advancement of predictive data analytics into areas such as medicine, policing, or education has been greeted with both enthusiasm and concern. Where advocates stress a hopeful narrative in which data analytics will allow scientists to solve some of the society’s most difficult problems, critics worry about the potentially narrow and reductive depiction of the world generated by data analytics and about negative outcomes of allowing analytics and algorithms (rather than seasoned experts) to direct decisions.

In a recent article, I explored this tension between experiential and data-driven knowledge by examining the use of predictive analytics in a neonatal intensive care unit, or NICU. I argue that data analytics filter into decision-making through an interpretive process. Although highly dependent upon institutional context, this process may buttress, rather than simply replace, experiential knowledge.

The neonatal intensive care unit as a site for predictive data analytics

For this project, I spent a year conducting observations and interviews with a medical analytics team that develops predictive algorithms and with the clinicians that use these predictions as they care for premature and sick infants in the NICU.

Nurses and doctors in the NICU face challenging knowledge problems. With these tiny patients, distinguishing between signs of illness and the more benign symptoms of premature infants can be difficult. As Dr. Walker shared with me, symptoms that might be concerning in older patients, are often just “standard things for premature babies.” How then, do doctors and nurses know when to make a medical intervention such as prescribing antibiotics?

A predictive monitoring system called Horizon is designed to ease this challenge. The algorithm contained in Horizon is meant to predict the likelihood that infants will develop an infection within the next twenty-four hours. Despite its intent as an early warning symptom capable of alerting clinicians to infections before the onset of other signs of illness, it became clear that clinicians do not always treat these predictions as determinants of a baby’s condition. Instead, they rely on two interpretive processes—what I call conditioned reading and accumulative readings—as they negotiate between two different modes of knowing.

How clinicians know if a baby is sick

In keeping tabs on a baby’s health, nurses and doctors assemble a large constellation of information. Each day, the medical team reviews a long list of (mostly) quantifiable information such as gestational age, how often they are feeding and how much they ate, how many times they have urinated and the number of stools they have had in 24 hours, white blood cell count, and measures of various nutrients. In addition, they discuss observations such as the baby’s physical appearance, mood, and reactions to various stimuli. From these pieces of information, the clinicians construct a depiction of an infant’s health.

They do so within the institutional context of evidence-based medicine. Evidence-based medicine is currently promoted as the proper way of making treatment decisions within mainstream American hospitals.

According to the guidelines of evidence-based medicine, studies based on randomized controlled trials are seen as the most legitimate way of determining if a particular treatment is effective. These trials rely upon metrics, the numeric operationalization of phenomena, and statistical methods to provide evidence. As a result, the values of evidence-based medicine contain an outlook in which quantifiable and measurable information is often seen as more powerful while other forms of information, such as narrative and qualitative descriptions, are seen as somewhat suspect.

Doctors and nurses within the NICU are strongly oriented toward these tenets of evidence-based medicine. They find measurable evidence of illness to be the most compelling way to know if a baby is sick or well. As Jenn, a nurse practitioner explained,

It’s fair to say that I think they would want to have some sort of lab work to back it up. We’re very numbers oriented and we want growth, we want a bacteria, you know. Something to say, ‘Oh, yes, this baby actually is sick.’

And yet, I found that the clinicians within the NICU also relied upon an alternative way of knowing. Though it was not as explicitly recognized by the nurses and doctors themselves, experiential knowledge plays a key role in determining the condition of a patient. This knowledge may come from years of accumulated experience in the NICU or from continued experience with individual patients.

For example, when I asked Terry, a registered nurse, what causes her to suspect that a baby might be sick, she said that “they just don’t—they don’t act right. A lot of them don’t act like themselves.” This kind of familiarity with an individual is so valuable that doctors often consult nurses and even parents (who spend considerably more time with the infants than doctors do) for opinions on a baby’s condition.

How Horizon’s predictive risk scores are interpreted

It is within the context of these practices and approaches to producing knowledge about patients that clinicians integrated Horizon’s predictive risk scores into treatment decisions, leading to the interpretive processes I call conditioned reading and accumulative reading.

Through conditioned readings of Horizon, clinicians temper, discount, or place trust in its output. Clinicians determine the usefulness of Horizon based upon their past experience of individual patients. This comes across clearly in a scenario that Anthony described:

If I know a baby really well, and I get report from a nurse who has never had the baby, and they’re like, ‘Oh my gosh, their Horizon is 3.’ And I’m like, ‘Oh no, it’s goes to 3 every night, don’t worry, it’ll come back down.’ Like kind of like that. We kind of learn their trend, I think.

This process of tracking Horizon scores over time and the resulting changes or lack thereof in a baby’s health allows clinicians to establish a baby’s version of “normal” and temper Horizon’s predictions. This can lead them to distrust Horizon, meaning that it drops out of the constellation of information through which clinicians construct knowledge about patients.

When trusted, Horizon does not simply dictate care, but contributes to decisions about care and treatment through a process of accumulative reading.

In accumulative readings, Horizon’s predictions are layered upon, and sometimes used to assess the meaning, of other information. As a result, in contrast to its intended use as an early warning, Horizon often helps to reinforce or dissuade existing suspicions of infection. Instead of describing experiences where the Horizon score was the first and primary indicator of illness, clinicians were more likely to describe instances in which they noticed Horizon after the onset of other symptoms or used it as a check on other troubling, but inconclusive signs. When they discovered that the Horizon score predicted the infant would get sick, clinicians would start antibiotics. If the Horizon score indicated no impending illness, they might decide not to treat the baby.

Predictive data can work with experiential knowledge

Ways of knowing are context dependent. Horizon functions as it does due to its location in a setting in which numbers and measurements are seen as the ultimate indicator of reality despite a frequent, though less explicitly recognized, reliance on experience and information that cannot be quantified. As a result, through the process of conditioned and accumulative reading, clinicians sometimes use Horizon to buttress non-measurable or non-quantifiable indications of illness. This is an important part of the means by which clinicians convince not only others, but also themselves, of the condition of a baby.

This case suggests that similar studies are needed of other settings in which data and algorithms are integrated into knowledge production and decision-making. What factors might facilitate conditioned and accumulative readings in other professions? How do these processes influence the production of knowledge claims, and how do they connect to experiential knowledge and expertise? Such investigations will allow scholars to develop a general framework for the means by which users turn analytics into knowledge and action.

Claire Maiers is a Ph.D. candidate in sociology at the University of Virginia. This article summarizes findings from “Analytics in Action: Users and Predictive Data in the Neonatal Intensive Care Unit” in Information, Communication, and Society

Image: Kaveri appala via Wikimedia Commons BY-SA