Risk Management: How a Machine Learning Model Predicts Hypoglycemic Events in Hospitalized Patients

A new software model is the first to provide a rolling prediction about the next 24 hours after each blood glucose measurement is taken in hospitalized patients with diabetes. Through machine learning, the model integrated dozens of clinical predictors via links to electronic hospital records.

A new study could move hospitals a step closer toward the goal of minimizing hypoglycemic events among inpatients with diabetes. A recent paper in JAMA Network Open describes machine learning software that mined health record information to predict a patient’s risk of hypoglycemia within 24 hours of each blood glucose measurement.

Nestoras Mathioudakis, MD, MHS, clinical director of endocrinology, diabetes, and metabolism at Johns Hopkins University School of Medicine and lead author of the study, says that the study breaks new ground compared with previous attempts at predicting hypoglycemia because of its focus on a rolling prediction about the coming 24-hour period. Previous studies merely focused on the possibility of such an event at any point during their admission. The focus on the next 24-hour timeframe is much more clinically relevant in terms of adjusting treatment to try to prevent the event, he says.

“Most people with diabetes who are hospitalized are put on insulin, regardless of what they are taking in the outpatient setting. And insulin is one of the highest risk medications used in the hospital setting,” Mathioudakis says.

Kristen Kulasa, MD, director of inpatient glycemic control at the University of California, San Diego, agrees on the importance of better clinical tools to help reduce hospital hypoglycemia: “Electronic assistance to help with highlighting the high-risk patient would be very helpful because most patients are not here for diabetes specifically. They are here for something else but also have diabetes, which can often be number 10 on their problem list and doesn’t necessarily get a ton of attention.”

Four-Year Study

The study examined almost 55,000 admissions over a four-year period at five hospitals — two academic and three community hospitals — in the Johns Hopkins Health System. The prediction model extracted and analyzed 43 clinical predictors from the patient’s electronic medical record, including demographic characteristics, diagnoses, procedures, laboratory data, medications, and vital signs. After every blood glucose measurement, the model predicted the likelihood of a hypoglycemic event — defined as a blood glucose measurement of 70 mg/dL or less — in the upcoming 24 hours.

The model was nearly perfect in negative predictive value — predicting when there would be no event — and did “fairly well” in predicting a hypoglycemic event, Mathioudakis says. The strongest predictors of an event were the size of the basal insulin dose, the variability of the blood glucose results, and the existence of previous hypoglycemic events.

“Electronic assistance to help with highlighting the high-risk patient would be very helpful because most patients are not here for diabetes specifically. They are here for something else but also have diabetes, which can often be number 10 on their problem list and doesn’t necessarily get a ton of attention.” – Kristen Kulasa, MD, director, inpatient glycemic control, University of California, San Diego, Calif.

The study evaluated four kinds of classification algorithms and had the greatest success with a stochastic gradient boosting machine learning model, a type of machine learning algorithm in which the machine learns by sorting the data into random subsets to figure out which of the parameters are driving the risk in order to incorporate them into the predictive model.

“Further studies are needed to translate this model into a real-time informatics alert and evaluate its effectiveness in reducing the incidence of inpatient iatrogenic hypoglycemia,” the study authors write. “Our next step will be to embed the prediction model in our electronic medical record system.”

The Challenges of Implementation

These next steps will be challenging, according to Robert J. Rushakoff, MD, medical director for inpatient diabetes at the University of California San Francisco Medical Center, who has been working on methods to decrease hypoglycemia in inpatients for many years.

“Going from a prediction model to implementation to decreased hypoglycemia is a big jump,” he says. Several years ago, his institution tested one of the first health-record-based hypoglycemia prediction systems developed at Washington University in St. Louis. The model was programmed into the electronic medical record system and ran in real time. They found that the biggest challenge to such a model was that most cases of hypoglycemia at their institution were not predictable because they were the result of unanticipated interruptions in enteral feedings. These incidents generally involved patients on insulin who did not receive their feedings for various reasons — their tubes were clogged, the patient pulled the tube out, or the feeding was stopped to prepare the patients for extubation, but intravenous dextrose was not started, or their insulin dosage was not adjusted in tandem.

Another problem was in figuring out how to react to the predictions. “It was a lot of work to figure out who gets a message, how do they do they get a message, and during what hours. Do you do it at 3 in the morning after a 2 a.m. blood sugar check? The bottom line was, we didn’t continue with this project because the committee on alerts felt that we shouldn’t push out alerts when the benefit was so small. In our case, we just didn’t have a lot of hypoglycemia that was significant enough to make a difference with the alerts,” Rushakoff says.

But he practices in an academic center that expends a lot of resources on glycemic management, so he believes that community hospitals that lack these resources could see the largest benefit from a prediction system. “If you can give people a warning that the patient is at risk for hypoglycemia in the next 24 hours, and give guidance on changing the insulin dose, that would be extremely helpful,” he says.

A Challenge from CGM?

Another potential change in glycemic management in hospitals is the greater use of continuous glucose monitoring (CGM). The Food and Drug Administration has still not approved CGM for general use in hospitals, but its temporary approval during the COVID-19 pandemic has led to several studies showing it to be very effective in highlighting glycemic trend lines.

“Most people with diabetes who are hospitalized are put on insulin, regardless of what they are taking in the outpatient setting. And insulin is one of the highest risk medications used in the hospital setting.” – Nestoras Mathioudakis, MD, MHS, clinical director of endocrinology, diabetes, and metabolism, Johns Hopkins University School of Medicine, Baltimore, Md.

But Mathioudakis says that even CGM does not detect the risk until it is imminent. “It happens with such a short lead time that if you haven’t adjusted the insulin dose, there is nothing you can do about it except give glucose proactively. What is potentially more useful is being able to predict with a longer lead time, so you can decrease the insulin dose and prevent the drop in glucose from happening at all,” he says.

Next Steps

“We are excited by the next step in this research and translating it into a real-time electronic medical record-based prediction model. This is a National Institutes of Health-funded grant, so we have a lot of inputs from our stakeholders about what they would want the alert to do and the desired functionality,” Mathioudakis says.

Kulasa looks forward to the next steps: “With the right alert, this could be very helpful. There is definitely a clinical need for it, so it is good to see this progressing nicely.”

Seaborg is a freelance writer based in Charlottesville, Va. In the August issue, he wrote about the use of the term “prediabetes” as it applies to older adults.