Computer science and bioinformatics researchers at Stony Brook University are trying to expand doctors’ understanding of COVID-19 with a little help from artificial intelligence.
Prateek Prasanna, an assistant professor in the biomedical informatics department, is one of many researchers across the globe who are focused on putting machine-learning algorithms to work in the fight against the coronavirus. He is a doctor of biomedical engineering who came to Stony Brook in January 2020, and has previously used AI to study and analyze lung cancers. Prasanna arrived on the cusp of the coronavirus outbreak, and just four months later, he and his team were selected to receive seed grants from the university’s COVID-19 research program, a fund created to accelerate projects that study the virus.
They have spent the months since shaping computer brains to recognize and label certain features on COVID-positive lungs from X-ray images. Once these brains are properly trained, they will be able to detect COVID-19, and even offer a prognosis for the potential severity of each case.
“We saw that there might be these subtle differences in imaging signatures between, let’s say, a patient who is COVID-positive and does not required mechanical ventilation and another COVID-positive patient who needs mechanical ventilation,” Prasanna said on a Zoom call, his face superimposed on an image of Stony Brook University Hospital.
He spoke with a careful, measured cadence, but his excitement resonated as he discussed his findings. “So we hypothesize that there might still be imaging differences between these cases, and using machine learning and AI techniques, we would be able to tease out what these differences are.”
Prasanna’s team includes two researchers from Stony Brook, four from Newark Beth Israel Medical Center in New Jersey and one from the Indian Institute of Technology in Bombay, India. Their expertise also varies, from bioinformatics — the analysis of biomedical data — to electrical engineering and radiology. They began their study with 514 chest X-ray images gathered from Stony Brook and Newark Beth Israel Hospitals. Some showed the lungs of COVID-positive patients and some showed those of COVID-negative patients.
Each image included demographic data about its subject: age, the severity of the disease and underlying medical conditions, among other data points. That data, collected by medical institutions and prepped by the researchers, became what is known in the world of AI as the “ground truth” on which an artificially intelligent computer formed its own understanding of each set of lungs. After processing a number of human-sorted positive and negative cases, the algorithm was able to begin classifying them by itself, generating the probability of sickness for each image.
Machine learning, the basis for Prasanna’s study and thousands of others like it, relies on a computer processor’s ability to observe patterns and by recognizing them, to refine itself — in other words, to learn. In this case, minute features in COVID-positive X-rays — one is called “ground-glass opacity” or GGO — began to emerge as distinct characteristics that the trained algorithm, when tested blind, can use to predict the presence and severity of COVID.
The flexibility of the resulting algorithms allows the team’s research goals to shift with ease. For example, they began to examine the experimental practice known as “proning,” or having certain COVID patients lie on their stomachs instead of their backs during hospital care. Their existing machine-learning model reviewed X-ray scans from patients who were placed in prone positions and those who were not, demonstrating the benefits of the practice.
In May 2020, their research was recognized among 17 research studies to receive seed grant funding from the university’s Office of Research. “In very short order, I think in just a matter of a week, we redirected some funds that we would have used for other seed funding, and we created a seed funding program,” said Richard Reeder, Stony Brook’s senior vice president for research. “We got the funding to them right away … and the idea was that they would carry out research [and] position themselves to be able to put together a proposal for federal funding.”
Prasanna’s team has been using that time and funding to develop algorithms and submit papers to national organizations. So far, their work has been accepted by the 2021 Society of Photo-optical Instrumentation Engineers (SPIE) Medical Imaging Conference, the Journal of Clinical Medicine, the 2021 Medical Imaging with Deep Learning Conference (MIDL) and the Medical Image Computing and Computer Assisted Intervention Society (MICCAI). An additional manuscript is still in revision. “The initial results are quite promising,” Prasanna said. “Still, there’s still a lot of research to be done … before we can even start thinking about, let’s say using some of these studies as a triage mechanism to decide … which patients we need to pay more attention to.”
That research is expected to come as national institutes such as the American College of Radiology and the Radiological Society of North America release larger sets of images and data. These will allow researchers like Prasanna to further sharpen their algorithms, delineating positive diagnoses from negative ones and severe COVID cases from minor ones.
“God forbid, if there’s another sort of wave where you might not have enough testing… can we use imaging to diagnose and predict the trajectory of COVID?”
– Prateek Prasanna, assistant professor in Stony Brook’s Bioinformatics department
Of the first 514 chest X-rays Prasanna’s team used, 463 came from Stony Brook University Hospital’s own COVID-19 database. Known as the Data Commons and Analytic Environment, it was created to enable fast research that could result in medical use.
Dr. Kenneth Kaushansky, the former dean of Stony Brook’s Renaissance School of Medicine and senior vice president of Health Sciences, explained the need for such a database.
“Early on in the pandemic, I felt like we ought to have a single site with highly reliable scrubbed data on every patient we see. I think it took something like 40 people in the Department of Biomedical Informatics — graduate students, volunteers, students, medical students, staff and faculty — to set this up. And now people in radiology or image analysis or computer sciences can get into that database and extract the images, and all the other clinical history that goes with that particular image, and now use deep learning algorithms, AI as well, to sort out ways to better diagnose and prognosticate from images.”
It will likely be months before Prasanna’s algorithm is put to work in a medical setting. Although Prasanna sees his study going in many directions for practical use, one easy application he foresees is as a supplement for existing clinical testing.
“God forbid, if there’s another sort of wave where you might not have enough testing … can we use imaging to diagnose and predict the trajectory of COVID?” Prasanna said. “That, I think, is ripe for transition to clinic.”
In the meantime, Prasanna and his team continue to receive images from Stony Brook as well as medical centers in Newark, India and other areas, feeding the data through their algorithm — and honing its accuracy as it keeps learning.
