Join leaders in Boston on March 27th for a special evening of networking, insight, and conversation.request an invitation here.
By using multimodal large-scale language models (LLM), hospital systems can create powerful virtual physician assistants that proactively track and diagnose everything for patients, says one of New York's leading hospital systems. says the medical director. new york presbyterian church (New York).
Dr. Ashley Vesey, medical director of AI operations at NYP, spoke on VentureBeat. AI Impact Tour Events that took place in New York last Friday. She said her hospital system is already experimenting with generative AI in several discrete areas that provide value while minimizing risk, such as summarizing conversations from patient visits. . But she expects enthusiasm for generative AI to drive workflow changes that will allow hospitals to build powerful, comprehensive assistants that will „change the paradigm in which I practice.“
Multimodal LLM technology allows you to provide comprehensive, preventive care
Vesey, who is also a cardiologist at the hospital, declined to say when this would be possible, but said he expected to see progress over the next year. She says patients who experience chest pain are referred to her. But she said she wants to know if a patient is going to have a heart attack before it happens. “And using this technology and all the data that we're collecting about patients, we can gain insight from a multimodality perspective: imaging, echocardiograms, electrocardiograms, things that are probably invisible to me as a human. You can gain insight from things that don’t exist.’ But AI can and will allow you to act on events before they happen. ”
She said much of the technical capacity for this is in place, but it will take adjusting internal workflows and processes to make it possible, or what she called „change management.“ said it was important. She acknowledged that this would require a lot of work and testing, as well as the sharing of ideas by national healthcare organizations, as it would require larger structural changes beyond her hospital. She envisions a path for hospital systems to first tackle low-risk administrative use cases for generative AI, such as summarizing verbal conversations during patient visits. The system will then work on clinical diagnostics using generative AI, such as how to better detect heart disease in individual cases. Only then can she bring all these elements together into the more ambitious steps she envisions.
“What I would like to see is a model that can cover all of these things at once, and determine when the next patient is coming in and how long to schedule based on how long that patient is. Have you visited them in the past? What is the summary of all your visits since you last saw them? Do you need replenishment to be able to interpret it? Can you automatically enter that into your electronic data? Record it so you can order it. It's all over the workflow because all these tasks are brought together.“ (Her video below) (See full comment).
Veasey said New York City officials have so far largely accepted generative AI and appear to be willing to participate in its use. NYP is affiliated with his two medical schools, Cornell University and Columbia University, and has approximately 49,000 employees and affiliated physicians.
Workflows and processes still need to be resolved
In a conversation moderated by senior AI writer Sharon Goldman, Veasey said the hospital is leveraging AI capabilities (pattern recognition, summarization, data extraction, content generation, etc.) as low-risk, most important He said it is tailored to high-value applications. One of her personal favorites, she said, is to record patient visits and reduce the administrative burden on doctors by transcribing conversations during her visits into notes. Doctors have become what she calls „ambient scribes,“ working in front of computers and only occasionally seeing patients' faces. She can then spend hours at night transcribing notes manually. “We have to change that,” she said. Transparency is essential, she said, so hospitals must obtain patient consent to record their visits. But the results could be significant, since content no longer needs to be created and instead can be reviewed and edited by doctors.
From applications around administrative tasks like this, it's more difficult to use AI for clinical diagnostic applications, she said. But one of the use cases that NYP is evaluating is using electrocardiograms to detect whether there is structural heart disease. This is a problem with the heart valves or muscles and is usually diagnosed by an ultrasound of the heart. Although not everyone has access to such an ultrasound, many people have an electrocardiogram. This is a snapshot of the heart's electrical activity, and AI can also be used to detect heart disease. “We can screen people and get them the care they need sooner,” she said.
Risks abound, but excitement around AI abounds
When asked if she was concerned about the risk that the AI generated by these applications would make mistakes, she said that „there's a lot to figure out“ when it comes to the issue of risk, but that doctors should review their examination and diagnostic summaries. I answered that there would be no problem as long as it was. Most risks can be avoided. She said the technology is „not 50% yet because we don't use it.“ It's not 100% because it would replace all our work. Probably he is 90%, which is why I say that the provider will check in the end. ”
Another risk, she says, is over-reliance on technology. LLM technology is very advanced, and Beecy cited the progress her ChatGPT made in moving from her GPT 3.5 to GPT 4, but that has made it difficult for humans to become too complacent and for humans to get involved. could lose value, she said.
Vesey said NYP is taking a conservative and cautious approach to the technology, aligning two key stakeholders: those who want to deploy generative AI tools and those who will use them. It is said that he is doing so. She said there was some concern from employees about how it would be integrated into their workflow and what it meant for them, but she said they were „very excited.“ Yes, we have employees who would like to pilot it,” he added.
Generative AI is proving to be a democratizing force
In the past, Vesey said, technology tended to be pushed down from the top to the employees. But this is the first time the technology has truly been democratized in that NYP doctors and other providers have access to ChatGPT, she said. „They're able to use her AI, they're able to communicate with it, and they're actually able to come up with use cases that they think will be valuable.“ Being born helps with engagement and change management, she said.
She said the hospital is collaborating on a patient group survey to understand how transparent NYP needs to be about the technology. Questions include whether the patient wants to know each time the technology is used. Veasey said these are complex issues that require an interdisciplinary team, perhaps including sociologists and bioethicists.
Sarah Bird, Global Lead for Responsible AI Engineering at Microsoft, spoke in a conversation moderated by Sharon Goldman and myself, following Beecy's session. We asked her if Veasey's ambitious vision of a comprehensive physician assistant is soon possible, given the current state of her AI technology at Microsoft. (Microsoft works closely with her OpenAI to provide generative AI to enterprise companies.)
Bird suggested that this technology could provide the necessary building blocks for such an assistant, such as dividing flows into specific tasks and providing the underlying technology to provide access to trusted information. But she said one of her concerns with generative AI summaries is that the technology can add information or omit information that may be incorrect. Omitting symptoms from a doctor's visit summary can completely change the meaning of a diagnosis, she says. „We've been experimenting with techniques that allow our models to better understand medical information and actually summarize it effectively.“
Full disclosure: Microsoft sponsored VentureBeat's event stop in New York. AI Impact Tourhowever, NewYork-Presbyterian and Citi speakers were independently selected by VentureBeat. Check out the next stop on the AI Impact tour, including how to request an invitation to the next event. Boston on March 27th and Atlanta on April 10th.
VentureBeat's mission will be a digital town square for technical decision makers to gain knowledge and transact on transformative enterprise technologies. Please see the briefing.