the study played by Google joint research with Google Deep Mindthe tech giant revealed that it has expanded the capabilities of its Med-Gemini-2D, Med-Gemini-3D, and Med-Gemini Polygenic AI models.
Google said it used histopathology, dermatology, 2D and 3D radiology, genomics and ophthalmology data to fine-tune Med-Gemini's capabilities.
The company's Med-Gemini-2 was trained on traditional medical images encoded in 2D, such as CT slices, pathology patches, and chest X-rays.
Med-Gemini-3D analyzes 3D medical data, and Google trained Med-Gemini-Polygenic on non-image features such as genomics.
In this study, Med-Gemini-2D's sophisticated model outperformed previous results for AI-powered chest It became clear that this was the case.
The model also outperformed previous performance for chest X-ray visual question answering, thanks to enhancements to Gemini's visual encoder and language components.
It also performed well in chest X-ray classification and radiology visual question answering, outperforming previous baselines in 17 out of 20 tasks. However, in ophthalmology, histopathology, and dermatology, Med-Gemini-2D outperformed the baseline in 18 out of 20 tasks.
Med-Gemini-3D can read 3D scans, such as CT scans, and answer questions about the images.
This model proved to be the first LLM capable of generating reports of 3D CT scans. However, only 53% of reports were clinically acceptable. The company acknowledged that additional research is needed for this technology to achieve quality as reported by expert radiologists.
Med-Gemini-Polygenic is the company's first model to use genomics data to predict health outcomes.
The authors wrote that the model „outperforms standard linear polygenic risk score-based approaches for disease risk prediction and can be generalized to genetically correlated diseases for which it has not been previously trained.“ There is.
bigger trends
The researchers report the limitations of this study, and to ensure safety and reliability in real-world situations, the multimodal model must be optimized for diverse relevant clinical applications and extensively tested with appropriate clinical datasets. said that it needs to be evaluated and tested outside of traditional academic benchmarks.
The study's authors also note that „an increasingly diverse set of medical professionals will need to be deeply involved in future iterations of this technology to guide the model toward features that will have valuable real-world utility.“ He also pointed out.
There are many areas that future assessments should focus on, including bridging the gap between benchmark and bedside, minimizing data contamination in large-scale models, and identifying and mitigating safety risks and data bias. The field was mentioned.
“Although advanced capabilities in individual medical tasks are useful in their own right, we need all these capabilities to be integrated into a comprehensive system to perform a variety of complex multidisciplinary clinical tasks and collaborate with humans. „We envision a future in which patients can maximize clinical effectiveness and improve clinical effectiveness. The results presented in this study represent a step toward realizing this vision,“ the researchers wrote.