JAMA Network
About The Study: In this quality improvement study using a controlled simulation, commercial large language models (LLM's) demonstrated substantial vulnerability to prompt-injection attacks (i.e., maliciously crafted inputs that manipulate an LLM's behavior) that could generate clinically dangerous recommendations; even flagship models with advanced safety mechanisms showed high susceptibility. These findings underscore the need for adversarial robustness testing, system-level safeguards, and regulatory oversight before clinical deployment.
/Public Release. This material from the originating organization/author(s) might be of the point-in-time nature, and edited for clarity, style and length. Mirage.News does not take institutional positions or sides, and all views, positions, and conclusions expressed herein are solely those of the author(s).View in full here.
