Inner Metaphysics of LLM

Dongguan Institute of Materials Science and Technology, CAS

The challenge

Large language models are increasingly used to explore scientific knowledge, analyze materials and generate ideas. Yet a correct answer alone does not reveal how a model arrived at it. The answer might reflect a physical relationship, a familiar phrase or a statistical shortcut.

Physical laws describe relationships between quantities. An increase in one quantity can increase a material's response under one law and decrease it under another. Testing whether an AI model represents that distinction requires more than examining its response to a single question.

The Solution

Markus J. Buehler, the Jerry McAfee Professor of Engineering at MIT, investigated this problem using three open-weight models from Google's Gemma family. Inspired by mechanics, he compared matched questions in which a physical condition changed while the surrounding context was controlled. He then measured how the model's hidden states, the internal numerical representations it creates while processing a question, moved in response.

Across 60 materials-science laws, the resulting changes tracked the ordering of direct, neutral and inverse relationships with a rank correlation of 0.910. They correctly oriented 39 of 40 directional laws. When equations and prescribed reasoning instructions were removed, the models still oriented 38 of 40 laws correctly at the smallest tested scale and all 40 at the two larger scales.

The study also tested whether scientific concepts could be read from intermediate hidden states when their names were omitted from the prompt. Some concepts remained recoverable, while others did not. This established a distinction between information that can be read from a model and information that is used in a decision.

To test causal use, the author modified selected internal representations without retraining the model or adding verbal hints. In one controlled test, a fixed direction associated with grain size shifted answer preferences appropriately across 12 grain-refinement and grain-coarsening cases. Further interventions across 24 relationships showed selective causal contributions across all three model sizes.

The effects were context-dependent: the same grain-size intervention did not transfer successfully to every new wording or physical regime tested. The experiments therefore distinguish three levels of evidence: readability, representation and causal use.

The Future

The next step is representation-aware scientific AI: training and evaluating models not only on their answers, but also on whether their internal representations respond consistently to changes in physical conditions. This opens a path to testing models against scientific relationships and identifying where they rely on shortcuts.

The present work establishes the measurements and interventions for selected, known relationships in controlled tasks. Extending them to new scientific problems and model architectures is the next challenge, including applications in developing enhanced training rewards for AI models that foster broader and deeper generalizations of physics principles. This can lead to new first-principles based AI models that think and reason from foundational concepts and that may be far more efficient and powerful than current models.

The Impact

By treating an AI model's internal representations as objects that can be measured and experimentally altered, this research brings an approach inspired by the physical sciences to the study of AI itself. It gives researchers a more precise way to ask what a model represents, what it uses and where its scientific reasoning breaks down.

Publication: M.J. Buehler, "Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model," AI for Science 2, 045001 (2026). https://doi.org/10.1088/3050-287X/aeb27b

Video: https://youtu.be/XtSQ7IXDQl8

/Public Release. This material from the originating organization/author(s) might be of the point-in-time nature, and edited for clarity, style and length. Mirage.News does not take institutional positions or sides, and all views, positions, and conclusions expressed herein are solely those of the author(s).View in full here.