Tsinghua University researchers introduce FuelProp-LM, a novel framework combining instruction tuning and dynamic in-context learning to predict multiple fuel properties directly from molecular SMILES strings.
Predicting the physicochemical properties of fuels is a cornerstone of clean energy transition and high-efficiency engine design. However, many established Quantitative Structure-Property Relationship (QSPR) workflows involve property-specific model development and molecular descriptor selection. Incorporating now measurements may also require model updates or retraining.
In a new study published in ENGINEERING Energy , researchers from the Department of Energy and Power Engineering and the Center for Combustion Energy at Tsinghua University demonstrate how general-purpose open-source Large Language Models (LLMs) can be adapted for fuel property prediction through instruction tuning and in-context learning. The research team developed FuelProp-LM, a unified AI framework capable of predicting multiple complex fuel properties simultaneously using only a molecule's Simplified Molecular-Input Line-Entry System (SMILES) string as input.
Exploring the Potential of Smaller Language ModelsTo construct FuelProp-LM, the researchers fine-tuned four compact open-source language models with fewer than 10 billion parameters (including Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Phi-3.5-mini-instruct, and DeepSeek-R1-Distill-Qwen-7B) on 100,000 data entries sourced from the PubChem database. Notably, no task-specific fuel property data was used during the instruction fine-tuning stage, allowing the model to learn transferable structural reasoning capabilities.
To enhance contextual inference during prediction, the team integrated a dynamic retrieval algorithm using 166-bit MACCS molecular fingerprints and Tanimoto similarity search via the Hierarchical Navigable Small World (HNSW) algorithm. This approach automatically retrieves structurally similar reference molecules from updated property databases to form informative in-context prompts.
Across comprehensive evaluations on 10 critical fuel property datasets—spanning ignitability, sooting tendency, volatility, and thermodynamics—the fine-tuned 7-billion-parameter FuelProp-LM consistently outperformed the 685-billion-parameter industry-leading DeepSeek-V3.2 baseline. FuelProp-LM also compared favorably with several conventional machine learning baselines.
Key Research Highlights
- Unified Multi-Property Prediction: Eliminates the need for multiple independent QSPR models by predicting 10 essential fuel properties within a single language model framework using SMILES strings and retrieved in-context examples
- Superior Parameter Efficiency: Demonstrates that a fine-tuned 7B-parameter model can perform well on the evaluated fuel-property tasks without additional fine-tuning on the downstream datasets.
- Dynamic In-Context Learning: Employs MACCS fingerprint similarity retrieval (Tanimoto metric) to dynamically incorporate neighboring contextual examples, enabling real-time database updating without model weight modifications.
- Mechanistic Attention Insights: Attention layer analysis reveals that instruction tuning shifts the model's focus toward structurally similar contextual examples and essential chemical descriptors, specifically within the intermediate Transformer layers responsible for semantic integration and reasoning.
Gaining Insights into the Model's "Black Box"To explain why instruction tuning so dramatically improves QSPR accuracy, the researchers conducted an in-depth attention layer analysis. The results demonstrate that fine-tuning fundamentally reallocates internal attention patterns in the intermediate Transformer layers:
- Enhanced Focus on Top Reference: Attention shifted heavily toward the property description and the first in-context example (the structurally most similar molecule), rather than distributing attention uniformly.
- SMILES Feature Extraction: Attention to the target molecule's SMILES string increased, strengthening the intrinsic mapping between chemical structure and target properties.
- Reduction of Irrelevant Tokens: Attention scores assigned to non-essential prompt components (such as repetitive question text or less relevant context examples) decreased significantly.
"FuelProp-LM bypasses the tedious process of manual descriptor construction and task-specific model retraining," said Professor Bin Yang, corresponding author of the study. "By offering high accuracy, data scalability, and deployment ease, FuelProp-LM provides an accessible and flexible tool for future AI-assisted sustainable fuel design."
About the Research
Journal: ENGINEERING Energy
