Professor Jack JIANG, Padma and Hari Harilela Professor in Strategic Information Management and Director of Artificial Intelligence Evaluation Lab of HKU Business School
HKU Business School today released the results of a groundbreaking study evaluating how well today's leading Artificial Intelligence models can perform as independent foreign exchange traders.
Over the past year, AI large language models (LLMs) have rapidly evolved from simply generating text to taking independent actions and using complex tools. However, a key question remains: can these AI agents actually make profitable decisions in real-world, unpredictable environments?
To test this, the Artificial Intelligence Evaluation Lab (AIEL) at HKU Business School, led by Professor Jack Jiang, launched Agentic Trader, through which they evaluated the autonomous trading capabilities of LLM agents in live foreign exchange markets.
Key Findings
- Alibaba's Qwen produced the strongest profit.
- The range of cumulative net asset value across models varied significantly e.g. 9.9% for Qwen to -15.1% for Deepseek.
- Models that traded more frequently were not necessarily more profitable — higher activity did not guarantee better returns. Likewise, taking on greater risk did not necessarily lead to higher returns.
Implications
The findings from this live foreign exchange trading experiment do not fully align with the reasoning evaluation results previously released by the AI Evaluation Lab (AIEL) at HKU Business School. Models that excel in tasks such as reasoning, knowledge question answering, or code generation do not necessarily achieve the best performance in real financial markets. This finding also suggests that, as LLMs evolve from tools for answering questions into autonomous agents that continuously participate in real-world decision-making, traditional static benchmarks are no longer sufficient to comprehensively evaluate their capabilities. Future AI evaluation should place greater emphasis on models' long-term decision-making performance in dynamic and uncertain environments, providing a stronger foundation for the real-world deployment, continuous evaluation, and governance of AI agents.
The Ultimate Stress Test for AI
Unlike standardised benchmarks such as mathematics or coding tests, financial markets are inherently volatile. This makes live trading one of the most demanding real-world tests of an LLM's capabilities. To succeed, a model cannot simply generate a correct static answer; it must process live information, make high-stakes judgments, and execute trades within strict time frames, while continuously managing the consequences of its past decisions.
To conduct this rigorous evaluation, Agentic Trader connected 10 leading models directly to live foreign exchange data. The tested models were developed by major tech organisations, including OpenAI, Anthropic, Google, Alibaba, Moonshot AI, Zhipu AI, ByteDance, and DeepSeek.
Ensuring a level playing field, all 10 models started with the exact same initial capital and operated in the identical live market environment. The study specifically measured whether these AI systems could move beyond basic reasoning to make timely decisions, actively control risk, and dynamically respond to market shifts to produce tangible results.
"The findings suggest that performance gaps between LLMs become increasingly pronounced when they are required to continuously interpret changing environments and adapt their strategies in real time," said Professor Jack Jiang, Padma and Hari Harilela Professor in Strategic Information Management and Director of AIEL of HKU Business School. "Financial trading tests not only a model's ability to process information and reason under uncertainty, but also its capacity for risk control, position management, and sustained decision-making over time."
Click here to view the complete report.
