Prof. Ping LUO is an Associate Director (AI Research and Tech Transfer) of School of Computing and Data Science (CDS), The University of Hong Kong
The Multimedia Laboratory (MMLab) at The University of Hong Kong (HKU) has spearheaded the development of "RoboDojo", an unified benchmarking platform designed to evaluate robotic manipulation across both simulated and physical environments. Co-initiated by Professor Ping LUO, Associate Director (AI Research and Tech Transfer) of the HKU School of Computing and Data Science (CDS), and his PhD student Mr Tianxing CHEN, this project was developed in collaboration with researchers from nearly 20 leading global universities, including the University of California, Berkeley, and Tsinghua University.
Embodied Artificial Intelligence (Embodied AI) represents the next frontier of robotics, aiming to equip machines with the ability to perceive, reason, and act autonomously in physical spaces. However, a major bottleneck in the field has been the lack of standardised evaluation metrics. Existing assessments are often confined to simulated environments or conducted under disparate hardware configurations and scoring criteria. Consequently, AI models that deliver impressive performance in isolated demonstrations frequently fail to exhibit the same level of reliability and robustness when deployed in complex, real-world scenarios.
Addressing this critical industry gap, RoboDojo seamlessly integrates simulation-based evaluation, standardised physical robot testing, and policy benchmarking under a single, cohesive framework. The platform currently encompasses 42 simulation tasks, 18 real-world robotic tasks, and 30 representative robot policies, rigorously assessing capabilities such as generalisation, memory, precision, and long-horizon task execution.
The benchmark's initial findings highlight a significant performance gap between current robotic systems and human capabilities. The top-performing AI model achieved success rates of only 8.80% in simulation and 12.8% in real-world testing, compared to 76.03% and 100% respectively achieved by human experts. These results underscore the urgent need for more robust AI models capable of executing complex, multi-step tasks in dynamic physical environments.
Professor Luo remarked, "To the best of our knowledge, RoboDojo is the first Hong Kong-led benchmark to unify simulation and standardised real-robot evaluation. It moves embodied AI beyond impressive demonstrations towards progress that can be measured, compared and trusted."
Since its launch, RoboDojo has garnered widespread acclaim from both academia and industry globally. The project's release generated over 100,000 views on X within its first week, while its open-source resources recorded more than 100,000 downloads on AI community platform Hugging Face. This rapid adoption underscores the global scientific community's strong interest in and demand for standardised evaluation tools.
By establishing a fair, transparent, and reproducible evaluation standard, RoboDojo is poised to foster deep academic-industrial synergy globally. The platform will drive the evolution of Embodied AI from proof-of-concept showcases to sustainable, real-world applications, paving the way for safer, more reliable, and highly adaptive robotic systems.
