25 June 2026
https://www.nature.com/articles/s41598-026-45593-z
Abstract
The automation of chemical research through self-driving laboratories (SDLs) promises to accelerate scientific discovery, yet the reliability and granular performance of the underlying AI agents remain critical, under-examined challenges. In this work, we introduce AutoLabs, a self-correcting, multi-agent architecture designed to autonomously translate natural-language instructions into executable protocols for a high-throughput liquid handler. The system engages users in dialogue, decomposes experimental goals into discrete tasks for specialized agents, performs tool-assisted stoichiometric calculations, and iteratively self-corrects its output before generating a hardware-ready file. We present a comprehensive evaluation framework featuring five benchmark experiments of increasing complexity, from simple sample preparation to multi-plate timed syntheses. Through a systematic ablation study of 20 agent configurations, we assess the impact of reasoning capacity, architectural design (single- vs. multi-agent), tool use, and self-correction mechanisms. Our results demonstrate that agent reasoning capacity is the most critical factor for success, reducing quantitative errors in chemical amounts (nRMSE) by over 85% in complex tasks. When combined with a multi-agent architecture and iterative self-correction, AutoLabs approaches expert-authored reference procedures on the benchmark (F1-score > 0.89) on challenging multi-plate syntheses. These findings establish a clear blueprint for developing robust and trustworthy AI partners for autonomous laboratories, highlighting the synergistic effects of modular design, advanced reasoning, and self-correction to ensure both performance and reliability in high-stakes scientific applications. Code: https://github.com/pnnl/autolabs
Introduction
Recent advances in artificial intelligence (AI)1,2,3,4,5,6,7,8 and laboratory automation9,10,11,12,13,14,15are fundamentally transforming chemical research, promising to accelerate discovery through autonomous experimental design and execution11,14,16,17,18,19,20,21. Traditionally constrained by manual processes, low throughput, and complex hardware integration challenges, chemical experimentation is now poised for a revolutionary transformation driven by the emergence of agentic AI systems powered by large language models (LLMs)22,23,24,25,26,27. The ultimate vision of fully autonomous self-driving laboratories (SDLs)28,29,30,31 encompasses the entire experimental lifecycle: from hypothesis generation and literature mining through experimental design, robotic execution, real-time monitoring, adaptive optimization, and results analysis and interpretation32,33,34,35,35,36,37,38,39. Recent systems have made significant strides toward this vision. The Coscientist system employs a GPT-4-based “Planner” to autonomously decompose synthesis goals, retrieve hardware documentation, and dispatch commands to cloud laboratories and liquid handlers23. Chemist-X uses retrieval-augmented LLM reasoning with on-the-fly program synthesis to optimize reaction conditions and control robotic wet-lab experiments40. Similarly, ORGANA is an assistive robotic system that automates diverse chemistry experiments using LLMs for task interpretation, visual monitoring, and feedback. It supports parallel experiment scheduling and 3D object perception, and it has been demonstrated on tasks like solubility screening and electrochemical characterization, reducing chemists’ workload while maintaining experiment quality41. Current automated chemical experiments span diverse applications including molecular synthesis, reaction optimization, catalyst discovery, materials screening, and functional characterization42,43,44. Despite these advancements, the application of LLM-based agents in high-stakes scientific domains presents challenges that go beyond proof-of-concept demonstrations.
The current evaluation methodologies for such agents tend to focus on holistic task success, often bypassing the granular examination of the agents’ underlying functionalities. This lack of systematic evaluation obscures an understanding of their failure modes, reliability, and areas for improvement. Addressing this gap is of paramount importance as these systems are deployed in domains where precision and trustworthiness are non-negotiable.
In this work, we present AutoLabs, an end-to-end LLM-based agentic system for automated chemical experiment design for Unchained Labs’ Big Kahuna high-throughput liquid handler45. AutoLabs engages users in natural-language dialogue, plans experimental steps via a modular LLM-agent architecture, performs stoichiometric calculations through tool-calling, self-validates the generated protocol, and finally outputs the XML hardware file that drives the robot. We validate AutoLabs across five benchmark experiments of increasing complexity, ranging from calibration-sample preparation to multi-plate timed synthesis, and evaluate it under three levels of human-in-the-loop collaboration (none, non-expert, and expert). We design quantitative metrics to measure the correctness of the inclusion and ordering of the generated procedural steps as well as the correctness of the final vial contents resulting from the procedure. Both these quantitative evaluations and a qualitative analysis by an expert systems engineer show that AutoLabs can produce procedures that closely approach expert-authored references on the benchmark, substantially lowering the barrier for automated experimentation.
The core AutoLabs architecture—comprising multi-agent orchestration, stoichiometric reasoning, vial arrangement, processing step identification, and self-validation—operates on platform-agnostic procedural representations (Add, Set, Transfer operations with chemical identities and quantities). This design could be adapted to other liquid handling platforms (e.g., Opentrons, Tecan) through modifications to system prompts (specifying platform capabilities and constraints) and output formatting. However, our empirical evaluation is conducted exclusively on Big Kahuna, using platform-specific hardware constraints and XML file generation, and should be interpreted as a case study on this specific system.
To ensure reliability and prevent deviations from expected behavior in laboratory environments, AutoLabs incorporates several robust guardrails. First, the system employs self-correction and validation mechanisms: dedicated self-check nodes and reasoning steps iteratively review and refine experiment steps before final output, minimizing procedural and quantitative errors. Second, curated Python functions are accessible for chemical operations, to ensure accurate stoichiometry and reduce manual mistakes. Third, AutoLabs prompts users to confirm ambiguous details (such as chemical purity or solution concentrations) before proceeding, preventing misinterpretation of instructions. Fourth, all outputs are validated against platform-specific hardware constraints (e.g., Big Kahuna XML requirements), ensuring compatibility and preventing unsafe or unsupported commands. Fifth, the architecture supports iterative user-agent dialogue, allowing users to review, correct, and approve experiment steps prior to execution. Finally, comprehensive logging and monitoring are implemented for traceability and post-hoc analysis, enabling identification and correction of any unexpected behaviors.
Our systematic evaluation reveals that reasoning capacity emerges as the dominant factor influencing performance, with reasoning-enabled configurations reducing numerical errors by over 85% in complex tasks compared to non-reasoning counterparts. When combined with a multi-agent architecture, and iterative self-correction, our system approaches expert-authored reference procedures on the benchmark (F1-score > 0.89) on challenging multi-plate syntheses. These findings establish a clear blueprint for developing robust AI partners for autonomous laboratories, ensuring both capability and trust-worthiness as these systems are deployed in high-stakes scientific environments.