Creating Data-Driven Models for Predicting Localization Failures for Mobile Robots

Abstract

Self-localization is critical for the safe and effective navigation of autonomous mobile robots (AMRs) and driverless industrial trucks. However, localization failures in highly dynamic, feature-poor, or ambiguous environments can lead to operational disruptions and safety hazards. While probabilistic methods like particle filters manage uncertainties in sensing and actuation, they lack mechanisms to anticipate and mitigate impending failures.

This tutorial presents a data-driven approach to predict localization failures before they occur, significantly enhancing the reliability and safety of autonomous navigation systems. We introduce a systematic methodology for generating comprehensive datasets tailored for training predictive models, leveraging simulated environments in NVIDIA Isaac Sim. These datasets include synchronized streams of localization estimates, ground-truth poses, and sensor measurements (e.g., LiDAR, IMU) collected under diverse, challenging scenarios such as dynamic obstacle interactions, featureless zones, and map ambiguities—conditions designed to replicate real-world complexities prone to localization faults.

We introduce Flowcean, a flexible framework that extends traditional machine learning libraries by providing high-level abstractions for generalized learning and modeling. Flowcean integrates with the Robot Operating System (ROS 2), enabling streamlined workflows from data collection to model deployment. A live demonstration showcases the practical impact of failure prediction, comparing robot navigation performance with and without predictive capabilities in simulated environments.

By equipping participants with tools to develop and deploy robust predictive models, this tutorial provides actionable insights into pre-failure conditions, fostering safer and more reliable autonomous navigation in complex, real-world settings. The presented methodologies and tools offer a scalable foundation for researchers and practitioners to enhance the resilience of robotic systems against localization challenges.

Tutors

Markus Knitt is a doctoral researcher at the Institute of Logistics Engineering at Hamburg University of Technology (TUHH). He earned his bachelor’s degree in Mechatronics from TUHH in 2020, followed by a master’s degree in Mechatronics in 2022, with a specialization in robotics and intelligent systems. Knitt’s research interest in localization developed during his master’s thesis on RFID-based localization using a mobile robot. His first published work introduced an approach to 6D pose estimation of logistics objects using RGB cameras and synthetic training data. His research interests are centered on robot localization and object pose estimation. His publications cover a range of topics, including methods for specifying location data requirements for intralogistics applications, benchmarking indoor localization of mobile robots, and the development of open-source modular mobile robot platforms. Additionally, he is part of a research project on the automatic generation of models for prediction, monitoring, and testing of cyber-physical systems, which resulted in the tool Flowcean.

Sean Maroofi is a research associate affiliated with the Institute of Logistics Engineering at TUHH since 2024. He obtained his master’s degree in Mechatronics and bachelor’s degree in mechanical engineering from TUHH. His research interest include robot control and machine learning. He worked on various student research projects, including the development of a framework for researching reinforcement learning algorithms to autonomous driving applications and dynamic model inversion of soft robots for trajectory tracking. As a research associate, he contributed to the development of process flows for automated delivery robots in urban environments. Within this project, he has created an outdoor mobile robotics dataset for the development of mapping and localization algorithms in urban scenery. Currently, he collaborates with Mr. Knitt on prediciting delocalization of mobile robots in warehouse applications and contributes to the development of Flowcean and its communication with ROS2