Semantic Mapping for Robotic Maintenance

Overview

This project develops a semantic mapping technique that is specifically tailored for robotic maintenance of mining equipment. A semantic map in this scenario contains objects that are part of a bigger piece of equipment (such as the screws on a wheel, the lever of a valve on a pipe, or the lights on a big haul truck). The semantic map records the position, orientation, and shape of these parts in 3D space. Such a semantically meaningful map is the foundation for fully or partially autonomous robotic manipulation: it allows a robot to assess if maintenance work needs to be conducted, and then to plan the necessary actions, e.g. to close a valve or tighten screws.

The project uses computer vision and deep machine learning to detect objects and object parts, and combines this with modern approaches to simultaneous localisation and mapping (SLAM), to produce an accurate semantic map.

In contrast to the semantic maps developed in this project, typical robotic mapping or SLAM techniques today produce maps that contain only geometric information, but do not carry any information about the nature of the objects or object parts. This makes it very hard for humans to correctly interpret these maps, yet alone collaborate with the robot for a challenging task such as remote maintenance.

Real World Impact

The project provides insights into the requirements and challenges of semantic mapping for robotic maintenance by adopting the currently developed general purpose semantic SLAM approach of the Australian Centre for Robotic Vision to the special scenario of mining.

Semantic maps will be a key foundation for robotic maintenance or robotic-assisted remote maintenance in mining.