Semantic Mapping for Robotic Maintenance

Project dates: 12/01/2018 - 26/10/2020

Technology Aim

This project develops a semantic mapping technique that is specifically tailored for robotic
maintenance of mining equipment. A semantic map in this scenario contains objects that are
part of a bigger piece of equipment (such as the screws on a wheel, the lever of a valve on a
pipe, or the lights on a big haul truck). The semantic map records the position, orientation,
and shape of these parts in 3D space. Such a semantically meaningful map is the foundation
for fully or partially autonomous robotic manipulation: it allows a robot to assess if
maintenance work needs to be conducted, and then to plan the necessary actions, e.g. to
close a valve or tighten screws.

The project uses computer vision and deep machine learning to detect objects and object
parts, and combines this with modern approaches to simultaneous localisation and mapping
(SLAM), to produce an accurate semantic map.

In contrast to the semantic maps developed in this project, typical robotic mapping or SLAM
techniques today produce maps that contain only geometric information, but do not carry any
information about the nature of the objects or object parts. This makes it very hard for
humans to correctly interpret these maps, yet alone collaborate with the robot for a
challenging task such as remote maintenance.

Industry Benefit

The project provides insights into the requirements and challenges of semantic mapping for
robotic maintenance by adopting the currently developed general purpose semantic SLAM
approach of the Australian Centre for Robotic Vision to the special scenario of mining.

Semantic maps will be a key foundation for robotic maintenance or robotic-assisted remote
maintenance in mining.

Funding / Grants

  • Mining3 (2018 - 2019)