Handheld 3D Thermography using Range Sensing and Computer Vision

Description:
This PhD thesis investigates methods for automatically generating 3D surface temperature models using a handheld sensor. A new device “HeatWave” is proposed, which includes a thermal-infrared camera  and simultaneously acquires range, color and thermal radiation data. A 3D reconstruction algorithm is then implemented which allows accurate 3D models to be generated and viewed, complete with surface temperature information, in real-time.

The applications for this system are diverse and include medical imaging, robotic search and rescue, industrial monitoring and mining, however, the specific application of Building Energy Auditing was chosen for special attention in the thesis.
Several novel solutions for fundamental problems were developed as part of the development of the proposed system. These include those relating to device calibration (both geometric and radiometric), video-based localization, 3D raycasting and visualization. In investigating these areas, the project was divided into 3 phases:

Sensor and System Calibration

The accurate geometric and radiometric calibration of thermal-infrared cameras, and multi-sensor systems containing these cameras, is critical for many computer vision and robotics algorithms. Calibration allows sensors to be used to extract accurate measurements which can be relied upon for crucial geometry and localization calculations. The key contribution in this area was the development of a new approach for calibrating thermal-infrared cameras which was proven to result in lower errors than conventional methods. In addition, convenient and accurate methods for radiometric (temperature) calibration were also investigated.

External Project Page:

• https://code.google.com/p/mm-calibrator/ (includes source code, multimedia)

Major Publications:
A mask-based approach for the geometric calibration of thermal-infrared cameras
S Vidas, R Lakemond, S Denman, C Fookes, S Sridharan, T Wark
IEEE Transactions on Instrumentation and Measurement, 61 (6), 1625-1635
http://eprints.qut.edu.au/49583/

Video-based Localization

Research in this area aims to develop a robust system for estimating the position and orientation of a hand-held thermal-infrared camera in 3D space using only video footage from the camera. Given the advantages of thermal-infrared cameras for navigation purposes, such as their ability to penetrate fog, dust and smoke, and to operate effectively without a light source, such a method could be used as a method for assisting robots and humans to navigate in difficult environments such as mines and rescue sites. This form of localization also has the potential to be integrated with other approaches, such as that used for the final 3D thermography system, in order to enhance accuracy and stability.

External Project Page:

• http://code.google.com/p/thermalvis-ros-pkg/ (includes source code, multimedia)

Major Publications:
An exploration of feature detector performance in the thermal-infrared modality
S Vidas, R Lakemond, S Denman, C Fookes, S Sridharan, T Wark
DICTA 2011: International Conference on Digital Image Computing Techniques and Applications
http://eprints.qut.edu.au/48161/

Hand-held monocular SLAM in thermal-infrared
S Vidas, S Sridharan
ICARCV 2012: International Conference on Control, Automation, Robotics and Vision
http://eprints.qut.edu.au/53486/

Towards robust night and day place recognition using visible and thermal imaging
W Maddern, S Vidas
RSS 2012: Robotics Science & Systems, Workshop: Beyond laser and vision: Alternative Sensing Modalities
http://eprints.qut.edu.au/52646/

3D Surface Temperature Modeling

This phase of research culminated in the development and testing of the proposed 3D thermography device and accompanying software, however, several more specific technical contributions were made. These include a method for weighted raycasting that utilizes geometric and other information to enhance surface temperature estimation accuracy, and a novel multi-modality visualization scheme.

External Project Page:

• https://wiki.csiro.au/display/ASL/HeatWave (includes multimedia)

Major Publications:
3D thermal mapping of building interiors using an RGB-D and thermal camera
S Vidas, P Moghadam, M Bosse
ICRA 2013: International Conference on Robotics and Automation
http://eprints.qut.edu.au/60103/

HeatWave: A handheld 3D thermography system for energy auditing
S Vidas, P Moghadam
Elsevier Energy and Buildings (In Press)