Segmentation Framework for Heat Loss Identification in Thermal Images: Empowering Scottish Retrofitting and Thermographic Survey Companies
In this ‘Research Snapshot’, we take a look at how an AI-driven approach to fuel poverty could help to mitigate the issue throughout Scotland.
The study, which was presented at the International Conference on Brain-Inspired Cognitive Systems (BICS) held in Kuala Lumpur, Malaysia in August 2023, was carried out by NSC Transparent Ocean Research Fellow, Dr Junayed Hasan. The work presented in this paper is the result of a joint collaboration between the School of Computing and Robert Gordon University, the National Subsea Centre and IRT Surveys, who jointly funded the project with The Data Lab Innovation Centre. The project is led by Professor Eyad Elyan from the School of Computing, Robert Gordon University and involves Transparent Ocean team members Professor Jinchang Ren and Dr Yijun Yan and Dr Mostafa Kamal Sarker of the University of Oxford.
Below, Junayed explains the motivation behind the study and the team’s approach to this rapidly evolving problem.
The Motivation
As researchers, we're at the forefront of change, driven by a strong commitment to tackle real-world challenges. Our goal is to uncover mysteries and create solutions that lead to a better future, step by step. This isn't just about academic pursuits, it's a solemn duty we undertake for the sake of our shared humanity.
In our daily lives, we're woven into the very issues we aim to solve. We feel the weight of responsibility as we confront problems that deeply affect us. Among these challenges, the threat of fuel poverty is significant, casting a shadow over 39% of households in Scotland due to a sudden surge in energy prices. This situation calls for urgent action.
At this critical moment, the Scottish Government has emerged as a leader. With unwavering determination, they're taking on carbon emissions and pledging to eliminate fuel poverty in their communities. The Scottish Fuel Poverty Act showcases their boldness, outlining a plan to achieve energy efficiency and environmental stewardship by 2040.
Notably, enhancing building efficiency plays a crucial role and opens doors for funding emerging technologies such as heat pumps, insulation and retrofitting projects. Collaborative efforts between retrofitting experts and social housing providers align well with the Energy Efficiency Standard for Social Housing (EESSH) policy. These stakeholders use ground-level thermographic surveys with infrared (IR) cameras to identify structures needing retrofitting. However, a challenge arises in the manual analysis of post-survey thermal images, which involves pinpointing sources of heat loss and meticulously removing obstacles. This complex and labour-intensive process, reliant on experts, limits scalability and hampers the full use of cloud-based statistical thermal profile analysis tools.
The Approach
In response to this challenge, our research initiative focuses on creating an advanced, automated solution using deep learning. Our goal is to accurately identify objects connected to heat loss in ground-level thermal images. To achieve this, we formed a strategic partnership with an industrial thermography collaborator and conducted thorough surveys, resulting in a dataset of 2,500 thermal images. With input from experts, we carefully selected 1,800 images and annotated target objects, creating a dataset tailored to our goals. We employed a state-of-the-art deep learning-based algorithm to develop a novel process for identifying, detecting and removing potential sources of heat loss and obstructions. Importantly, our approach eliminates the need for manual thresholding/labour-intensive analysis, enhancing precision and overall accuracy. Given the complexity of our dataset, which includes varying object shapes and weather conditions, we skilfully used transfer learning techniques. Through rigorous testing, we pinpointed the best fine-tuned model, achieving an impressive mean average precision of 77.2% for precise object segmentation.
Simultaneously, we designed and built an intuitive web-based tool that allows stakeholders to assess the system's effectiveness in real-time alongside their existing methods. This comprehensive effort highlights the promising potential of data-driven solutions in addressing the multifaceted challenge of fuel poverty.
To read the research paper, please click here.
To discover more about how our Transparent Ocean team is solving real-world problems and the other impactful research projects that are currently being undertaken, view our dedicated Transparent Ocean webpage.
