Following our Net Zero Operations Research Assistant, Martin Fyvie’s presentation at the Genetic and Evolutionary Computation Conference’s 2023 Explainable AI Workshop, we have taken a closer look at his project and explored the importance of XAI and how his project is helping industry projects here at the NSC.
What is Explainable AI?
Explainable AI (XAI) is a term that covers a broad selection of concepts regarding the understanding of Artificial Intelligence (AI) decision-making processes. In the area of AI, there is an increasing demand to demystify how these systems reach their conclusions. This has led to the emergence of Explainable Artificial Intelligence (XAI) which aims to make the intricate decision-making processes of AI models transparent and relatable to humans. Rather than only accepting AI's decisions, XAI empowers decision-makers to probe the "why" behind them, ensuring that as AI's influence expands, its functions are clear or easy to understand.
XAI in Non-Deterministic Solvers and EC
Both Evolutionary Computing (EC) and Non-Deterministic Solvers (NDS) are branches of AI that often operate in ways that can seem mysterious to end users. EC, inspired by natural evolution, refines solutions over iterations, drawing parallels to processes like mutation and natural selection. On the other hand, non-deterministic solvers, which include a range of algorithms that can produce different outcomes with the same input, add another layer of complexity to the AI decision-making landscape.
XAI's role in this context is to help explain the inner workings of these systems. For non-deterministic solvers, XAI can offer insights into the variability in outcomes and the main drivers behind solution quality. By applying existing and new XAI techniques to these solvers, we can help highlight the potential of EC and NDS’ to decision-makers while also helping build trust between the two.
What are the main aims of this project?
Over the past decade, the field of XAI has witnessed significant growth in both academic and industrial research and applications. The primary focus has been on developing XAI techniques that demystify the decision-making processes of systems, such as neural networks, that often emulate human reasoning. While evolutionary computing, especially non-deterministic solvers like Genetic Algorithms (GAs), have been influential in this, they were rarely the subject of such study. Our project is dedicated to the research and development of novel XAI methods tailored for NDS’ including GAs which are commonly employed in AI systems to address optimization challenges. Our goal is to help demystify the decisions and solutions these systems offer, enabling end-users and decision-makers to better understand the rationale behind the recommended solutions.
Current challenges of the project
Non-Deterministic Solvers, by their very nature, utilize some form of randomness to help evolve and discover newer, better solutions to an optimization problem. It is this randomness that provides the greatest challenge in this project. These algorithms, such as GAs, evolve a set of solutions over time by applying specific operators and internal methods. In both benchmarking problems and real-world applications, each run by these algorithms to solve the problem may result in a distinct set of solutions even when the inputs do not change. While this is challenging from the point of view of explaining the decisions being made, it is also difficult to build trust between decision-makers and these systems when highlighting that there are many possible, equally good solutions to their problems and each run may return a subset of these due to the search path it has taken.
Take, for instance, a work schedule generated by an AI system. An administrator responsible for workforce planning would benefit from insights into the AI's decision-making process: understanding the factors influencing the proposed schedule, discerning what contributes to optimal or suboptimal outcomes, and recognizing the flexibility they have in adjusting the solution without compromising its quality. Such transparency not only helps increase the quality of the schedules but also builds trust between the end user and the AI system. It is this level of transparency and understanding that we aim to build by overcoming these challenges.
Industry-facing applications
Here at the NSC, we are applying this approach to our multiple projects. Currently, our work with the BT Group, joint funders with The Data Lab on the ENDs project itself, is an excellent example of a project undertaken with XAI methodologies at its core. This collaboration has taken the form of research into applying novel XAI techniques to their staff rostering system to help explain key drivers. This work aims to help gain a better understanding of the underlying technologies used in this process as well as support decision-makers with key information.
Further applications of this research include the “UK Continental Shelf Offshore Workforce Planning” work, as part of the wider Data for Net Zero (D4NZ) project, which is being undertaken with XAI methodologies at its core. We are in the process of researching and developing a system designed to forecast evolving skill requirements, assisting companies in their long-term strategic workforce planning. The primary objective is to equip these companies with the foresight to prepare their existing workforce for the roles and responsibilities that will emerge as we shift towards a green energy landscape. This in turn can help minimize disruption while maximising their potential for future projects.
Companies interested in further information on how this work and research can be applied to their own projects can reach out to the NSC via email at NSCenquiries@rgu.ac.uk. The NSC's dedicated team will be happy to discuss possible collaborations and partnerships.
To discover more about how our Net Zero Operations team is solving real-world problems and the other impactful research projects that are currently being undertaken, view our dedicated Net Zero Operations webpage.
