Research Challenges

Research often presents a myriad of challenges. Here’s a general overview of the challenges faced in this research project:

MMI Challenges

One major challenge is that the explanation mechanism and methods provided are tailored to match the current user, the user’s needs and context. Meaning, explanations should be customized based on who the user is, making the information more relevant and understandable. This requires a good understanding of the mental models of the end users, that are shaped by experiences, education, culture, and personal beliefs and describe how people perceive, think, and understand. By understanding the mental model of the user, you can provide explanations that align with their existing knowledge, fill the gaps without overwhelming, keep them interested and enhance their understanding. This will help to overcome the challenge of generating high quality, consistent, and ma-chine-usable feedback. It is an approach of customizing explanations to make communication user-centric and more effective as it helps individuals to process information and make decisions. This helps to answer one of the key questions of how to effectively interact with explanations and high dimensional industrial data? It requires incentives, effort, workflows, a comprehensive interaction design and end-user confidence.

Algorithmic Challenges

Industrial data is often high dimensional and sequential. It consists of multivariate timeseries or signal data and the direct application of feature attribution methods like LIME or SHAP, that aim to make machine learning models more interpretable and transparent and to under-stand why a model made a certain prediction, are not suitable. Due, that the weights on the individual features (e.g. points in the multivariate time series) are difficult to interpret. It requires a step-by-step process to solve those challenges of enabling machine learning training to process domain expert’s feedback and produce meaningful explanations for industrial applications in a reliable fashion.

ML Life Cycle Challenges in EXPLAIN Life Cycle

The following describes the challenges faced in a Machine Learning Life Cycle in the context of the Explain Life Cycle which is explained here.

The end-to-end Machine Learning Operations (MLOps) framework, that refers to the process of managing the lifecycle of machine learning, from development to deployment and monitoring, must manage the complex and changing training data.
Some explanation types must access training data at the time of operation. Similar past situations are valuable information for end users. Hence, the MLOps system must manage the data-dependencies of ML models and explainers.
The MLOps system must version the data, code and ML models. Adding explanation and explanatory trainings to the system makes the versioning more complex.
MLOps includes monitoring ML models in operations. This includes monitoring of the model performance, data and concept drifts. Introducing explanations adds new information to track the health of ML models.