Interactive Explainable AI Dashboard for Multi-Modal AI Systems
Problem Statement
In recent years, AI models have become ubiquitous in today’s world, but the outcomes of many AI models are challenging to comprehend and trust due to their black-box nature. Therefore, many explainable AI methods have been designed to make the decision-making processes of AI systems transparent and understandable to humans, aiming to improve trust in AI models. These methods help us understand how AI models arrive at their conclusions, which means it supports getting insights regarding how the AI model works and how it makes predictions based on the input dataset.
However, despite their potential, many users struggle to use these explainable AI methods effectively, especially with AI researchers and students. This is because they often require a high level of different technical and programmable expertise, making them challenging to understand and apply. Additionally, these explainable methods require unique configurations and specific programming steps for different AI models, even lacking intuitive explanations or guided workflows that can help users to set up and run. As a result, it takes much more time and effort to investigate and use these methods to perform model explanation.
To address this challenge, this assignment aims to develop a web-based application that integrates multiple open-source explainable AI techniques into a single, user-friendly platform. By simplifying the process of AI model explanation and removing technical barriers, the platform will help researchers and students efficiently view the model explanation from their own AI models without extensive programming efforts, ultimately accelerating their research process and improving accessibility to AI explainability tools.
Current Progress
Currently, the application already completed some features:
✅ Allow users choosing AI modality and task:
● Computer Vision: Image Classification, Segmentation
● Natural Language Processing: Text Classification, Q&A
● Tabular Data: Classification
✅ Allow users uploading their own datasets, models (.pth).
✅ Visualize the AI model explanations using top XAI methods:
● GradCAM for Computer Vision
● SHAP for Text and Tabular
A result of implemented features.
See more on: https://www.youtube.com/watch?v=mg45xZVvgyQ
Tasks
Based on the current development, students continue to improve the application by implementing some additional features:
Explainability for Time-Series or Audio Data
● Extend modalities to Time-Series (e.g., stock prediction explanations) or Audio Classification (e.g., speech or sound event detection), Text Generation.
Explainability Summary via Generative AI
Auto-generate summaries about model explanations by leveraging Generative AI, it could be:
● Implementing a microservice, allows communicating with OpenAI’s API (or others) to transfer the explanation results from previous steps (under image format), then receiving the results from API and displaying the result on UI (the result in this case will be the textual explanation, describing the model explanation results).
Or:
● Fine-tune a small LLM to generate better textual summaries for XAI results.
Data Cleaning and Pre-processing Enhancement
Improving the phase of data-pre-processing for multi-modal AI explainability (Image Classification/Segmentation, Text Classification, Text Question Answering, Tabular Classification) before generating the model explanation.
User Management
Implement a system to manage users, including registration, login, logout, and role-based access control to ensure secure and personalized access to the application features.
Multi-XAI Method Comparison
Implement the model explanation using different XAI techniques for each modality, it could be:
● Allows users choosing which XAI method they want to use to perform the explanation.
● Let users compare multiple XAI methods side-by-side on the same task, such as GradCAM, Score-CAM, LIME ….
Support Diverse Model & Data Formats
Add support for different model and dataset formats (.onnx, .h5, tf, etc), allowing users to upload dataset/model via URL. Auto searching and filling other additional data (label files, token files).
Cloud Deployment
Run and scale the application on AWS or Azure - practice real-world DevOps.
Collect users’ feedback:
Students are encouraged to define criteria to help users effectively evaluate the application. Examples include:
● Usability: Is the application easy to navigate and use for generating AI explanations?
● Clarity: Are the explanation results clear and easy to understand ?
● Insightfulness: Does the explanation help users gain insights from the model’s decision.
Contact
Viet Duc Le (v.d.le@utwente.nl)
Minh Thanh Nguyen (m.t.nguyen@utwente.nl)