Alpha version.

This is a toolkit for the exploration of bias in AI systems and create recommendations for fairer system creation. You can set it up either locally or in your organization's server for developers to access it remotely. The toolkit can load a broad range of datatypes (e.g., tabular, graph, vision) and models, and can analyze them with a variety of tools. Loader and analysis modules are dockerized components to ensure independent execution. This repository holds the main toolkit's implementation only; an overview of components implemented by the MAMMOth consortium can be found in the mammoth-commons catalogue. There, you will also find the component build process and instructions on how to generate custom ones, for example to handle your own proprietary data.

• Integration of MAMMOth's technical components: 1/6
• User management: keycloack, shared KFP instance
• Secure API: krackend
• Toolkit dockerized and KFP in local K3D instance
• Protected characteristics: fixed through the lists
• Installation: scripts
• Tested: Linux, WSL
• Debugging: pipeline result may not be available always to toolkit
• Bias analysis pipelines: 1/2 (missing dataset bias analysis)

You will be guided to install the prerequisite software for the Mammoth toolkit to run. All scripts can be found in the scripts/ folder.

Step 1: Docker
Step 2: K3D
Step 3: KFP
Step 4: Toolkit Startup
Links

Install Docker only if it is not already installed. On Windows enable WSL and install Docker Desktop instead.

chmod  x docker_install.sh  # make the script executable
./docker_install.sh

Install Kubectl (Kubernetes - Command line tool) and the K3D mini Kubernetes distribution if the latter not already installed. K3D is a lightweight wrapper to run K3S (Rancher Lab’s minimal Kubernetes distribution) in docker. Tested with UBUNTU based OS but it could work also for other linux distributions. The docker engine must be installed and active.

chmod  x k3d_install.sh
./k3d_install.sh
k3d cluster create kfp --api-port 6550 -p "8082:80@loadbalancer" --agents 1  # create a K3D cluster

Install KFP (Kubeflow Pipelines) standalone version 2 if not already installed. The K3D cluster needs to be running already. Tested with K3D cluster but it should work with other Kubernetes distributions also. The toolkit maked use of the KFP Python SDK to compile pipelines to an intermediate representation YAML. These pipelines are submitted to run on a KFP-conformant backend such as the open source KFP backend or Google Cloud Vertex AI Pipelines.

chmod  x kfp_install.sh
./kfp_install.sh

kubectl -n kubeflow get pods

kubectl apply -f kfp_ingress.yaml

Before proceeding please create an .env on the top level folder, if it does not already exists. Make sure that K3D with KFP installed is up and running and that the port forward to KFP is active. In this same top-level folder run the command

docker compose up -d

Wait until the system loads. The toolkit is available at the following URL in your browser: http://localhost:5173

http://keycloak.local.exus.ai:8080

COMPOSE_PROJECT_NAME=mammoth_kk

KEYCLOAK_POSTGRES_USER=mammoth_kc_db_user
KEYCLOAK_POSTGRES_PASSWORD=mammoth_kc_db_pass
KEYCLOAK_POSTGRES_DATABASE=mammoth_kc_db
KEYCLOAK_HOSTNAME=keycloak.local.exus.ai
KEYCLOAK_ADMIN_USER=kc_admin
KEYCLOAK_ADMIN_PASSWORD=kc_admin_pass
KEYCLOAK_LOGLEVEL=INFO
KC_HEALTH_ENABLED=true
KC_METRICS_ENABLED=true
API_POSTGRES_USER=mammoth_api_db_user
API_POSTGRES_PASSWORD=mammoth_api_db_pass
API_POSTGRES_DATABASE=mammoth_api_db
API_POSTGRES_HOST=api-db
API_POSTGRES_PORT=5432

VITE_KEYCLOAK_URL=http://keycloak.local.exus.ai:8080
VITE_KEYCLOAK_CLIENT_ID=kraken
VITE_KEYCLOAK_REALM=toolkit
VITE_LOGOUT_REDIRECT_URI=http://localhost:5173
VITE_BACKEND_URL=http://krakend.local.exus.ai:8081

Docker Engine
K3D
K3S
Kubectl
Kubeflow Pipelines
KFP GitHub