FDFI Documentation
FDFI (Flow-Disentangled Feature Importance) is a Python library for computing feature importance using disentangled methods, inspired by SHAP. This package implements both OT-based DFI and flow-based FDFI methods.
Key Features
Multiple Explainer Types: OTExplainer, EOTExplainer, and FlowExplainer (TreeExplainer/LinearExplainer/KernelExplainer coming soon)
OT-Based DFI: Gaussian OT (OTExplainer) and Entropic OT (EOTExplainer)
Shared Diagnostics: Latent independence (dCor) and distribution fidelity (MMD) checks for OT/EOT/Flow
Visualization: Summary, waterfall, force, dependence, CI, and diagnostics plots
Statistical Inference: Confidence intervals, one-sided tests, FDR correction, and group-level importance
Easy to Use: Simple API; background data replaces the
predict_probabaselineExtensible: Built with modularity for future enhancements
Note
New in 0.0.8: confidence_interval_plot() now supports
one-sided confidence intervals. When conf_int() is called with
alternative='greater' or alternative='less', the plot renders the
open bound as a stub with a native Matplotlib caret, following forest-plot
conventions. See Confidence Intervals and Statistical Inference for worked examples.
Quick Example
import numpy as np
from fdfi.explainers import OTExplainer
from fdfi.plots import summary_bar
rng = np.random.default_rng(0)
def model(X):
return X.sum(axis=1)
X_background = rng.standard_normal((100, 10))
explainer = OTExplainer(model, data=X_background, nsamples=50)
X_test = rng.standard_normal((50, 10))
results = explainer(X_test)
# Confidence intervals (one-sided, with FDR correction)
ci = explainer.conf_int(alpha=0.05, alternative="greater", multitest_method="fdr_bh")
feature_names = [f"X{i}" for i in range(X_background.shape[1])]
summary_bar(results["phi_X"], results["se_X"], feature_names, show=False)
Installation
git clone https://github.com/jinhongdu-lab/FDFI.git
cd FDFI
pip install -e .
# With optional dependencies
pip install -e ".[flow]" # For flow matching models
pip install -e ".[dev]" # For development
User Guide
- Installation
- Core Concepts
- What is Feature Importance?
- SHAP and Shapley Values
- Disentangled Feature Importance (DFI)
- Gaussian vs Entropic OT
- Flow-Disentangled Feature Importance (Flow-DFI)
- Shared Disentanglement Diagnostics
- Relationship to Other Methods
- Confidence Intervals
- Multiple Testing Correction
- Group-Level Importance
- Further Reading
- Choosing an Explainer
- Quick Decision Guide
- OTExplainer (Gaussian OT)
- EOTExplainer (Entropic OT)
- FlowExplainer (Flow-Based DFI)
- Shared Diagnostics (OT / EOT / Flow)
- TreeExplainer
- LinearExplainer
- KernelExplainer
- Crossfitting (Cross-Fitted Inference)
- Statistical Inference: One-sided Tests and FDR
- Computing Confidence Intervals
- Interpreting Results
- Statistical Inference with FDFI
- Frequently Asked Questions
Tutorials
- Tutorials
- Quickstart: FDFI in 5 Minutes
- OTExplainer: Gaussian Optimal Transport
- EOTExplainer: Semicontinuous Entropic Optimal Transport
- FlowExplainer: Flow-Disentangled Feature Importance
- Confidence Intervals and Statistical Inference
- Visualization with FDFI
- Getting Started
- Tutorial Overview
- Running the Tutorials
- Prerequisites
Case Studies
- Case Studies
- Case Study 1: EOT FDFI Data Analysis Pipeline
- Load Data and Feature Groups
- Fit Flow Model on All Data
- Individual Feature Inference With conf_int()
- Feature-Level Summary
- Group Importance
- Group Summary Table
- Group Mapping
- Group Importance Plot
- Visualize and Compare Results: Flow vs EOT
- Note on Variance Floor (
var_floor_c)
API Reference
Development