[ ]:

import fdfi
print('FDFI version:', fdfi.__version__)

FlowExplainer: Flow-Disentangled Feature Importance

This tutorial covers the FlowExplainer, which uses normalizing flows to compute feature importance. By the end, you’ll understand:

  1. How Flow-DFI differs from OT-based methods

  2. The difference between CPI and SCPI methods

  3. How to use FlowExplainer with default and custom flow models

  4. When to choose FlowExplainer over OTExplainer

Setup

First, let’s import the necessary libraries:

[ ]:

import sys
sys.path.insert(0, '../..')  # Add project root to path

import numpy as np
import warnings
warnings.filterwarnings('ignore')

from fdfi.explainers import FlowExplainer, OTExplainer
from fdfi.plots import confidence_interval_plot, diagnostics_plot, summary_bar

# Set random seed for reproducibility
np.random.seed(42)

Background: Why Flow-DFI?

OTExplainer and EOTExplainer use optimal transport to create a disentangled representation. However, they rely on parametric assumptions (Gaussian or kernel-based).

FlowExplainer uses a normalizing flow — a learned neural network that maps between the original feature space X and a disentangled latent space Z. This is:

  • More flexible: Learns complex non-linear transformations

  • Data-driven: No parametric assumptions

  • Invertible: Can map X → Z and Z → X

The tradeoff is that it requires training a neural network, which takes longer.

Create Test Data

Let’s create data with correlated features where only some features are truly important:

[ ]:

# Create correlated features
n_samples = 500
n_features = 8

# Covariance matrix with correlations
cov = np.eye(n_features)
cov[0, 1] = cov[1, 0] = 0.7  # Features 0 and 1 are correlated
cov[2, 3] = cov[3, 2] = 0.5  # Features 2 and 3 are correlated

X = np.random.multivariate_normal(np.zeros(n_features), cov, size=n_samples)

# Model: only features 0, 1, 2 are active
def model(X):
    return X[:, 0] + 2 * X[:, 1] + 0.5 * X[:, 2]

# Split into train/test
X_train, X_test = X[:400], X[400:]

print(f"Training data: {X_train.shape}")
print(f"Test data: {X_test.shape}")
print(f"\nActive features: 0, 1, 2")
print(f"Null features: 3, 4, 5, 6, 7")

Basic Usage: CPI Method (Default)

The simplest way to use FlowExplainer is with the default CPI (Conditional Permutation Importance) method:

[ ]:

# Create FlowExplainer with CPI method
explainer_cpi = FlowExplainer(
    model,
    data=X_train,
    method='cpi',        # Conditional Permutation Importance
    nsamples=30,         # Monte Carlo samples per feature
    num_steps=200,       # Flow training steps (use more for better results)
    random_state=42,
)

# Compute importance
results_cpi = explainer_cpi(X_test)

print("\nCPI Feature Importance:")
print("-" * 40)
for i, phi in enumerate(results_cpi['phi_X']):
    marker = "*" if i < 3 else ""  # Mark active features
    print(f"  Feature {i}: {phi:8.4f} {marker}")
print("\n* = active feature")

CPI vs SCPI: What’s the Difference?

FlowExplainer provides two methods for computing importance in the latent Z-space:

CPI (Conditional Permutation Importance)

  • Squared difference after averaging predictions

  • Formula: \(\phi_{Z,j}^{CPI} = (Y - \mathbb{E}_b[f(\tilde{X}_b^{(j)})])^2\)

SCPI (Sobol-CPI)

  • Conditional variance of counterfactual predictions

  • Formula: \(\phi_{Z,j}^{SCPI} = \text{Var}_b[f(\tilde{X}_b^{(j)})]\)

  • Related to Sobol total-order sensitivity indices

For L2 loss with independent (disentangled) features, CPI and SCPI give similar results, since both measure how much the model output changes when feature \(j\) is permuted.

Jacobian Transformation to X-space

Both methods compute importance in Z-space (disentangled features). To attribute importance to the original features \(X_l\), FlowExplainer uses the Jacobian \(H = \frac{\partial X}{\partial Z}\):

\[\phi_{X,l} = \sum_{k=1}^{d} H_{lk}^2 \cdot \phi_{Z,k}\]

This correctly accounts for how each latent dimension affects each original feature.

[ ]:

# Create FlowExplainer with SCPI method
explainer_scpi = FlowExplainer(
    model,
    data=X_train,
    method='scpi',       # Sobol-CPI
    nsamples=30,
    num_steps=200,
    random_state=42,
)

results_scpi = explainer_scpi(X_test)

# Compare CPI vs SCPI
print("Comparison: CPI vs SCPI")
print("-" * 50)
print(f"{'Feature':>8} {'CPI':>12} {'SCPI':>12} {'Active':>10}")
print("-" * 50)
for i in range(n_features):
    active = "Yes" if i < 3 else "No"
    print(f"{i:>8} {results_cpi['phi_X'][i]:>12.4f} {results_scpi['phi_X'][i]:>12.4f} {active:>10}")

Computing Both Methods at Once

Use method='both' to compute CPI and SCPI simultaneously (more efficient than running twice):

[ ]:

# Compute both CPI and SCPI
explainer_both = FlowExplainer(
    model,
    data=X_train,
    method='both',
    nsamples=30,
    num_steps=200,
    random_state=42,
)

results_both = explainer_both(X_test)

print("Result keys:", list(results_both.keys()))
print("\nCPI importance (phi_Z):", results_both['phi_Z'][:3].round(4))
print("SCPI importance (phi_Z_scpi):", results_both['phi_Z_scpi'][:3].round(4))

Confidence Intervals and Summary

Use the built-in conf_int() and summary() methods for quick, reproducible inference. This avoids custom ad hoc diagnostics code and keeps reporting consistent across explainers.

[ ]:

# Shared diagnostics (computed automatically)
flow_diag = explainer_cpi.diagnostics
print("Flow diagnostics")
print("-" * 72)
print(f"Latent independence (median dCor): {flow_diag['latent_independence_median']:.6f} [{flow_diag['latent_independence_label']}]")
print(f"Distribution fidelity (MMD):       {flow_diag['distribution_fidelity_mmd']:.6f} [{flow_diag['distribution_fidelity_label']}]")

# Standardized inference summary (v0.0.2 defaults use mixture methods)
print("\nX-space summary")
_ = explainer_cpi.summary(alpha=0.05, target='X', alternative='greater')


feature_names = [f"X{i}" for i in range(n_features)]
summary_bar(
    results_cpi["phi_X"],
    results_cpi["se_X"],
    feature_names,
    show=False,
)

ci_flow = explainer_cpi.conf_int(alpha=0.05, target="X", alternative="greater")
confidence_interval_plot(ci_flow, feature_names=feature_names, show=False)
diagnostics_plot(flow_diag, feature_names=feature_names, show=False)

Sampling Methods

FlowExplainer supports different ways to generate counterfactual values in Z-space:

  • 'resample': Sample from background data (default)

  • 'permutation': Permute within test set

  • 'normal': Sample from standard normal

  • 'condperm': Conditional permutation (regress Z_j | Z_{-j})

[ ]:

# Try different sampling methods
sampling_methods = ['resample', 'permutation', 'normal']
results_by_method = {}

for method in sampling_methods:
    exp = FlowExplainer(
        model, X_train,
        sampling_method=method,
        nsamples=30,
        num_steps=200,
        random_state=42,
    )
    results_by_method[method] = exp(X_test)

print("Importance by Sampling Method:")
print("-" * 55)
print(f"{'Feature':>8} {'resample':>12} {'permutation':>12} {'normal':>12}")
print("-" * 55)
for i in range(n_features):
    r = results_by_method['resample']['phi_X'][i]
    p = results_by_method['permutation']['phi_X'][i]
    n = results_by_method['normal']['phi_X'][i]
    print(f"{i:>8} {r:>12.4f} {p:>12.4f} {n:>12.4f}")

Using a Custom Flow Model

You can train a flow model separately and pass it to FlowExplainer. This is useful when:

  • You want more control over flow training

  • You have a pre-trained flow

  • You want to use the same flow for multiple explainers

[ ]:

from fdfi.models import FlowMatchingModel

# Train a custom flow model
custom_flow = FlowMatchingModel(
    X=X_train,
    dim=n_features,
    hidden_dim=64,
    num_blocks=2,
)
custom_flow.fit(num_steps=300, verbose='final')

# Use the pre-trained flow
explainer_custom = FlowExplainer(
    model,
    data=X_train,
    flow_model=custom_flow,  # Pass pre-trained flow
    fit_flow=False,          # Don't retrain
    nsamples=30,
    random_state=42,
)

results_custom = explainer_custom(X_test)
print("\nImportance with custom flow:", results_custom['phi_X'][:4].round(4))

Comparison: FlowExplainer vs OTExplainer

Let’s compare FlowExplainer to OTExplainer on the same data:

[ ]:

# OTExplainer for comparison
explainer_ot = OTExplainer(
    model,
    data=X_train,
    nsamples=30,
    random_state=42,
)
results_ot = explainer_ot(X_test)

print("Comparison: FlowExplainer vs OTExplainer")
print("-" * 55)
print(f"{'Feature':>8} {'Flow (CPI)':>12} {'Flow (SCPI)':>12} {'OT':>12}")
print("-" * 55)
for i in range(n_features):
    f_cpi = results_cpi['phi_X'][i]
    f_scpi = results_scpi['phi_X'][i]
    ot = results_ot['phi_X'][i]
    print(f"{i:>8} {f_cpi:>12.4f} {f_scpi:>12.4f} {ot:>12.4f}")

# Summary statistics
active_mask = np.array([True, True, True, False, False, False, False, False])

print("\n" + "=" * 55)
print("Summary: Active vs Null Feature Importance")
print("=" * 55)
for name, phi in [('Flow CPI', results_cpi['phi_X']),
                  ('Flow SCPI', results_scpi['phi_X']),
                  ('OT', results_ot['phi_X'])]:
    active_mean = phi[active_mask].mean()
    null_mean = phi[~active_mask].mean()
    ratio = active_mean / null_mean if null_mean > 0 else float('inf')
    print(f"{name:>12}: active={active_mean:.4f}, null={null_mean:.4f}, ratio={ratio:.2f}x")

Z-space vs X-space Importance

FlowExplainer provides both:

  • phi_Z: Importance in the disentangled latent space

  • phi_X: Importance attributed to original features (via Jacobian)

Space

Meaning

Z-space (phi_Z)

Importance of independent latent factors

X-space (phi_X)

Importance of original correlated features

For linear transformations (like OTExplainer), these are related by phi_X = H^T @ H @ phi_Z where H is the Cholesky factor. For flows, the Jacobian varies with position.

When to Use FlowExplainer

Summary

In this tutorial, you learned:

  1. FlowExplainer uses normalizing flows for flexible, data-driven feature importance.

  2. CPI averages predictions first, SCPI averages squared differences (Sobol-style).

  3. Use method='both' to compute CPI and SCPI simultaneously.

  4. Different sampling methods offer different tradeoffs.

  5. You can use custom flow models for more control.

  6. Shared diagnostics and conf_int/summary provide a consistent inference workflow.

  7. Strict one-sided testing (alternative='greater') is useful for feature screening.

Next Steps

  • Try FlowExplainer on your own data.

  • Compare X-space and Z-space significance.

  • Cross-check with OT/EOT for consistency.