Explainers 

fdfi.losses.absolute_error(y_true, y_pred)[source]

Absolute-error (L1) loss: |y_true - y_pred|.

Parameters:

y_true (ndarray)
y_pred (ndarray)

Return type:

fdfi.losses.huber(y_true, y_pred, delta=1.0)[source]

Huber loss with threshold delta.

Quadratic for residuals |r| <= delta and linear beyond, giving robustness to outliers while remaining smooth at the origin.

Parameters:

y_true (ndarray)
y_pred (ndarray)
delta (float)

Return type:

fdfi.losses.pinball(y_true, y_pred, quantile=0.5)[source]

Pinball (quantile) loss for the given quantile in (0, 1).

At quantile=0.5 this is half the absolute error.

Parameters:

y_true (ndarray)
y_pred (ndarray)
quantile (float)

Return type:

fdfi.losses.log_loss(y_true, y_pred, eps=1e-12)[source]

Binary cross-entropy (log loss).

y_true are labels in {0, 1} and y_pred are predicted probabilities P(y = 1); probabilities are clipped to [eps, 1 - eps] for numerical stability.

Parameters:

y_true (ndarray)
y_pred (ndarray)
eps (float)

Return type:

fdfi.losses.brier(y_true, y_pred)[source]

Brier score for binary probabilities: (y_true - y_pred) ** 2.

Numerically identical to squared_error() but kept separate to signal a classification (probability) use case.

Parameters:

y_true (ndarray)
y_pred (ndarray)

Return type:

fdfi.losses.zero_one(y_true, y_pred, threshold=0.5)[source]

0-1 misclassification loss for binary predictions.

y_pred is thresholded at threshold to obtain a hard label, which is compared with y_true; returns 1.0 for a mistake and 0.0 otherwise. Non-differentiable — intended for reporting, not optimisation.

Parameters:

y_true (ndarray)
y_pred (ndarray)
threshold (float)

Return type:

fdfi.losses.available_losses()[source]

Return the sorted list of built-in loss keys accepted by resolve_loss().

Return type:: list

fdfi.losses.resolve_loss(loss)[source]

Resolve a loss specifier to a callable loss(y_true, y_pred) -> ndarray.

Parameters:

loss (str, callable, or None) –

None → squared_error() (the DFI default).
str → a built-in key (see available_losses()); aliases such as 'l2', 'mse', 'bce' are accepted (case-insensitive).
callable → returned unchanged. Must accept (y_true, y_pred) and return the per-sample loss.

Returns:

The resolved per-sample loss function.

Return type:

callable

Raises:

ValueError – If loss is an unknown string key.

Base Explainer

class fdfi.explainers.Explainer(model, data=None, **kwargs)[source]

Bases: object

Base class for DFI explainers.

This class provides the interface for computing feature importance using disentangled methods, similar to SHAP explainers. It also provides post-hoc confidence intervals via conf_int() and formatted summaries via summary().

Parameters:

model (callable) – The model to explain. Should be a function that takes a numpy array and returns predictions.
data (numpy.ndarray, optional) – Background data to use for explanations.
**kwargs (dict) –
Additional parameters for the explainer. Notable keys:
- loss (str or callable, default None → squared error): the loss L(y_true, y_pred) used to define importance. String keys such as 'l1', 'huber', 'pinball', 'log_loss', 'brier' are accepted (see fdfi.losses.available_losses()), or pass any callable returning the per-sample loss. Passing true labels y at call time uses the loss-difference (DFI) form; otherwise a label-free divergence from the model’s own prediction is used.

model

The model being explained.

Type:: callable

data

Background data for explanations.

Type:: numpy.ndarray or None

Examples

>>> import numpy as np
>>> from fdfi import Explainer
>>>
>>> # Define a simple model
>>> def model(x):
...     return x.sum(axis=1)
>>>
>>> # Create an explainer
>>> explainer = Explainer(model)
>>>
>>> # Compute explanations (when implemented)
>>> # explanations = explainer(X_test)

__init__(model, data=None, **kwargs)[source]

Initialize the Explainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray | None)
kwargs (Any)

conf_int(alpha=0.05, target='X', groups=None, threshold_null=True, multitest_method=None, var_floor_c=0.1, var_floor_method='mixture', var_floor_quantile=0.95, margin=0.0, margin_method='auto', margin_quantile=0.95, alternative='two-sided', verbose=False)[source]

Compute confidence intervals and significance statistics for feature importance.

If groups is provided, computes importance and uncertainty at the group level.

Parameters:

alpha (float, default=0.05) – Significance level.
target (str, default='X') – Which space to use: ‘X’ (original) or ‘Z’ (latent).
groups (dict, numpy.ndarray, or pandas.DataFrame, optional) – Group assignment for features. Accepts: - dict: {group_name: [feature_indices]} - numpy.ndarray: 1-D array of length d with group labels. - pandas.DataFrame: binary indicator matrix (features x groups).
threshold_null (bool, default=True) – Zero out per-feature uncentered UEIFs with negative mean before summing.
multitest_method (str, optional) – Multiple testing correction method. Supports methods from statsmodels.stats.multitest.multipletests, e.g., ‘bonferroni’, ‘holm’, ‘fdr_bh’ (Benjamini-Hochberg), ‘fdr_by’.
var_floor_c (float, default=0.1) – Constant for the variance floor.
var_floor_method (str, default='mixture') – Method for variance floor calculation (‘mixture’ or ‘fixed’).
var_floor_quantile (float, default=0.95) – Quantile for the ‘mixture’ variance floor method.
margin (float, default=0.0) – Hypothesized margin for null hypothesis.
margin_method (str, default='auto') – Method to estimate the margin (‘auto’, ‘mixture’, ‘gap’, or ‘fixed’).
margin_quantile (float, default=0.95) – Quantile for the ‘mixture’ margin method.
alternative (str, default='two-sided') – Alternative hypothesis (‘two-sided’, ‘greater’, or ‘less’).
verbose (bool, default=False) – Whether to print debug information.

Returns:

Dictionary with the following keys (each an array of length d or G):

'score': estimated feature importance (mean UEIF).
'se': standard error of the mean UEIF (after variance floor).
'zscore': signed z-statistic (score - margin) / se.
'ranking': integer rank by descending z-score (1 = most important).
'ci_lower': lower confidence interval bound.
'ci_upper': upper confidence interval bound.
'reject_null': boolean array, True where null is rejected.
'pvalue': two-sided or one-sided p-value.
'margin': null hypothesis margin used.
'margin_method': method used to select the margin.
'alternative': alternative hypothesis string.

Additional keys added when applicable:

'groups': list of group names (when groups is provided).
'pvalue_adj': multiple-testing-adjusted p-values (when multitest_method is provided).

Return type:

summary(alpha=0.05, print_output=True, **kwargs)[source]

Print and return a formatted feature importance summary table.

Computes confidence intervals via conf_int() and formats the results as a human-readable table. Supports both individual-feature and group-level summaries, as well as multiple-testing correction.

Parameters:

alpha (float, default=0.05) – Significance level passed to conf_int().
print_output (bool, default=True) – If True, print the table to stdout.
**kwargs –
All keyword arguments are forwarded to conf_int(). Common options include:
- target ('X' or 'Z') — which feature space to report.
- groups — dict, 1-D array, or binary DataFrame for group-level summaries (new in 0.0.5).
- multitest_method — e.g. 'bonferroni', 'fdr_bh' for multiple-testing correction (new in 0.0.5).
- threshold_null — zero out negative-mean UEIFs before group aggregation (new in 0.0.5).
- var_floor_method, var_floor_c, var_floor_quantile
- margin, margin_method, margin_quantile
- alternative ('two-sided', 'greater', 'less')
- verbose

Returns:

The formatted summary string (same text that is printed when print_output=True).

Return type:

str

Examples

Individual-feature summary:

explainer(X_test, y=y_test)
explainer.summary(alpha=0.05, target="X")

Group-level summary with Bonferroni correction:

explainer.summary(
    alpha=0.05,
    target="X",
    groups=df_groups,
    threshold_null=True,
    multitest_method="bonferroni",
)

group_importance(groups, target='X', threshold_null=True, se_adjustment=0.1, alpha=0.05)[source]

Compute group-level feature importance with uncertainty.

Deprecated since version 0.0.5: Use conf_int() with the groups argument instead.

Parameters:

groups (dict, numpy.ndarray, or pandas.DataFrame) –
Group assignment for features. Accepts:
- dict: {group_name: [feature_indices]}
- numpy.ndarray: 1-D array of length d with group labels.
- pandas.DataFrame: binary indicator matrix (features × groups).
target (str, default='X') – Which space to aggregate: 'X' or 'Z'.
threshold_null (bool, default=True) – Zero out per-feature UEIFs with negative mean before summing.
se_adjustment (float, default=0.1) – Finite-sample SE correction constant. Set to 0.0 to disable.
alpha (float, default=0.05) – Significance level.

Returns:

'groups', 'importance', 'se', 'zscore', 'pvalue' — each an array of length G (number of groups).

Return type:

diagnose(X_orig=None, Z_full=None, report_title=None)[source]

Compute (or recompute) disentanglement diagnostics.

Evaluates latent independence via pairwise distance correlation (dCor) and distribution fidelity via Maximum Mean Discrepancy (MMD). Called automatically during __init__ when compute_diagnostics=True. Use this method to recompute diagnostics on a custom subset or after calling set_flow().

Parameters:

X_orig (np.ndarray of shape (n_samples, n_features), optional) – Original-space data to use for MMD fidelity check. When None the background data stored during __init__ is used.
Z_full (np.ndarray of shape (n_samples, n_features), optional) – Pre-encoded latent representations. When None the background latent data stored during __init__ is used.
report_title (str, optional) – Label shown in verbose logging output.

Returns:

diagnostics – Dictionary with keys:

latent_independence_dcornp.ndarray: Pairwise dCor matrix of shape (d, d).
latent_independence_medianfloat: Median off-diagonal dCor (lower = more independent).
latent_independence_labelstr: Qualitative label 'GOOD', 'MODERATE', or 'POOR'.
distribution_fidelity_mmdfloat: MMD between original and reconstructed distributions.
distribution_fidelity_labelstr: Qualitative label 'GOOD', 'MODERATE', or 'POOR'.

Return type:

Raises:

ValueError – If diagnostics are unavailable (e.g. compute_diagnostics=False was set and no latent data is accessible).

Examples

>>> diag = explainer.diagnose()
>>> print(diag["latent_independence_label"])  # 'GOOD' / 'MODERATE' / 'POOR'
>>> print(diag["distribution_fidelity_mmd"])

__call__(X, **kwargs)[source]

Compute feature importance for the given input.

Parameters:

X (numpy.ndarray) – Input data to explain. Shape (n_samples, n_features).
**kwargs (dict) – Additional parameters for explanation.

Returns:

Feature importance values. Shape (n_samples, n_features).

Return type:

Raises:

NotImplementedError – This method must be implemented by subclasses.

shap_values(X, **kwargs)[source]

Compute SHAP-like values (alias for __call__).

Parameters:

X (numpy.ndarray) – Input data to explain.
**kwargs (dict) – Additional parameters.

Returns:

Feature importance values.

Return type:

Tree-Based Models

class fdfi.explainers.TreeExplainer(model, data=None, **kwargs)[source]

Bases: Explainer

Explainer for tree-based models.

Note

Placeholder — not yet implemented. Calling this explainer raises NotImplementedError. Use OTExplainer or EOTExplainer for working implementations. A native tree-structure explainer is planned for a future release.

Parameters:

model (object) – A tree-based model (e.g. sklearn RandomForestClassifier, XGBoost, LightGBM).
data (np.ndarray, optional) – Background data.
**kwargs – Additional keyword arguments forwarded to Explainer.

__init__(model, data=None, **kwargs)[source]

Initialize the TreeExplainer.

Parameters:

model (Any)
data (ndarray | None)
kwargs (Any)

__call__(X, **kwargs)[source]

Compute feature importance for tree-based models.

Parameters:

X (numpy.ndarray) – Input data to explain.
**kwargs (dict) – Additional parameters.

Returns:

Feature importance values.

Return type:

Linear Models

class fdfi.explainers.LinearExplainer(model, data=None, **kwargs)[source]

Bases: Explainer

Explainer for linear models.

Note

Placeholder — not yet implemented. Calling this explainer raises NotImplementedError. Use OTExplainer for a working model-agnostic alternative.

Parameters:

model (object) – A linear model (e.g. sklearn LinearRegression, LogisticRegression).
data (np.ndarray, optional) – Background data.
**kwargs – Additional keyword arguments forwarded to Explainer.

__init__(model, data=None, **kwargs)[source]

Initialize the LinearExplainer.

Parameters:

model (Any)
data (ndarray | None)
kwargs (Any)

__call__(X, **kwargs)[source]

Compute feature importance for linear models.

Parameters:

X (numpy.ndarray) – Input data to explain.
**kwargs (dict) – Additional parameters.

Returns:

Feature importance values.

Return type:

Kernel Methods

class fdfi.explainers.KernelExplainer(model, data, **kwargs)[source]

Bases: Explainer

Model-agnostic explainer using kernel-based methods.

Note

Placeholder — not yet implemented. Calling this explainer raises NotImplementedError. Use OTExplainer or EOTExplainer for working model-agnostic implementations.

Parameters:

model (callable) – The model to explain; must accept np.ndarray and return np.ndarray.
data (np.ndarray) – Background data (required).
**kwargs – Additional keyword arguments forwarded to Explainer.

__init__(model, data, **kwargs)[source]

Initialize the KernelExplainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray)
kwargs (Any)

__call__(X, **kwargs)[source]

Compute feature importance using kernel methods.

Parameters:

X (numpy.ndarray) – Input data to explain.
**kwargs (dict) – Additional parameters.

Returns:

Feature importance values.

Return type:

Gaussian Optimal Transport (OTExplainer)

The OTExplainer implements Gaussian optimal-transport DFI (Disentangled Feature Importance) without cross-fitting. This is the recommended starting point for most use cases.

class fdfi.explainers.OTExplainer(model, data, nsamples=50, sampling_method='resample', random_state=0, method='cpi', **kwargs)[source]

Bases: Explainer

Optimal-transport DFI explainer using Gaussian transport.

Computes Disentangled Feature Importance (DFI) by mapping observed features to an uncorrelated (whitened) latent space via a Gaussian optimal-transport linear map, computing per-sample UEIFs in that space, and projecting back to the original feature space via the Jacobian.

This is the recommended starting point for most use cases with continuous data. For non-Gaussian or mixed-type data, prefer EOTExplainer. For rigorous inference with a small sample, consider wrapping this class with Crossfitting.

Parameters:

model (callable) – Prediction function with signature f(X) -> np.ndarray where X has shape (n_samples, n_features).
data (np.ndarray of shape (n_background, n_features)) – Background data used to estimate the Gaussian transport map (mean and covariance). Larger backgrounds give more stable estimates; 100–500 samples is typically sufficient.
nsamples (int, default=50) – Number of Monte Carlo resamples per feature used to estimate the marginal-replacement expectation.
sampling_method ({'resample', 'permutation', 'normal'}, default='resample') –
Strategy for drawing replacement values for each feature:
- 'resample' – draw with replacement from the background latent distribution (recommended).
- 'permutation' – permute the test-set latent values.
- 'normal' – draw i.i.d. standard normal samples.
random_state (int, default=0) – Seed for the random number generator used in resampling.
method ({'cpi', 'scpi'}, default='cpi') –
Averaging order for the counterfactual predictions:
- 'cpi' – average the prediction over resamples first, then apply the loss (L(r, E_b[ŷ_b])).
- 'scpi' – apply the loss per resample first, then average (E_b[L(r, ŷ_b)]; for squared error this equals CPI plus the prediction variance).
verbose (bool, default=False) – Print progress messages during setup and inference.
compute_diagnostics (bool, default=True) – Compute latent-independence (dCor) and distribution-fidelity (MMD) diagnostics during initialisation.
diagnostics_subset_max_samples (int, default=1000) – Maximum number of background samples used for the dCor computation.
latent_independence_thresholds (tuple of float, default=(0.1, 0.25)) – (good, poor) thresholds for the median off-diagonal dCor. Values below the first threshold receive label 'GOOD'.
distribution_fidelity_thresholds (tuple of float, default=(0.05, 0.15)) – (good, poor) thresholds for the MMD. Values below the first threshold receive label 'GOOD'.
**kwargs – Additional keyword arguments forwarded to Explainer. Useful keys include regularize (float, default 1e-6) which clips small eigenvalues of the covariance before computing the Cholesky factor.

mean

Background mean used for centring.

Type:: np.ndarray of shape (1, n_features)

L

Square-root of the background covariance (Cholesky-like factor); used as the decoder Z → X.

Type:: np.ndarray of shape (n_features, n_features)

L_inv

Inverse of L; used as the encoder X → Z.

Type:: np.ndarray of shape (n_features, n_features)

Z_full

Background data projected into the latent space.

Type:: np.ndarray of shape (n_background, n_features)

ueifs_X

Per-sample UEIFs in the original X-space after calling the explainer.

Type:: np.ndarray of shape (n_test, n_features)

ueifs_Z

Per-sample UEIFs in the latent Z-space after calling the explainer.

Type:: np.ndarray of shape (n_test, n_features)

diagnostics

Disentanglement quality metrics; see diagnose().

Type:: dict

Examples

>>> import numpy as np
>>> from fdfi.explainers import OTExplainer
>>> from fdfi.plots import summary_bar
>>>
>>> rng = np.random.default_rng(0)
>>> X_bg  = rng.standard_normal((200, 6))
>>> X_test = rng.standard_normal((50, 6))
>>> def model(X): return X[:, 0] + 2 * X[:, 1]
>>>
>>> explainer = OTExplainer(model, data=X_bg, nsamples=50)
>>> results = explainer(X_test)
>>> print(results["phi_X"])        # global importance, X-space
>>> print(results["phi_Z"])        # global importance, Z-space
>>>
>>> ci = explainer.conf_int(alpha=0.05, alternative="greater")
>>> summary_bar(results["phi_X"], results["se_X"], show=False)

__init__(model, data, nsamples=50, sampling_method='resample', random_state=0, method='cpi', **kwargs)[source]

Initialize the OTExplainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray)
nsamples (int)
sampling_method (str)
random_state (int)
method (str)
kwargs (Any)

__call__(X, y=None, **kwargs)[source]

Compute feature importance for the given input.

Parameters:

X (numpy.ndarray) – Input data to explain. Shape (n_samples, n_features).
**kwargs (Any) – Additional parameters for explanation.
y (ndarray | None)
**kwargs

Returns:

Feature importance values. Shape (n_samples, n_features).

Return type:

Raises:

NotImplementedError – This method must be implemented by subclasses.

Example:

import numpy as np
from fdfi.explainers import OTExplainer

# Create model and data
def model(X):
    return X[:, 0] + 2 * X[:, 1]

X_background = np.random.randn(100, 10)
X_test = np.random.randn(10, 10)

# Create explainer and compute importance
explainer = OTExplainer(model, data=X_background, nsamples=50)
results = explainer(X_test)

print("Feature importance (X-space):", results["phi_X"])
print("Standard errors:", results["se_X"])

# Compute confidence intervals with FDR control (Benjamini-Hochberg)
ci = explainer.conf_int(multitest_method='fdr_bh', alpha=0.05)
print("Significant features after FDR control:", np.where(ci["reject_null"])[0])
print("Adjusted p-values:", ci["pvalue_adj"])

Entropic Optimal Transport (EOTExplainer)

The EOTExplainer uses entropic optimal transport with Sinkhorn iterations. It supports adaptive epsilon, stochastic transport sampling, and both Gaussian and empirical transport targets.

class fdfi.explainers.EOTExplainer(model, data, nsamples=50, epsilon=0.1, auto_epsilon=False, sampling_method='resample', random_state=0, method='cpi', **kwargs)[source]

Bases: Explainer

Entropic optimal-transport DFI explainer using semicontinuous transport and population backward attribution.

Uses the population EOT coupling between the empirical source and continuous N(0, I) target. The forward map is analytical:

Z = c_ε · X_whitened, c_ε = √(1 + ε) / (1 + ε/2)

Backward attribution uses the best linear projection:

E[X_whitened | Z] = M_w · Z

where M_w = E_π[ZZ^T]^{-1} E_π[ZX_w^T] is computed analytically from the semicontinuous coupling moments. This gives the weight matrix W = L @ M_w used for the decomposition:

φ_X_j = Σ_k W[j,k]² · φ_Z_k

Feature importance is measured via the uncentered efficient influence function (UEIF):

UEIF_{i,j} = (Y_i - ŷ_{-j,i})²

where ŷ_{-j} averages predictions over counterfactual resamples of feature j.

Parameters:

model (callable) – The model to explain. Takes (n, d) array, returns (n,) predictions.
data (numpy.ndarray) – Background data for whitening and resampling. Shape (n, d).
nsamples (int, default=50) – Number of Monte Carlo samples per feature for counterfactual resampling.
epsilon (float, default=0.1) – EOT regularization parameter. Smaller ε → closer to exact OT; larger ε → more Gaussian shrinkage.
auto_epsilon (bool, default=False) – If True, set ε from a median-distance heuristic in whitened space.
sampling_method (str, default='resample') – How to draw counterfactual Z_j values: - ‘resample’: sample from the background Z pool - ‘permutation’: permute within the test set - ‘normal’: sample from N(0, 1)
random_state (int, default=0) – Random seed for reproducibility.
method ({'cpi', 'scpi'}, default='cpi') – Averaging order for counterfactual predictions (CPI averages the prediction before the loss; SCPI averages the per-resample loss).
**kwargs (dict) – Extra arguments forwarded to the base Explainer.

__init__(model, data, nsamples=50, epsilon=0.1, auto_epsilon=False, sampling_method='resample', random_state=0, method='cpi', **kwargs)[source]

Initialize the Explainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray)
nsamples (int)
epsilon (float)
auto_epsilon (bool)
sampling_method (str)
random_state (int)
method (str)
kwargs (Any)

__call__(X, y=None, **kwargs)[source]

Compute feature importance for the given input.

Parameters:

X (numpy.ndarray) – Input data to explain. Shape (n_samples, n_features).
**kwargs (Any) – Additional parameters for explanation.
y (ndarray | None)
**kwargs

Returns:

Feature importance values. Shape (n_samples, n_features).

Return type:

Raises:

NotImplementedError – This method must be implemented by subclasses.

Example with advanced options:

from fdfi.explainers import EOTExplainer

explainer = EOTExplainer(
    model.predict,
    X_background,
    auto_epsilon=True,           # Adaptive regularization
    stochastic_transport=True,   # Sample from transport kernel
    n_transport_samples=10,      # Number of transport samples
    target="gaussian",           # or "empirical"
)
results = explainer(X_test)

Shared Disentanglement Diagnostics

OTExplainer, EOTExplainer, and FlowExplainer expose a shared diagnostics interface via:

explainer.diagnostics (computed at setup by default)
explainer.diagnose(...) (recompute manually)

The diagnostics dictionary contains:

latent_independence_dcor (pairwise dCor matrix)
latent_independence_median and latent_independence_label
distribution_fidelity_mmd and distribution_fidelity_label

diag = explainer.diagnostics
# or: diag = explainer.diagnose()
print(diag["latent_independence_median"], diag["latent_independence_label"])
print(diag["distribution_fidelity_mmd"], diag["distribution_fidelity_label"])

Flow-Based DFI (FlowExplainer)

The FlowExplainer implements Flow-Disentangled Feature Importance using normalizing flows. It supports both CPI (Conditional Permutation Importance) and SCPI (Sobol-CPI). The key difference is the order of averaging:

CPI: Average the prediction first, then apply the loss: $L(Y, E_b[f(tilde{X}_b)])$
SCPI: Apply the loss per sample first, then average: $E_b[L(Y, f(tilde{X}_b))]$

Both use the configurable loss (default squared error); for the squared-error loss, $phi^{SCPI} = phi^{CPI} + mathrm{Var}_b[f(tilde{X}_b)]$.

class fdfi.explainers.FlowExplainer(model, data, flow_model=None, fit_flow=True, nsamples=50, sampling_method='resample', permuter=None, method='cpi', random_state=None, verbose='final', compute_diagnostics=True, **kwargs)[source]

Bases: Explainer

Flow-based DFI explainer using normalizing flows.

Implements CPI (Conditional Permutation Importance) and SCPI (Sobol-CPI) methods. Both measure feature importance in Z-space:

CPI: Squared difference after averaging predictions: (Y - E[f(X_tilde)])^2
SCPI: Conditional variance of predictions: Var[f(X_tilde)]

For L2 loss with independent (disentangled) features, CPI and SCPI give similar results. SCPI is related to the Sobol total-order sensitivity index.

Z-space importance is transformed to X-space using the Jacobian of the flow phi_X[l] = sum_k H[l,k]^2 * phi_Z[k] where H = dX/dZ is the Jacobian of the decoder transformation.

Parameters:

model (callable) – The model to explain. Should take (n, d) array and return (n,) predictions.
data (numpy.ndarray) – Background data for fitting flow and resampling. Shape (n, d).
flow_model (object, optional) – Pre-trained flow model. If None, will create default FlowMatchingModel.
fit_flow (bool, default=True) – Whether to fit flow model during initialization.
nsamples (int, default=50) – Number of Monte Carlo samples per feature.
sampling_method (str, default='resample') – Method for generating counterfactual Z values: - ‘resample’: Sample from encoded background data - ‘permutation’: Permute within test set - ‘normal’: Sample from standard normal - ‘condperm’: Conditional permutation (regress Z_j | Z_{-j})
permuter (object, optional) – Regressor for conditional permutation method. Defaults to LinearRegression.
method (str, default='cpi') – Which importance method to use: - ‘cpi’: Conditional Permutation Importance - average predictions first - ‘scpi’: Sobol-CPI - average squared differences - ‘both’: Compute both CPI and SCPI
random_state (int, optional) – Random seed for reproducibility.
verbose (bool or str, default='final') – Controls training output: - True or ‘all’: Show full progress bar - ‘final’: Only print final step status (default) - False: Silent
compute_diagnostics (bool, default=True) – Whether to compute disentanglement diagnostics at setup time.
flow_solver_rtol (float, default=1e-3) – Relative tolerance for default ODE integration in flow encode/decode.
flow_solver_atol (float, default=1e-5) – Absolute tolerance for default ODE integration in flow encode/decode.
diagnostics_solver_rtol (float, default=1e-6) – Relative tolerance for diagnostics round-trip integration.
diagnostics_solver_atol (float, default=1e-8) – Absolute tolerance for diagnostics round-trip integration.
**kwargs (dict) – Additional arguments passed to FlowMatchingModel if creating default.

flow_model

The fitted normalizing flow model.

Type:: object

Z_full

Encoded background data in latent space.

Type:: numpy.ndarray

method

The importance method being used (‘cpi’, ‘scpi’, or ‘both’).

Type:: str

Examples

>>> import numpy as np
>>> from fdfi.explainers import FlowExplainer
>>>
>>> # Define a simple model
>>> def model(x):
...     return x[:, 0] + 2 * x[:, 1]
>>>
>>> # Create background data
>>> X_train = np.random.randn(200, 5)
>>> X_test = np.random.randn(50, 5)
>>>
>>> # CPI only (default)
>>> explainer = FlowExplainer(model, X_train, method='cpi')
>>> results = explainer(X_test)
>>>
>>> # SCPI (Sobol-CPI - different averaging order)
>>> explainer = FlowExplainer(model, X_train, method='scpi')
>>> results = explainer(X_test)

__init__(model, data, flow_model=None, fit_flow=True, nsamples=50, sampling_method='resample', permuter=None, method='cpi', random_state=None, verbose='final', compute_diagnostics=True, **kwargs)[source]

Initialize the FlowExplainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray)
flow_model (Any | None)
fit_flow (bool)
nsamples (int)
sampling_method (str)
permuter (Any | None)
method (str)
random_state (int | None)
verbose (bool | str)
compute_diagnostics (bool)
kwargs (Any)

fit_flow(X=None, num_steps=5000, verbose=None, **kwargs)[source]

Fit the flow model on data.

Can be called after initialization with fit_flow=False, or to refit on new data.

Parameters:

X (numpy.ndarray, optional) – Data to fit on. If None, uses self.data.
num_steps (int, default=5000) – Number of training steps.
verbose (bool or str, optional) – Controls training output. If None, uses self.verbose. - True or ‘all’: Show full progress bar - ‘final’: Only print final step status (default) - False: Silent
**kwargs – Additional arguments passed to flow_model.fit().

Returns:

For method chaining.

Return type:

self

set_flow(flow_model)[source]

Set a user-provided flow model.

The flow model must have a sample_batch(x, t_span) method where: - t_span=(1, 0) encodes X to Z - t_span=(0, 1) decodes Z to X

Parameters:: flow_model (object) – A flow model with sample_batch(x, t_span) method.
Returns:: For method chaining.
Return type:: self

__call__(X, y=None, **kwargs)[source]

Compute feature importance.

Parameters:

X (numpy.ndarray) – Input data to explain. Shape (n_samples, n_features).
y (numpy.ndarray, optional) – True outcomes, shape (n_samples,). When provided, uses the DFI loss-difference form; when omitted, the label-free divergence form is used.
**kwargs (dict) – Additional parameters (unused, for API compatibility).

Returns:

Dictionary containing: - phi_Z: Z-space importance (d,) - CPI or SCPI depending on method - std_Z: Standard deviation (d,) - se_Z: Standard error (d,) - phi_X: X-space importance (d,) - transformed via Jacobian - std_X: Standard deviation (d,) - se_X: Standard error (d,) When method=’both’, also includes phi_Z_scpi, std_Z_scpi, se_Z_scpi.

Return type:

Example with CPI (default):

from fdfi.explainers import FlowExplainer

explainer = FlowExplainer(
    model.predict,
    X_background,
    fit_flow=True,           # Fit normalizing flow during init
    method='cpi',            # CPI (default)
    num_steps=200,           # Flow training iterations
    nsamples=50,             # Monte Carlo samples
    random_state=42,
)
results = explainer(X_test)

print("Z-space importance (CPI):", results["phi_Z"])
print("Confidence intervals:")
ci = explainer.conf_int(alpha=0.05, target="Z")

Example with SCPI (Sobol-CPI):

from fdfi.explainers import FlowExplainer

explainer = FlowExplainer(
    model.predict,
    X_background,
    fit_flow=True,
    method='scpi',           # SCPI (Sobol-CPI)
    num_steps=200,
    nsamples=50,
)
results = explainer(X_test)

print("Importance (SCPI):", results["phi_Z"])

Using external flow models:

from fdfi.explainers import FlowExplainer
from fdfi.models import FlowMatchingModel

# Train flow externally
flow = FlowMatchingModel(X_background, dim=X_background.shape[1])
flow.fit(num_steps=500, verbose='final')

# Use in explainer
explainer = FlowExplainer(model.predict, X_background, fit_flow=False)
explainer.set_flow(flow)
results = explainer(X_test)

DFIExplainer Alias

DFIExplainer is an alias for OTExplainer for backward compatibility:

fdfi.explainers.DFIExplainer: Alias for OTExplainer.

Cross-Fitting (Crossfitting)

The Crossfitting class wraps any of the above explainers and performs K-fold cross-fitting so that the disentanglement map is never evaluated on its own training data. This yields valid standard errors and confidence intervals even when the sample size is small.

class fdfi.explainers.Crossfitting(model, data, explainer_class=<class 'fdfi.explainers.OTExplainer'>, cv=5, y=None, groups=None, cv_kwargs=None, random_state=None, **kwargs)[source]

Bases: Explainer

Cross-fitted DFI explainer for valid inference at small sample sizes.

Wraps any Explainer subclass and performs cross-fitting using a scikit-learn cross-validation splitter. The disentanglement map is fitted on the training portion of each split and importance is evaluated on the held-out portion. Final estimates are the ensemble average of cross-fitted predictors.

Parameters:

model (callable) – The model to explain. Takes (n, d) array, returns (n,) predictions.
data (numpy.ndarray) – Full dataset. Shape (n, d).
explainer_class (type, default=OTExplainer) – The explainer class to instantiate per split. Must be a subclass of Explainer (e.g., OTExplainer, EOTExplainer, FlowExplainer).
cv (int or sklearn cross-validation splitter, default=5) – Controls how data is split for cross-fitting. Pass an int for KFold(n_splits=cv, shuffle=True), or any scikit-learn splitter instance (e.g. KFold, StratifiedKFold, ShuffleSplit, RepeatedKFold, GroupKFold). Any object implementing .split(X, y, groups) is accepted.
y (array-like of shape (n,), optional) – Target / response variable. Required only when using a stratified splitter so that fold assignment preserves class distribution.
groups (array-like of shape (n,), optional) – Group labels for group-aware splitters (GroupKFold, etc.).
random_state (int or None, default=None) – Random seed for the default KFold splitter (when cv is int) and passed to child explainers.
**kwargs (dict) – Additional keyword arguments forwarded to each split’s explainer constructor (e.g., nsamples, epsilon, sampling_method, num_steps).
cv_kwargs (dict | None)

cv_

The resolved cross-validation splitter.

Type:: sklearn splitter instance

fold_explainers

The fitted explainer instances (one per split).

Type:: list[Explainer]

fold_indices

(train_idx, test_idx) for each split.

Type:: list[tuple[numpy.ndarray, numpy.ndarray]]

ueifs_X

Per-sample X-space UEIFs, shape (n, d), after calling with X=None.

Type:: numpy.ndarray or None

ueifs_Z

Per-sample Z-space UEIFs, shape (n, d), after calling with X=None.

Type:: numpy.ndarray or None

__init__(model, data, explainer_class=<class 'fdfi.explainers.OTExplainer'>, cv=5, y=None, groups=None, cv_kwargs=None, random_state=None, **kwargs)[source]

Initialize the Explainer.

Parameters:

model (Callable[[ndarray], ndarray])
data (ndarray)
explainer_class (type)
cv (int | Any)
y (ndarray | None)
groups (ndarray | None)
cv_kwargs (dict | None)
random_state (int | None)
kwargs (Any)

__call__(X=None, **kwargs)[source]

Compute cross-fitted feature importance.

If X is None, performs full cross-fitting on self.data: each split’s test set is the held-out portion of the data.

If X is provided, uses the ensemble of fitted fold explainers to compute importance on X and averages the results.

Parameters:

X (numpy.ndarray or None) – If None, cross-fit on self.data (recommended for valid inference). If provided, shape (m, d), ensemble-predict on new data.
kwargs (Any)

Returns:

Same format as OTExplainer / FlowExplainer: phi_X, std_X, se_X, phi_Z, std_Z, se_Z.

Return type: