This report analyzes how well the model maintains performance when distribution shifts occur between baseline and target data.
Model: RandomForestClassifier
Date: 2025-04-21 18:34:24
Good resilience with limited performance degradation under distribution shifts.
This overview provides a high-level assessment of the model's resilience to distribution shifts between baseline and target distributions.
This analysis evaluates how the model's performance changes when data distributions shift. A resilient model maintains consistent performance despite distribution shifts between training and deployment data.
Performance gap chart data will display here.
Distribution shift chart data will display here.
Feature impact chart data will display here.
The model experiences a moderate impact when exposed to distribution shifts, with an average performance gap of 12.0%.
The dataset exhibits minor distribution shifts, with an average distance metric of 0.14.
5 features show high sensitivity to distribution shifts, with the greatest impact observed in the most affected features.
7 different shift scenarios were analyzed to evaluate the model's resilience under various conditions.
This table shows model performance across different distribution shift scenarios.
Shift Scenario | Baseline Performance | Target Performance | Performance Gap | Shift Magnitude | Resilience Score |
---|---|---|---|---|---|
Loading shift results data...
|
This table analyzes how distribution shifts in each feature impact model performance.
Feature | Type | Feature Importance | Shift Magnitude | Performance Impact | Resilience Impact | Shift Sensitivity |
---|---|---|---|---|---|---|
Loading feature impact data...
|
Set up ongoing monitoring of feature distributions in production to detect shifts early.
This section provides detailed metrics and performance analysis for each shift scenario.
This table provides detailed performance metrics for each distribution shift scenario across multiple evaluation metrics.
Shift Scenario | Accuracy | F1 Score | AUC | Composite Score | ||||||
---|---|---|---|---|---|---|---|---|---|---|
Baseline | Target | Gap | Baseline | Target | Gap | Baseline | Target | Gap | ||
Loading detailed metrics data...
|
Metric | Avg. Baseline | Avg. Target | Avg. Gap | Max Gap | Resilience |
---|---|---|---|---|---|
Accuracy | - | - | - | - | - |
F1 Score | - | - | - | - | - |
AUC | - | - | - | - | - |
Composite | - | - | - | - | - |
This analysis shows how different data groups or classes are affected by distribution shifts.
This analysis examines individual samples that are most affected by distribution shifts.
Sample ID | Baseline Prediction | Target Prediction | Prediction Change | Key Features | Actions |
---|---|---|---|---|---|
Loading sample data...
|
This section analyzes the distribution shifts between baseline and target datasets.
Statistic | Baseline | Target | Change |
---|---|---|---|
Mean | - | - | - |
Median | - | - | - |
Std Dev | - | - | - |
IQR | - | - | - |
This table provides detailed statistical distance metrics between baseline and target distributions for each feature.
Feature | Type | KL Divergence | JS Distance | Wasserstein Distance | Hellinger Distance | Shift Severity |
---|---|---|---|---|---|---|
Loading distance metrics data...
|
Metric | Range | Minor Shift | Moderate Shift | Significant Shift | Description |
---|---|---|---|---|---|
KL Divergence | [0, ∞) | 0 - 0.5 | 0.5 - 2.0 | > 2.0 | Measures information gain when updating from baseline to target distribution |
JS Distance | [0, 1] | 0 - 0.2 | 0.2 - 0.4 | > 0.4 | Symmetric measure of similarity between distributions |
Wasserstein | [0, ∞) | 0 - 0.1 | 0.1 - 0.3 | > 0.3 | Earth mover's distance between distributions |
Hellinger | [0, 1] | 0 - 0.2 | 0.2 - 0.5 | > 0.5 | Probabilistic measure of distribution similarity |
This analysis examines how feature correlations change between baseline and target distributions.