{% if favicon_base64 %} {% endif %}

Model Validation Report

Resilience Analysis Report

Resilience Score
0.9 %} stroke="#28a745" {% elif (resilience_score|default(0)) > 0.7 %} stroke="#ffc107" {% elif (resilience_score|default(0)) > 0.5 %} stroke="#fd7e14" {% else %} stroke="#dc3545" {% endif %} stroke-width="10" stroke-dasharray="{{ (resilience_score|default(0) * 314) }} 314" transform="rotate(-90 60 60)" > {{ (resilience_score|default(0) * 100) | round(1) }}%
{{ avg_performance_gap|default(0) | round(3) }} Perf. Gap
{{ dist_shift|default(0) | round(3) }} Dist. Shift
{% if (resilience_score|default(0)) > 0.9 %} Excellent resilience {% elif (resilience_score|default(0)) > 0.7 %} Good resilience {% elif (resilience_score|default(0)) > 0.5 %} Moderate resilience {% else %} Needs improvement in resilience {% endif %}
Model Information
{% if sensitive_features %} {% endif %} {% if report_data and report_data.alternative_models %} {% endif %}
Type: {{ model_type|default('Unknown') }}
Features: {{ features|length|default(0) }}
Primary Metric: {{ metric|default('Accuracy')|upper }}
Sensitive Features: {{ sensitive_features|length|default(0) }}
Alternative Models: {{ report_data.alternative_models|length|default(0) }}
Test Summary
{% if most_affected_scenario %} {% endif %}
Resilience Score: {{ resilience_score|default(0)|round(4) }}
Performance Gap: {{ avg_performance_gap|default(0)|round(4) }}
Distribution Shift: {{ dist_shift|default(0)|round(4) }}
Most Affected: {{ most_affected_scenario }}

Test Information

Test Type

{{ test_type|capitalize }}
Static report

Model Type

{{ model_type }}
Algorithm

Features

{{ features|length }}
Total features
{% if sensitive_features %}

Sensitive Features

{{ sensitive_features|length }}
For resilience analysis
{% endif %}

Test Configuration

{% if sensitive_features %} {% endif %}
Generation Time {{ timestamp }}
Sensitive Features {{ sensitive_features|join(', ') }}
Metric {{ metric|default('Accuracy') }}
Report Type Static (non-interactive)

Performance Metrics

Resilience Metrics

{% if metrics %} {% for metric_name in metrics|sort %} {% if metric_name not in ['resilience_score', 'performance_gap', 'distribution_shift'] %} {% endif %} {% endfor %} {% endif %} {% if metrics %} {% for metric_name, metric_value in metrics.items() %} {% if metric_name not in ['resilience_score', 'performance_gap', 'distribution_shift'] %} {% endif %} {% endfor %} {% endif %}
Model Resilience Score Performance Gap Distribution Shift{{ metric_name|title }}
{{ model_name }} {{ "%.4f"|format(resilience_score) }} {{ "%.4f"|format(avg_performance_gap) }} {{ "%.4f"|format(dist_shift|default(0)) }}{{ "%.4f"|format(metric_value|default(0)) }}

Overview

Resilience Score

{{ "%.4f"|format(resilience_score) }}
Higher is better

Performance Gap

{{ "%.4f"|format(avg_performance_gap) }}
Lower is better

Distribution Shift

{{ "%.4f"|format(dist_shift|default(0)) }}
Feature distribution change
{% if most_affected_scenario %}

Most Affected Scenario

{{ most_affected_scenario }}
Worst performing
{% endif %}

Resilience Metrics Overview

Shows the key resilience metrics for the model, including resilience score and performance gap.

{% if charts.resilience_score_chart %}
Resilience metrics overview
{% else %}

No resilience metrics data available for visualization.

{% endif %}

Performance Comparison: Worst vs Remaining Samples

Compares model performance on the worst-affected samples vs. the remaining dataset. A smaller gap indicates better resilience.

{% if charts.performance_gap_chart %}
Performance gap analysis
{% else %}

No performance gap data available for visualization.

{% endif %}

Feature Analysis

{% if charts.feature_distribution_shift %}

Feature Distribution Shift

Shows the distribution shift of features between normal and stressed conditions, highlighting features with significant changes.

Feature distribution shift
{% endif %} {% if charts.critical_feature_distributions %}

Critical Feature Distributions

Compares the distributions of the most critical features between normal and stressed conditions.

Critical feature distributions
{% endif %} {% if charts.feature_importance_chart %}

Feature Importance for Resilience

Shows the most important features affecting model resilience. Features with higher importance have greater impact on model performance under stress.

Feature importance for resilience
{% endif %} {% if charts.feature_comparison_chart %}

Feature Importance: Model vs Resilience Analysis

Compares feature importance from the model's original training versus from resilience analysis, highlighting differences in what impacts general performance versus resilience.

Feature comparison
{% endif %} {% if feature_importance %}

Feature Importance

{% for feature, importance in feature_importance|dictsort(by='value', reverse=true) %} {% endfor %}
Feature Importance
{{ feature }} {{ "%.4f"|format(importance) }}
{% endif %}
{% if charts.residual_distribution or charts.feature_residual_correlation %}

Residual Analysis

{% if charts.residual_distribution %}

Model Residual Distribution

Shows the distribution of residuals (prediction errors) across different datasets, helping identify biases under stress conditions.

Residual distribution
{% endif %} {% if charts.feature_residual_correlation %}

Feature-Residual Correlation

Shows which features are most correlated with model errors, helping identify potential areas for model improvement.

Feature-residual correlation
{% endif %}
{% endif %} {% if charts.distance_metrics_comparison or charts.feature_distance_heatmap %}

Distribution Metrics Analysis

{% if charts.distance_metrics_comparison %}

Distance Metrics Comparison by Alpha

Compares different distance metrics (PSI, WD1, KS, etc.) across alpha levels, showing how distribution shift is captured by different metrics.

Distance metrics comparison by alpha
{% endif %} {% if charts.feature_distance_heatmap %}

Feature Distance Heatmap by Metric

Shows the distribution shift of each feature as measured by different metrics, visualizing which features are most affected by different types of distribution shifts.

Feature distance heatmap by metric
{% endif %}
{% endif %} {% if charts.model_comparison_chart or charts.performance_gap_by_alpha or charts.model_resilience_scores %}

Model Comparison

{% if charts.model_comparison_chart %}

Model Resilience Comparison

Compares resilience performance across different models under increasing stress levels. Models with more gradual decline are more resilient.

Model resilience comparison
{% endif %} {% if charts.performance_gap_by_alpha %}

Performance Gap by Alpha Level

Shows how the performance gap changes across different alpha levels for each model. Models with smaller gaps at higher alpha levels demonstrate better resilience.

Performance gap by alpha level
{% endif %} {% if charts.model_resilience_scores %}

Resilience Scores by Model

Compares the overall resilience score for each model. Higher scores indicate better performance under distribution shifts.

Model resilience scores
{% endif %} {% if charts.distance_metrics_comparison %}

Distance Metrics Comparison by Alpha

Compares different distance metrics (PSI, WD1, KS, etc.) across alpha levels, showing how distribution shift is captured by different metrics.

Distance metrics comparison by alpha
{% endif %}
{% endif %}