--- title: Giacomini-White Test keywords: fastai sidebar: home_sidebar nb_path: "nbs/losses__gw_test.ipynb" ---
{% raw %}
{% endraw %}

Newey-West estimator

Estimator of the Newey-West Heteroskedasticity and Autocorrelation Consistent Covariance Matrix (Newey-West HAC) to calculate robust error estimations.

{% raw %}
{% endraw %} {% raw %}

Newey_West[source]

Newey_West(Z, n_lags)

Newey-West HAC estimator

Parameters

Z: (n, k) ndarray n_lags: int number of lags to consider as available information.

Returns

omega_hat: Newey-West HAC estimator of the covariance matrix

{% endraw %} {% raw %}
{% endraw %} {% raw %}

GW_CPA_test[source]

GW_CPA_test(loss1:ndarray, loss2:ndarray, tau:int, alpha:float=0.05, conditional:bool=False, verbose:bool=True)

Giacomini-White Conditional Predictive Ability Test

Parameters

loss1: numpy array losses of model 1 loss2: numpy array losses of model 2 tau: int the past information treated as 'available' for the test. unconditional: boolean, True if unconditional (DM test), False if conditional (GW test). verbose: boolean, True if prints of test are needed

Returns

test_stat: test statistic of the conditional predictive ability test crit_val: critical value of the chi-square test for a 5% confidence level p-vals: (k,) p-value of the test

{% endraw %} {% raw %}
{% endraw %} {% raw %}
np.random.seed(1)

loss1 = np.random.randint(low=1, high=10, size=(10,1))
loss2 = np.random.randint(low=1, high=10, size=(10,1))

GW_CPA_test(loss1=loss1, loss2=loss2, tau=1, conditional=True)
Conditional test:

Forecast horizon: 1, Nominal Risk Level: 0.05
Test-statistic: -0.06536064947319775 (-)
Critical value: 5.991464547107979
p-value: 1.0

(-0.06536064947319775, 5.991464547107979, 1.0)
{% endraw %} {% raw %}

GW_test_pvals[source]

GW_test_pvals(y:ndarray, y_hat:ndarray, horizon:int, tau:int, conditional:bool, alpha:float=0.05, verbose:bool=False)

Function to calculate model-pair-wise GW-Test p-values

Parameters

y: numpy array flat array with actual test values y_hat: numpy array matrix with predicted values model_names: string list with the names of the models. horizon: int the multi horizon for which the predictions were created, the test is performed against over the mean differences in said multi horizon losses. tau: int the past information treated as 'available' for the test. alpha: float level of significance for the test. unconditional: boolean, True if unconditional (DM test), False if conditional (GW test). verbose: boolean. True for partial test results.

Returns

p_vals: (n_models, n_models) symmetric numpy array with the model-pair-wise p-values.

{% endraw %} {% raw %}
{% endraw %} {% raw %}

get_nbeatsx_cmap[source]

get_nbeatsx_cmap()

{% endraw %} {% raw %}

get_epftoolbox_cmap[source]

get_epftoolbox_cmap()

{% endraw %} {% raw %}

plot_GW_test_pvals[source]

plot_GW_test_pvals(pvals, labels, title)

{% endraw %} {% raw %}
{% endraw %}

TEST/DEBUG GW TEST

{% raw %}
np.random.seed(117)

# Observed values for 3 different days
y = np.random.randint(low=1, high=10, size=(100*24)) 

# Predicted values for 3 different days, from 5 different models
y_hat = np.random.randint(low=1, high=10, size=(100*24, 5))
model_names = ['Model1', 'Model2', 'Model3', 'Model4', 'Model5']

pvals = GW_test_pvals(y=y, y_hat=y_hat, horizon=24, tau=1,
                      conditional=True, alpha=0.05, verbose=False)

plot_GW_test_pvals(pvals=pvals, labels=model_names, title='GW_test')
losses.shape (5, 2400)
n_models, n_ds, n_fcds
5 2400 100
losses.shape (5, 100)
{% endraw %} {% raw %}
# data = pd.read_csv('./results/ver.csv')

# model_names = ['AR1', 'ESRNN', 'NBEATS', 'ARX1', 'LEAR', 'DNN', 'NBEATS_X_G', 'NBEATS_X_I']
# pvals = GW_test_pvals(y=np.expand_dims(data['y'].to_numpy(), axis=0),
#                       y_hat=data.loc[:, model_names].to_numpy(),
#                       horizon=24,
#                       tau=1,
#                       conditional=True,
#                       alpha=0.05,
#                       verbose=False)

# plot_GW_test_pvals(pvals=pvals, labels=model_names, title='GW_test')
{% endraw %}