--- title: N-BEATS: Neural Basis Expansion Analysis keywords: fastai sidebar: home_sidebar summary: "API details." description: "API details." nb_path: "nbs/models_nbeats__nbeats.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
# from google.colab import drive
# drive.mount('/content/drive')
# os.chdir('./drive/MyDrive/nixtlats')
# print(os.getcwd())
{% endraw %} {% raw %}
# !pip install torchinfo
# !pip install fastcore
# !pip install s3fs
# !pip install patool
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

class IdentityBasis[source]

IdentityBasis(backcast_size:int, forecast_size:int) :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

class TrendBasis[source]

TrendBasis(degree_of_polynomial:int, backcast_size:int, forecast_size:int) :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

class SeasonalityBasis[source]

SeasonalityBasis(harmonics:int, backcast_size:int, forecast_size:int) :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}
{% endraw %} {% raw %}

class ExogenousBasisInterpretable[source]

ExogenousBasisInterpretable() :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

class ExogenousBasisWavenet[source]

ExogenousBasisWavenet(out_features, in_features, num_levels=4, kernel_size=3, dropout_prob=0) :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

class ExogenousBasisTCN[source]

ExogenousBasisTCN(out_features, in_features, num_levels=4, kernel_size=2, dropout_prob=0) :: Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}
{% endraw %} {% raw %}

init_weights[source]

init_weights(module, initialization)

{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
 
{% endraw %} {% raw %}
from torchinfo import summary

model = _NBEATS(n_time_in=168,
                n_time_out=24,
                n_s=1,
                n_x=10,
                n_s_hidden=100,
                n_x_hidden=30,
                n_polynomials=2,
                n_harmonics=4,
                stack_types=['trend', 'seasonality', 'exogenous_wavenet'],
                n_blocks=[1, 1, 1],
                n_layers=[2, 2, 2],
                n_theta_hidden=3 * [[128, 128]],
                dropout_prob_theta=0,
                activation='SELU',
                initialization='lecun_normal',
                batch_normalization=True,
                shared_weights=True)

# inputs: S, Y, X, insample_mask
# S.shape (n_batch,n_s)
# Y.shape (n_batch,n_time_in+n_time_out) 
# X.shape (n_batch,n_x,n_time_in+n_time_out)
# insample_mask.shape (n_batch,n_time_in+n_time_out)
# summary(model, input_size=[(256, 1), (256, 168+24), (256, 10, 168+24), (256, 168+24)])
{% endraw %}

N-BEATS model wrapper

{% raw %}
{% endraw %} {% raw %}

class NBEATS[source]

NBEATS(n_time_in, n_time_out, n_x, n_x_hidden, n_s, n_s_hidden, shared_weights, activation, initialization, stack_types, n_blocks, n_layers, n_harmonics, n_polynomials, n_theta_hidden, batch_normalization, dropout_prob_theta, learning_rate, lr_decay, lr_decay_step_size, weight_decay, loss_train, loss_hypar, loss_valid, frequency, random_seed, seasonality) :: LightningModule

Helper class that provides a standard way to create an ABC using inheritance.

{% endraw %} {% raw %}
{% endraw %}

N-BEATS Usage Example

Load Data

{% raw %}
import multiprocessing

import pandas as pd
from nixtlats.data.datasets.epf import EPF#, EPFInfo
from nixtlats.data.tsloader import TimeSeriesLoader

import pylab as plt
from pylab import rcParams
plt.style.use('seaborn-whitegrid')
plt.rcParams['font.family'] = 'serif'

FONTSIZE = 19

# Load and plot data
Y_df, X_df, S_df = EPF.load_groups(directory='./data', groups=['NP'])

fig = plt.figure(figsize=(15, 6))
plt.plot(Y_df.ds, Y_df.y.values, color='#628793', linewidth=0.4)
plt.ylabel('Price [EUR/MWh]', fontsize=19)
plt.xlabel('Date', fontsize=15)
# plt.savefig('./results/NP.png', bbox_inches = 'tight')
plt.show()
{% endraw %}

Declare Model and Data Parameters

{% raw %}
mc = {}
mc['model'] = 'nbeats'
mc['mode'] = 'simple'
mc['activation'] = 'SELU'

mc['n_time_in'] = 24*7
mc['n_time_out'] = 24*7
mc['n_x_hidden'] = 8
mc['n_s_hidden'] = 0

# mc['input_size_multiplier'] = 7
# mc['output_size'] = 24

mc['stack_types'] = ['trend', 'seasonality', 'exogenous_wavenet']
mc['n_blocks'] = [1, 1, 1]
mc['n_layers'] = [2, 2, 2]
# mc['stack_types'] = ['trend', 'seasonality']
# mc['n_blocks'] = [1, 1]
# mc['n_layers'] = [2, 2]

mc['n_hidden'] = 128
mc['shared_weights'] = False
mc['n_harmonics'] = 4
mc['n_polynomials'] = 2

# Optimization and regularization parameters
mc['initialization'] = 'lecun_normal'
mc['learning_rate'] = 0.0007
mc['batch_size'] = 128
mc['lr_decay'] = 0.5
mc['lr_decay_step_size'] = 2
mc['max_epochs'] = 1#_000
mc['max_steps'] = 20#_000
mc['early_stop_patience'] = 20
mc['eval_freq'] = 500
mc['batch_normalization'] = False
mc['dropout_prob_theta'] = 0.51
mc['dropout_prob_exogenous'] = 0.44
mc['l1_theta'] = 0
mc['weight_decay'] = 0
mc['loss_train'] = 'MAE'
mc['loss_hypar'] = 0.5
mc['loss_valid'] = mc['loss_train']
mc['random_seed'] = 1

# Data Parameters
mc['len_sample_chunks'] = None
mc['idx_to_sample_freq'] = 1
mc['val_idx_to_sample_freq'] = 24 * 7
mc['n_val_weeks'] = 52
mc['window_sampling_limit'] = 500_000
mc['normalizer_y'] = None
mc['normalizer_x'] = 'median'
mc['complete_inputs'] = False
mc['frequency'] = 'H'
mc['seasonality'] = 24

# # Within decomposition
mc['learning_rate'] = 0.0005
mc['batch_size'] = 256
mc['weight_decay'] = 0.00006
mc['n_harmonics'] = 2
mc['n_polynomials'] = 4
mc['dropout_prob_theta'] = 0
mc['dropout_prob_exogenous'] = 0

print(65*'=')
print(pd.Series(mc))
print(65*'=')

mc['n_theta_hidden'] = len(mc['stack_types']) * [ [int(mc['n_hidden']), int(mc['n_hidden'])] ]
=================================================================
model                                                      nbeats
mode                                                       simple
activation                                                   SELU
n_time_in                                                     168
n_time_out                                                    168
n_x_hidden                                                      8
n_s_hidden                                                      0
stack_types               [trend, seasonality, exogenous_wavenet]
n_blocks                                                [1, 1, 1]
n_layers                                                [2, 2, 2]
n_hidden                                                      128
shared_weights                                              False
n_harmonics                                                     2
n_polynomials                                                   4
initialization                                       lecun_normal
learning_rate                                              0.0005
batch_size                                                    256
lr_decay                                                      0.5
lr_decay_step_size                                              2
max_epochs                                                      1
max_steps                                                      20
early_stop_patience                                            20
eval_freq                                                     500
batch_normalization                                         False
dropout_prob_theta                                              0
dropout_prob_exogenous                                          0
l1_theta                                                        0
weight_decay                                              0.00006
loss_train                                                    MAE
loss_hypar                                                    0.5
loss_valid                                                    MAE
random_seed                                                     1
len_sample_chunks                                            None
idx_to_sample_freq                                              1
val_idx_to_sample_freq                                        168
n_val_weeks                                                    52
window_sampling_limit                                      500000
normalizer_y                                                 None
normalizer_x                                               median
complete_inputs                                             False
frequency                                                       H
seasonality                                                    24
dtype: object
=================================================================
{% endraw %}

Instantiate Loaders and Model

{% raw %}
from nixtlats.experiments.utils import create_datasets

train_dataset, val_dataset, test_dataset, scaler_y = create_datasets(mc=mc,
                                                                     S_df=S_df, Y_df=Y_df, X_df=X_df,
                                                                     f_cols=['Exogenous1', 'Exogenous2'],
                                                                     ds_in_val=294*24,
                                                                     ds_in_test=728*24,
                                                                     n_uids=None, n_val_windows=None,freq=None, 
                                                                     is_val_random=False)

train_loader = TimeSeriesLoader(dataset=train_dataset,
                                batch_size=int(mc['batch_size']),
                                #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                                shuffle=True)

val_loader = TimeSeriesLoader(dataset=val_dataset,
                              batch_size=int(mc['batch_size']),
                              #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                              shuffle=False)

test_loader = TimeSeriesLoader(dataset=test_dataset,
                               batch_size=int(mc['batch_size']),
                               #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                               shuffle=False)

mc['n_x'], mc['n_s'] = train_dataset.get_n_variables()
INFO:root:Train Validation splits

INFO:root:                              ds                    
                             min                 max
unique_id sample_mask                               
NP        0           2016-03-08 2018-12-24 23:00:00
          1           2013-01-01 2016-03-07 23:00:00
INFO:root:
Total data 			52416 time stamps 
Available percentage=100.0, 	52416 time stamps 
Insample  percentage=53.21, 	27888 time stamps 
Outsample percentage=46.79, 	24528 time stamps 

INFO:root:Train Validation splits

INFO:root:                              ds                    
                             min                 max
unique_id sample_mask                               
NP        0           2013-01-01 2018-12-24 23:00:00
          1           2016-03-08 2016-12-26 23:00:00
INFO:root:
Total data 			52416 time stamps 
Available percentage=100.0, 	52416 time stamps 
Insample  percentage=13.46, 	7056 time stamps 
Outsample percentage=86.54, 	45360 time stamps 

INFO:root:Train Validation splits

INFO:root:                              ds                    
                             min                 max
unique_id sample_mask                               
NP        0           2013-01-01 2016-12-26 23:00:00
          1           2016-12-27 2018-12-24 23:00:00
INFO:root:
Total data 			52416 time stamps 
Available percentage=100.0, 	52416 time stamps 
Insample  percentage=33.33, 	17472 time stamps 
Outsample percentage=66.67, 	34944 time stamps 

{% endraw %} {% raw %}
model = NBEATS(n_time_in=int(mc['n_time_in']),
               n_time_out=int(mc['n_time_out']),
               n_x=mc['n_x'],
               n_s=mc['n_s'],
               n_s_hidden=int(mc['n_s_hidden']),
               n_x_hidden=int(mc['n_x_hidden']),
               shared_weights=mc['shared_weights'],
               initialization=mc['initialization'],
               activation=mc['activation'],
               stack_types=mc['stack_types'],
               n_blocks=mc['n_blocks'],
               n_layers=mc['n_layers'],
               n_theta_hidden=mc['n_theta_hidden'],
               n_harmonics=int(mc['n_harmonics']),
               n_polynomials=int(mc['n_polynomials']),
               batch_normalization = mc['batch_normalization'],
               dropout_prob_theta=mc['dropout_prob_theta'],
               learning_rate=float(mc['learning_rate']),
               lr_decay=float(mc['lr_decay']),
               lr_decay_step_size=float(mc['lr_decay_step_size']),
               weight_decay=mc['weight_decay'],
               loss_train=mc['loss_train'],
               loss_hypar=float(mc['loss_hypar']),
               loss_valid=mc['loss_valid'],
               frequency=mc['frequency'],
               seasonality=int(mc['seasonality']),
               random_seed=int(mc['random_seed']))
{% endraw %}

Train Model

{% raw %}
from pytorch_lightning.callbacks import EarlyStopping

early_stopping = EarlyStopping(monitor="val_loss", 
                               min_delta=1e-4, 
                               patience=mc['early_stop_patience'],
                               verbose=False,
                               mode="min")

trainer = pl.Trainer(max_epochs=mc['max_epochs'], 
                     max_steps=mc['max_steps'],
                     gradient_clip_val=1.0,
                     progress_bar_refresh_rate=10, 
                     log_every_n_steps=500, 
                     check_val_every_n_epoch=1,
                     callbacks=[early_stopping])

trainer.fit(model, train_loader, val_loader)
GPU available: True, used: False
TPU available: False, using: 0 TPU cores
/home/ubuntu/anaconda3/envs/nixtla/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: GPU available but not used. Set the gpus flag in your trainer `Trainer(gpus=1)` or script `--gpus=1`.
  warnings.warn(*args, **kwargs)

  | Name  | Type    | Params
----------------------------------
0 | model | _NBEATS | 1.6 M 
----------------------------------
1.5 M     Trainable params
113 K     Non-trainable params
1.6 M     Total params
6.436     Total estimated model params size (MB)
/home/ubuntu/anaconda3/envs/nixtla/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: The dataloader, val dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  warnings.warn(*args, **kwargs)
/home/ubuntu/anaconda3/envs/nixtla/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: The dataloader, train dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  warnings.warn(*args, **kwargs)
{% endraw %}

Make Predictions

{% raw %}
model.return_decomposition = True
outputs = trainer.predict(model, val_loader)

print("outputs[0][0].shape", outputs[0][0].shape)
print("outputs[0][1].shape", outputs[0][1].shape)
print("outputs[0][2].shape", outputs[0][2].shape)
/home/ubuntu/anaconda3/envs/nixtla/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: The dataloader, predict dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  warnings.warn(*args, **kwargs)
outputs[0][0].shape torch.Size([42, 168])
outputs[0][1].shape torch.Size([42, 168])
outputs[0][2].shape torch.Size([42, 4, 168])
{% endraw %}