Models

Model

class sconce.models.Model[source]

The base class of all Models in Sconce. It is only an interface, describing what must be implemented if you want to define a model.

active_parameter_groups

A list of all active parameter groups.

add_parameter_group(group, inactivate_default=True)[source]

Add a new parameter group to this model.

Parameters:
  • group (ParameterGroup) – the parameter group to add.
  • inactivate_default (bool, optional) – if True, then the default parameter group will have is_active set to False.
build_parameter_groups()[source]

This can be overridden to build additional parameter groups. This can be useful if you’re doing different layerwise optimization schedules.

calculate_loss(*, inputs, outputs, targets, **kwargs)[source]

This method must accept arbitrary keyword arguments. The base class of trainer will pass inputs, outputs, and targets, but subclasses may modify that behavior to include other keyword arguments.

It must return a dictionary. The dictionary is expected to include at least the key ‘loss’, but may include any otehr keys you like. The value of the key loss is expected to be the torch.Tensor output of the loss function, used to back-propagate the gradients used by the optimizer.

calculate_metrics(*, inputs, outputs, targets, loss, **kwargs)[source]

This method must accept arbitrary keyword arguments. The base class of trainer will pass inputs, outputs, targets, and loss, but subclasses may modify that behavior to include other keyword arguments.

It must return a dictionary. No restrictions are made on the keys or values of this dictionary.

default_parameter_group

The default parameter group is created automatically and includes all of the trainable parameters for the model.

forward(*, inputs, targets, **kwargs)[source]

It must accept arbitrary keyword arguments. The base class of trainer will pass inputs and targets, but subclasses may modify that behavior to include other keyword arguments.

It must return a dictionary. The dictionary is expected to include at least the key outputs but may include any other keys you like. The value of the key outputs is expected to be the torch.Tensor output of the model, used for calculating the loss.

get_num_trainable_parameters()[source]

The number of trainable parameters that the models has.

get_optimizers()[source]

Returns a list of optimizers for the parameters of this model.

get_parameter_group(name)[source]

Get a parameter group by name.

get_trainable_parameters()[source]

The trainable parameters that the models has.

parameter_groups

A list of all parameter groups, inactive as well as active.

prepare_for_step(step, current_state)[source]

First, it handles any hyperparameter schedules added to the model itself before gathering up the results of calling ‘prepare_for_step’ on all the model’s parameter groups and combining the result.

print_schedule_summary()[source]

Print out a summary of the scheduled hyperparameters on this model and it’s parameter groups.

set_optimizer(*args, **kwargs)[source]

Set the optimizer for all of the active parameter groups on this model.

set_schedule(name, schedule)[source]

Set the schedule for a hyperparameter on this model.

Parameters:
  • name (string) – the name of the hyperparameter you want to schedule.
  • ( (schedule) – py:class:~sconce.schedules.base.Schedule): the schedule for that hyperparameter.

Note

Some name values are interpreted specially. Setting name to ‘learning_rate’, ‘momentum’, or ‘weight_decay’ will delegate to setting schedules on all active parameter groups instead of on the model.

start_session(num_steps)[source]

Called by the :py:class:~sconce.trainer.Trainer when a training session starts.

Parameters:num_steps (int) – the number of steps the trainer will take during this training session.

BasicAutoencoder

class sconce.models.BasicAutoencoder(image_height, image_width, hidden_size, latent_size)[source]

A basic 2D image autoencoder built up of fully connected layers, three each in the encoder and the decoder.

Loss:
This model uses binary cross-entropy for the loss.
Metrics:
None
Parameters:
  • image_height (int) – image height in pixels.
  • image_width (int) – image width in pixels.
  • hidden_size (int) – the number of activations in each of the 4 hidden layers.
  • latent_size (int) – the number of activations in the latent representation (encoder output).

BasicConvolutionalAutoencoder

class sconce.models.BasicConvolutionalAutoencoder(image_channels, conv_channels)[source]

A basic 2D image autoencoder built up of convolutional layers, three each in the encoder and the decoder.

Loss:
This model uses binary cross-entropy for the loss.
Metrics:
None
Parameters:
  • image_channels (int) – the number of channels in the input images.
  • conv_channels (list of python:int) – a list of length three of integers describing the number of channels in each of the three convolutional layers.

VariationalAutoencoder

class sconce.models.VariationalAutoencoder(image_size, image_channels, conv_channels=[32, 32, 32], hidden_sizes=[256, 256], latent_size=10, beta=1.0)[source]

A variational autoencoder built up of convolutional layers and dense layers in the encoder and decoder.

Loss:
This model uses binary cross entropy for the reconstruction loss and KL Divergence for the latent representation loss.
Metrics:
None
Parameters:
  • image_channels (int) – the number of channels in the input images.
  • conv_channels (list of python:int) – a list (of length three) of integers describing the number of channels in each of the three convolutional layers.
  • hidden_sizes (list of python:int) – a list (of length two) of integers describing the number of hidden units in each hidden layer.
  • latent_size (int) – the size of the latent representation.

BasicClassifer

class sconce.models.BasicClassifier(image_height, image_width, image_channels, convolutional_layer_kwargs, fully_connected_layer_kwargs, num_categories=10)[source]

A basic 2D image classifier built up of some number of convolutional layers followed by some number of densly connected layers.

Loss:
This model uses cross-entropy for the loss.
Metrics:
classification_accuracy: [0.0, 1.0] the fraction of correctly predicted labels.
Parameters:
  • image_height (int) – image height in pixels.
  • image_width (int) – image width in pixels.
  • image_channels (int) – number of channels in the input images.
  • convolutional_layer_kwargs (list[dict]) – a list of dictionaries describing the convolutional layers. See Convolution2dLayer for details.
  • fully_connected_layer_kwargs (list[dict]) – a list of dictionaries describing the fully connected layers. See FullyConnectedLayer for details.
  • num_categories (int) – [2, inf) the number of different image classes.
classmethod new_from_yaml_file(yaml_file)[source]

Construct a new BasicClassifier from a yaml file.

Parameters:yaml_file (file) – a file-like object that yaml contents can be read from.

Example yaml file contents:

---
# Values for MNIST and FashionMNIST
image_height: 28
image_width: 28
image_channels: 1
num_categories: 10

# Remaining values are not related to the dataset
convolutional_layer_attributes: ["out_channels", "stride", "padding", "kernel_size"]
convolutional_layer_values:  [ # ==============  ========  =========  =============
                                [16,             1,        4,         9],
                                [8,              2,        1,         3],
                                [8,              2,        1,         3],
                                [8,              2,        1,         3],
                                [8,              2,        1,         3],
]

fully_connected_layer_attributes: ['out_size', 'dropout']
fully_connected_layer_values:  [ # ======      =========
                                  [100,        0.4],
                                  [100,        0.8],
]
classmethod new_from_yaml_filename(yaml_filename)[source]

Construct a new BasicClassifier from a yaml file.

Parameters:filename (path) – the filename of a yaml file. See new_from_yaml_file() for more details.

MultilayerPerceptron

class sconce.models.MultilayerPerceptron(image_height, image_width, image_channels, layer_kwargs, num_categories=10)[source]

A basic 2D image multi-layer perceptron built up of a number of densly connected layers.

Loss:
This model uses cross-entropy for the loss.
Metrics:
classification_accuracy: [0.0, 1.0] the fraction of correctly predicted labels.
Parameters:
  • image_height (int) – image height in pixels.
  • image_width (int) – image width in pixels.
  • image_channels (int) – number of channels in the input images.
  • layer_kwargs (list[dict]) – a list of dictionaries describing layers. See FullyConnectedLayer for details.
  • num_categories (int) – [2, inf) the number of different image classes.
classmethod new_from_yaml_file(yaml_file)[source]

Construct a new MultilayerPerceptron from a yaml file.

Parameters:yaml_file (file) – a file-like object that yaml contents can be read from.

Example yaml file contents:

---
# Values for MNIST and FashionMNIST
image_height: 28
image_width: 28
image_channels: 1
num_categories: 10

layer_attributes: ['out_size', 'dropout', 'with_batchnorm']
layer_values:  [ # ======      =========  ================
                  [100,        0.4,       true],
                  [100,        0.8,       true],
]
classmethod new_from_yaml_filename(yaml_filename)[source]

Construct a new MultilayerPerceptron from a yaml file.

Parameters:filename (path) – the filename of a yaml file. See new_from_yaml_file() for more details.

WideResnetImageClassifier

class sconce.models.WideResnetImageClassifier(image_channels=1, depth=28, widening_factor=10, num_categories=10)[source]

A wide resnet image classifier, based on this paper

Loss:
This model uses cross-entropy for the loss.
Metrics:
classification_accuracy: [0.0, 1.0] the fraction of correctly predicted labels.
Parameters:
  • image_channels (int) – number of channels in the input images.
  • depth (int) – total number of convolutional layers in the network. This should be divisible by (6n + 4) where n is a positive integer.
  • widening_factor (int) – [1, inf) determines how many convolutional channels are in the network (see paper above for details).
  • num_categories (int) – [2, inf) the number of different image classes.