thelper.nn package

Neural network and model package.

This package contains classes that define blocks and modules used in various neural network architectures. Most of these classes have been adapted from external sources; see their individual headers for more information.

Submodules

thelper.nn.common module

class thelper.nn.common.ConvBlock(input_size, output_size, kernel_size=4, stride=2, padding=1, bias=True, activation='relu', norm='batch', groups=1, prelu_params=1)[source]

Bases: torch.nn.modules.module.Module

__init__(input_size, output_size, kernel_size=4, stride=2, padding=1, bias=True, activation='relu', norm='batch', groups=1, prelu_params=1)[source]

Initialize self. See help(type(self)) for accurate signature.

forward_act_bn(x)[source]
forward_bn_act(x)[source]
class thelper.nn.common.DeconvBlock(input_size, output_size, kernel_size=4, stride=2, padding=1, bias=True, activation='relu', norm='batch')[source]

Bases: torch.nn.modules.module.Module

__init__(input_size, output_size, kernel_size=4, stride=2, padding=1, bias=True, activation='relu', norm='batch')[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.common.DenseBlock(input_size, output_size, bias=True, activation='relu', norm='batch')[source]

Bases: torch.nn.modules.module.Module

__init__(input_size, output_size, bias=True, activation='relu', norm='batch')[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.common.PSBlock(input_size, output_size, scale_factor, kernel_size=3, stride=1, padding=1, bias=True, activation='relu', norm='batch')[source]

Bases: torch.nn.modules.module.Module

__init__(input_size, output_size, scale_factor, kernel_size=3, stride=1, padding=1, bias=True, activation='relu', norm='batch')[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.common.ResNetBlock(num_filter, kernel_size=3, stride=1, padding=1, bias=True, activation='relu', norm='batch')[source]

Bases: torch.nn.modules.module.Module

__init__(num_filter, kernel_size=3, stride=1, padding=1, bias=True, activation='relu', norm='batch')[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.common.Upsample2xBlock(input_size, output_size, bias=True, upsample='deconv', activation='relu', norm='batch')[source]

Bases: torch.nn.modules.module.Module

__init__(input_size, output_size, bias=True, upsample='deconv', activation='relu', norm='batch')[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

thelper.nn.common.shave(imgs, border_size=0)[source]
thelper.nn.common.weights_init_kaiming(m)[source]
thelper.nn.common.weights_init_xavier(m)[source]

thelper.nn.coordconv module

class thelper.nn.coordconv.AddCoords(radius_channel=False)[source]

Bases: torch.nn.modules.module.Module

__init__(radius_channel=False)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(in_tensor)[source]

in_tensor: (batch_size, channels, x_dim, y_dim) [0,0,0,0] [0,1,2,3] [1,1,1,1] [0,1,2,3] << (i,j)th coordinates of pixels added as separate channels [2,2,2,2] [0,1,2,3] taken from mkocabas.

class thelper.nn.coordconv.CoordConv2d(in_channels, *args, radius_channel=False, **kwargs)[source]

Bases: torch.nn.modules.module.Module

add any additional coordinate channels to the input tensor

__init__(in_channels, *args, radius_channel=False, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(in_tensor)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.coordconv.CoordConvTranspose(radius_channel, *args, **kwargs)[source]

Bases: torch.nn.modules.module.Module

CoordConvTranspose layer for segmentation tasks.

__init__(radius_channel, *args, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(in_tensor)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

thelper.nn.densenet module

class thelper.nn.densenet.DenseNet(growth_rate=32, block_config=(6, 12, 24, 16), num_init_features=64, bn_size=4, drop_rate=0, num_classes=1000)[source]

Bases: torch.nn.modules.module.Module

Densenet-BC model class, based on “Densely Connected Convolutional Networks”

Parameters
  • growth_rate (int) – how many filters to add each layer (k in paper)

  • block_config (list of 4 ints) – how many layers in each pooling block

  • num_init_features (int) – the number of filters to learn in the first convolution layer

  • bn_size (int) – multiplicative factor for number of bottle neck layers (i.e. bn_size * k features in the bottleneck layer)

  • drop_rate (float) – dropout rate after each dense layer

  • num_classes (int) – number of classification classes

__init__(growth_rate=32, block_config=(6, 12, 24, 16), num_init_features=64, bn_size=4, drop_rate=0, num_classes=1000)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

thelper.nn.densenet.densenet121(pretrained=False, **kwargs)[source]

Densenet-121 model from “Densely Connected Convolutional Networks”

Parameters

pretrained (bool) – If True, returns a model pre-trained on ImageNet

thelper.nn.densenet.densenet169(pretrained=False, **kwargs)[source]

Densenet-169 model from “Densely Connected Convolutional Networks”

Parameters

pretrained (bool) – If True, returns a model pre-trained on ImageNet

thelper.nn.densenet.densenet201(pretrained=False, **kwargs)[source]

Densenet-201 model from “Densely Connected Convolutional Networks”

Parameters

pretrained (bool) – If True, returns a model pre-trained on ImageNet

thelper.nn.densenet.densenet161(pretrained=False, **kwargs)[source]

Densenet-161 model from “Densely Connected Convolutional Networks”

Parameters

pretrained (bool) – If True, returns a model pre-trained on ImageNet

thelper.nn.fcn module

class thelper.nn.fcn.FCN32s(task, init_vgg16=True)[source]

Bases: thelper.nn.utils.Module

__init__(task, init_vgg16=True)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

init_vgg16_params(vgg16, copy_fc8=True)[source]
set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

thelper.nn.fcn.get_upsampling_weight(in_channels, out_channels, kernel_size)[source]

thelper.nn.inceptionresnetv2 module

class thelper.nn.inceptionresnetv2.BasicConv2d(in_planes, out_planes, kernel_size, stride, padding=0)[source]

Bases: torch.nn.modules.module.Module

__init__(in_planes, out_planes, kernel_size, stride, padding=0)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.Block17(scale=1.0)[source]

Bases: torch.nn.modules.module.Module

__init__(scale=1.0)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.Block35(scale=1.0)[source]

Bases: torch.nn.modules.module.Module

__init__(scale=1.0)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.Block8(scale=1.0, noReLU=False)[source]

Bases: torch.nn.modules.module.Module

__init__(scale=1.0, noReLU=False)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.InceptionResNetV2(task, input_channels=3)[source]

Bases: thelper.nn.utils.Module

__init__(task, input_channels=3)[source]

Receives a task object to hold internally for model specialization.

features(input)[source]
forward(input)[source]

Transforms an input tensor in order to generate a prediction.

logits(features)[source]
set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

class thelper.nn.inceptionresnetv2.Mixed_5b[source]

Bases: torch.nn.modules.module.Module

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.Mixed_6a[source]

Bases: torch.nn.modules.module.Module

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.inceptionresnetv2.Mixed_7a[source]

Bases: torch.nn.modules.module.Module

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

thelper.nn.lenet module

class thelper.nn.lenet.LeNet(task, input_shape=(1, 28, 28), conv1_filters=6, conv2_filters=16, hidden1_size=120, hidden2_size=84, output_size=10)[source]

Bases: thelper.nn.utils.Module

LeNet CNN implementation.

See http://yann.lecun.com/exdb/lenet/ for more information.

This is NOT a modern architecture; it is only provided here for tutorial purposes.

__init__(task, input_shape=(1, 28, 28), conv1_filters=6, conv2_filters=16, hidden1_size=120, hidden2_size=84, output_size=10)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

thelper.nn.mobilenet module

class thelper.nn.mobilenet.InvertedResidual(inp, oup, stride, expand_ratio)[source]

Bases: torch.nn.modules.module.Module

__init__(inp, oup, stride, expand_ratio)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.mobilenet.MobileNetV2(task, input_size=224, width_mult=1.0)[source]

Bases: thelper.nn.utils.Module

__init__(task, input_size=224, width_mult=1.0)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

thelper.nn.mobilenet.conv_1x1_bn(inp, oup)[source]
thelper.nn.mobilenet.conv_bn(inp, oup, stride)[source]

thelper.nn.resnet module

class thelper.nn.resnet.BasicBlock(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Bases: thelper.nn.resnet.Module

__init__(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.resnet.Bottleneck(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Bases: thelper.nn.resnet.Module

__init__(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Initialize self. See help(type(self)) for accurate signature.

expansion = 4
forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.resnet.ConvTailNet(n_inputs, num_classes)[source]

Bases: torch.nn.modules.module.Module

__init__(n_inputs, num_classes)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.resnet.FCResNet(task, ckptdata, map_location='cpu', avgpool_size=0)[source]

Bases: thelper.nn.resnet.ResNet

__init__(task, ckptdata, map_location='cpu', avgpool_size=0)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

class thelper.nn.resnet.Module(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Bases: torch.nn.modules.module.Module

__init__(inplanes, planes, stride=1, downsample=None, coordconv=False, radius_channel=True)[source]

Initialize self. See help(type(self)) for accurate signature.

expansion = 1
class thelper.nn.resnet.ResNet(task, block=<class 'thelper.nn.resnet.BasicBlock'>, layers=[3, 4, 6, 3], strides=[1, 2, 2, 2], input_channels=3, flexible_input_res=False, pool_size=7, coordconv=False, radius_channel=True, pretrained=False)[source]

Bases: thelper.nn.utils.Module

__init__(task, block=<class 'thelper.nn.resnet.BasicBlock'>, layers=[3, 4, 6, 3], strides=[1, 2, 2, 2], input_channels=3, flexible_input_res=False, pool_size=7, coordconv=False, radius_channel=True, pretrained=False)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

class thelper.nn.resnet.ResNetFullyConv(task, block=<class 'thelper.nn.resnet.BasicBlock'>, layers=[3, 4, 6, 3], strides=[1, 2, 2, 2], input_channels=3, flexible_input_res=False, pool_size=7, coordconv=False, radius_channel=True, pretrained=False)[source]

Bases: thelper.nn.resnet.ResNet

__init__(task, block=<class 'thelper.nn.resnet.BasicBlock'>, layers=[3, 4, 6, 3], strides=[1, 2, 2, 2], input_channels=3, flexible_input_res=False, pool_size=7, coordconv=False, radius_channel=True, pretrained=False)[source]

Receives a task object to hold internally for model specialization.

forward(x)[source]

Transforms an input tensor in order to generate a prediction.

set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

class thelper.nn.resnet.SqueezeExcitationBlock(inplanes, planes, stride=1, downsample=None, reduction=16, coordconv=False, radius_channel=True)[source]

Bases: thelper.nn.resnet.Module

__init__(inplanes, planes, stride=1, downsample=None, reduction=16, coordconv=False, radius_channel=True)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class thelper.nn.resnet.SqueezeExcitationLayer(channel, reduction=16)[source]

Bases: torch.nn.modules.module.Module

__init__(channel, reduction=16)[source]

Initialize self. See help(type(self)) for accurate signature.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

thelper.nn.utils module

Neural network utility functions and classes.

This module contains base interfaces and utility functions used to define and instantiate neural network models.

class thelper.nn.utils.ExternalClassifModule(model_type, task, **kwargs)[source]

Bases: thelper.nn.utils.ExternalModule

External model interface specialization for classification tasks.

This interface will try to ‘rewire’ the last fully connected layer of the models it instantiates to match the number of classes to predict defined in the task object.

__init__(model_type, task, **kwargs)[source]

Receives a task object to hold internally for model specialization, and tries to rewire the last ‘fc’ layer.

set_task(task)[source]

Rewires the last fully connected layer of the wrapped network to fit the given number of classification targets.

class thelper.nn.utils.ExternalDetectModule(model_type, task, **kwargs)[source]

Bases: thelper.nn.utils.ExternalModule

External model interface specialization for object detection tasks.

This interface will try to ‘rewire’ the last fully connected layer of the models it instantiates to match the number of classes to predict defined in the task object.

__init__(model_type, task, **kwargs)[source]

Receives a task object to hold internally for model specialization, and tries to rewire the last ‘fc’ layer.

set_task(task)[source]

Rewires the last fully connected layer of the wrapped network to fit the given number of classification targets.

class thelper.nn.utils.ExternalModule(model_type, task, **kwargs)[source]

Bases: thelper.nn.utils.Module

Model inteface used to hold a task object for an external implementation.

This interface is built on top of torch.nn.Module and should remain fully compatible with it. It is automatically used when instantiating a model via thelper.nn.utils.create_model() that is not derived from thelper.nn.utils.Module. Its only purpose is to hold the task object, and redirect thelper.nn.utils.Module.forward() to the actual model’s transformation function. It can also be specialized to automatically adapt some external models after their construction using the knowledge contained in the task object.

__init__(model_type, task, **kwargs)[source]

Receives a task object to hold internally for model specialization.

forward(*input, **kwargs)[source]

Transforms an input tensor in order to generate a prediction.

get_name()[source]

Returns the name of this module (by default, the fully qualified class name of the external model).

load_state_dict(state_dict, strict=True)[source]

Loads the state dict of an external model.

set_task(task)[source]

Stores the new task internally.

Note that since this external module handler is generic, it does not know what to do with the task, so it just assumes that the model is already set up. Specialized external module handlers will instead attempt to modify the model they wrap.

state_dict(destination=None, prefix='', keep_vars=False)[source]

Returns the state dict of the external model.

summary()[source]

Prints a summary of the model using the thelper.nn logger.

class thelper.nn.utils.Module(task, **kwargs)[source]

Bases: torch.nn.modules.module.Module

Model interface used to hold a task object.

This interface is built on top of torch.nn.Module and should remain fully compatible with it.

All models used in the framework should derive from this interface, and therefore expect a task object as the first argument of their constructor. Their implementation may decide to ignore this task object when building their internal layers, but using it should help specialize the network by specifying e.g. the number of classes to support.

__init__(task, **kwargs)[source]

Receives a task object to hold internally for model specialization.

abstract forward(*input)[source]

Transforms an input tensor in order to generate a prediction.

get_name()[source]

Returns the name of this module (by default, its fully qualified class name).

abstract set_task(task)[source]

Adapts the model to support a new task, replacing layers if needed.

summary()[source]

Prints a summary of the model using the thelper.nn logger.

thelper.nn.utils.create_model(config, task, save_dir=None, ckptdata=None)[source]

Instantiates a model based on a provided task object.

The configuration must be given as a dictionary object. This dictionary will be parsed for a ‘model’ field. This field is expected to be a dictionary itself. It may then specify a type to instantiate as well as the parameters to provide to that class constructor, or a path to a checkpoint from which a model should be loaded.

All models must derive from thelper.nn.utils.Module, or they must be instantiable through thelper.nn.utils.ExternalModule (or one of its specialized classes). The provided task object will be used to make sure that the model has the required input/output layers for the requested objective.

If checkpoint data is provided by the caller, the weights it contains will be loaded into the returned model.

Usage examples inside a session configuration file:

# ...
# the function will look for a 'model' field in the provided config dict
"model": {
    # the type provides the class name to instantiate an object from
    "type": "thelper.nn.mobilenet.MobileNetV2",
    # the parameters listed below are passed to the model's constructor
    "params": {
        # ...
    }
# ...
Parameters
  • config – a session dictionary that provides a ‘model’ field containing a dictionary.

  • task – a task object that will be passed to the model’s constructor in order to specialize it. Can be None if a checkpoint is provided, and if the previous task is wanted instead of a new one.

  • save_dir – if not None, a log file containing model information will be created there.

  • ckptdata – raw checkpoint data loaded via torch.load(); the model will be given its previous state.

Returns

The instantiated model, compatible with the interface of both thelper.nn.utils.Module and torch.nn.Module.