precision_modulated_supervised_explicit_pc module

Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.

precision_modulated_supervised_explicit_pc.infer(weights, precisions, data, ir=0.025, T=200, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (-P * e + tranpose(W) * P * e)
return r
Parameters
  • weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • precisions – list of precision matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • ir (float, optional) – inference rate, defaults to 0.025

  • T (int, optional) – number of inference steps, defaults to 200

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

Returns

latent representations

Return type

list of 3d tf.Tensor of float32

precision_modulated_supervised_explicit_pc.learn(weights, precisions, data, target, ir=0.1, lr=0.001, pr=0.001, T=20, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, diagonal=False)

Implements the following logic:

Initialize representations
do T times
    e = r - W * r
    r += ir * (- P * e + tranpose(W) * P * e)
W += lr * (P * e * tranpose(r))
P += pr * (I - P * e * transpose(e))
Parameters
  • weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • precisions – list of precision matrices, can be generated e.g. using tf_utils.mlp()

  • data (3d tf.Tensor of float32) – inuput data batch

  • target (3d tf.Tensor of float32) – output target batch

  • ir (float, optional) – inference rate, defaults to 0.1

  • lr (float, optional) – learning rate for weights, defaults to 0.001

  • pr (float, optional) – learning rate for precision, defaults to 0.001

  • T (int, optional) – number of inference steps, defaults to 20

  • f (function, optional) – activation function, defaults to tf.nn.relu

  • df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False