precisions_utils module

precisions_utils.precision_modulated_energy(model, r, theta=[], predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)

Energy (total squared L2 norm of precision-weighted errors + regularizer) computation and autodifferentiation with respect to representations and learnable parameters

Parameters
  • model (list of tf_utils.Dense or tf_utils.BiasedDense) – description of a sequential network by a list of layers, can be generated e.g. using tf_utils.mlp()

  • r (list of 3d tf.Tensor of float32) – representations

  • theta (list of variable tf.Tensor of float32, optional) – learnable parameters, defaults to []

  • predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

  • gamma (float32, optional) – Regularization scalar, defaults to .5

Returns

energy, autodiff gradient tape

Return type

float32, tf.GradientTape

precisions_utils.precision_modulated_inference_step_backward_predictions(e, r, w, P, ir, f, df, update_last=True)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • ir (float32) – inference rate

  • f (function) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

precisions_utils.precision_modulated_inference_step_forward_predictions(e, r, w, P, ir, f, df, update_last=True)

Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • ir (float32) – inference rate

  • f (float32) – activation function

  • df (function) – derivate of the activation function

  • update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

precisions_utils.precision_modulated_weight_update_backward_predictions(w, e, r, P, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

  • f (function) – activation function

precisions_utils.precision_modulated_weight_update_forward_predictions(w, e, r, P, lr, f)

Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using tf_utils.mlp()

  • e (list of 3d tf.Tensor of float32) – prediction errors

  • r (list of 3d tf.Tensor of float32) – representations

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

  • f (function) – activation function

precisions_utils.precisions_update_backward_predictions(e, P, lr)

Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from higher layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate

precisions_utils.precisions_update_forward_predictions(e, P, lr, diagonal=False)

Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from lower layer)

Parameters
  • e (list of 3d tf.Tensor of float32) – prediction errors

  • P (list of 2d tf.Tensor of float32) – list of precision matrices

  • lr (float32) – learning rate