Supervised Autodiff Predictive Coding
Supervised predictive coding with implicit gradients using tensorflow’s autodifferentiation of the energy with respect to representations and learnable parameters.
- supervised_autodiff_pc.infer(model, data, ir=0.025, T=200, predictions_flow_upward=False, target_shape=None)
Implements the following logic:
Initialize representations do T times E = 0.5 * norm(r - model(r)) ^ 2 r -= ir * dE/dr return r
- Parameters
model (list of
tf_utils.Dense
ortf_utils.BiasedDense
) – description of a sequential network by a list of layers, can be generated e.g. usingtf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
ir (float, optional) – inference rate, defaults to 0.025
T (int, optional) – number of inference steps, defaults to 200
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False
target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None
- Returns
latent representations
- Return type
list of 3d tf.Tensor of float32
- supervised_autodiff_pc.learn(model, data, target, ir=0.1, lr=0.001, T=40, predictions_flow_upward=False)
Implements the following logic:
Initialize representations do T times E = 0.5 * norm(r - model(r)) ^ 2 r -= ir * dE/dr W -= lr * dE/dW
- Parameters
model (list of
tf_utils.Dense
ortf_utils.BiasedDense
) – description of a sequential network by a list of layers, can be generated e.g. usingtf_utils.mlp()
data (3d tf.Tensor of float32) – inuput data batch
target (3d tf.Tensor of float32) – output target batch
ir (float, optional) – inference rate, defaults to 0.1
lr (float, optional) – learning rate, defaults to 0.001
T (int, optional) – number of inference steps, defaults to 40
predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False