Schedules¶
Schedule¶
-
class
sconce.schedules.
Schedule
[source]¶ The base class for all schedules in Sconce. It is only an interface, describing what must be implemented if you want to define a schedule.
-
get_value
(step, current_state)[source]¶ Returns the value one should set, based on this schedule.
Parameters: - step (float) – (0.0, inf) the training step that is about to be completed.
Fractional steps are possible (see batch_multiplier option on
sconce.trainer.Trainer.train()
). - current_state (dict) – a dictionary describing the current training state.
- step (float) – (0.0, inf) the training step that is about to be completed.
Fractional steps are possible (see batch_multiplier option on
-
Cosine¶
Exponential¶
-
class
sconce.schedules.
Exponential
(initial_value, final_value, stop_factor=None, loss_key='training_loss')[source]¶ A Schedule where the value, adjusts exponentially from <initial_value> to <final_value>, over <num_steps>.
Parameters: - initial_value (float) – the initial value of the hyperparameter.
- final_value (float) – the final value of the hyperparameter.
- stop_factor (float, optional) – a StopTrainingError will be raised if <loss_key> rises to <stop_factor> above it’s observed minimum value so far.
- loss_key (string, optional) – the name of the quantity (in
current_state
) to observe for <stop_factor>.
Linear¶
Step¶
-
class
sconce.schedules.
Step
(initial_value, final_value, num_changes=1)[source]¶ A Schedule where the value starts at <initial_value> and changes <num_changes> times over the course of <num_steps> to a final value of <final_value>.
Parameters: Note
The parameter <num_steps> is set during training based on the size of the batch_size and number of samples in the training_data_generator, and the batch_multiplier value.
Triangle¶
-
class
sconce.schedules.
Triangle
(initial_value, peak_value, peak_fraction=0.5)[source]¶ A Schedule where the value begins with value <initial_value>, then changes linearly to <peak_value>, and then back to <initial_value> by the end of <num_steps>. Peak value will occur after <peak_steps>.
Parameters: