sconce.schedules package¶
sconce.schedules.base module¶
-
class
sconce.schedules.base.
Schedule
[source]¶ Bases:
abc.ABC
The base class for all schedules in Sconce. It is only an interface, describing what must be implemented if you want to define a schedule.
-
get_value
(step, current_state)[source]¶ Returns the value one should set, based on this schedule.
Parameters: - step (float) – (0.0, inf) the training step that is about to be completed.
Fractional steps are possible (see batch_multiplier option on
sconce.trainer.Trainer.train()
). - current_state (dict) – a dictionary describing the current training state.
- step (float) – (0.0, inf) the training step that is about to be completed.
Fractional steps are possible (see batch_multiplier option on
-
sconce.schedules.cosine module¶
-
class
sconce.schedules.cosine.
Cosine
(initial_value, final_value)[source]¶ Bases:
sconce.schedules.base.Schedule
A Schedule where the hyperparameter follows a scaled and shifted cosine function from [0, pi]. It will begin at <initial_value> and end at <final_value>, after <num_steps>.
Parameters:
sconce.schedules.exponential module¶
-
class
sconce.schedules.exponential.
Exponential
(initial_value, final_value, stop_factor=None, loss_key='training_loss')[source]¶ Bases:
sconce.schedules.base.Schedule
A Schedule where the value, adjusts exponentially from <initial_value> to <final_value>, over <num_steps>.
Parameters: - initial_value (float) – the initial value of the hyperparameter.
- final_value (float) – the final value of the hyperparameter.
- stop_factor (float, optional) – a StopTrainingError will be raised if <loss_key> rises to <stop_factor> above it’s observed minimum value so far.
- loss_key (string, optional) – the name of the quantity (in
current_state
) to observe for <stop_factor>.
sconce.schedules.linear module¶
-
class
sconce.schedules.linear.
Linear
(initial_value, final_value)[source]¶ Bases:
sconce.schedules.base.Schedule
A Schedule where the value begins with value <initial_value>, then changes linearly to <final_value>.
Parameters:
sconce.schedules.step module¶
-
class
sconce.schedules.step.
Step
(initial_value, final_value, num_changes=1)[source]¶ Bases:
sconce.schedules.base.Schedule
A Schedule where the value starts at <initial_value> and changes <num_changes> times over the course of <num_steps> to a final value of <final_value>.
Parameters: Note
The parameter <num_steps> is set during training based on the size of the batch_size and number of samples in the training_feed, and the batch_multiplier value.
sconce.schedules.triangle module¶
-
class
sconce.schedules.triangle.
Triangle
(initial_value, peak_value, peak_fraction=0.5)[source]¶ Bases:
sconce.schedules.base.Schedule
A Schedule where the value begins with value <initial_value>, then changes linearly to <peak_value>, and then back to <initial_value> by the end of <num_steps>. Peak value will occur after <peak_steps>.
Parameters: