sconce.schedules package

sconce.schedules.base module

class sconce.schedules.base.Schedule[source]

Bases: abc.ABC

The base class for all schedules in Sconce. It is only an interface, describing what must be implemented if you want to define a schedule.

get_value(step, current_state)[source]

Returns the value one should set, based on this schedule.

Parameters:
  • step (float) – (0.0, inf) the training step that is about to be completed. Fractional steps are possible (see batch_multiplier option on sconce.trainer.Trainer.train()).
  • current_state (dict) – a dictionary describing the current training state.
set_num_steps(value)[source]
This is the canonical method to set num_steps, taking care of setting any other state needed
by the schedule as well.
class sconce.schedules.base.ScheduledMixin[source]

Bases: object

This mixin defines the interface for scheduled objects in Sconce.

prepare_for_step(step, current_state)[source]
remove_schedule(name)[source]
set_schedule(name, schedule)[source]
start_session(num_steps)[source]

sconce.schedules.cosine module

class sconce.schedules.cosine.Cosine(initial_value, final_value)[source]

Bases: sconce.schedules.base.Schedule

A Schedule where the hyperparameter follows a scaled and shifted cosine function from [0, pi]. It will begin at <initial_value> and end at <final_value>, after <num_steps>.

Parameters:
  • initial_value (float) – the initial value of the hyperparameter.
  • final_value (float) – the final value of the hyperparameter.
set_num_steps(num_steps)[source]

sconce.schedules.exponential module

class sconce.schedules.exponential.Exponential(initial_value, final_value, stop_factor=None, loss_key='training_loss')[source]

Bases: sconce.schedules.base.Schedule

A Schedule where the value, adjusts exponentially from <initial_value> to <final_value>, over <num_steps>.

Parameters:
  • initial_value (float) – the initial value of the hyperparameter.
  • final_value (float) – the final value of the hyperparameter.
  • stop_factor (float, optional) – a StopTrainingError will be raised if <loss_key> rises to <stop_factor> above it’s observed minimum value so far.
  • loss_key (string, optional) – the name of the quantity (in current_state) to observe for <stop_factor>.
set_num_steps(num_steps)[source]
should_continue(current_state)[source]

sconce.schedules.linear module

class sconce.schedules.linear.Linear(initial_value, final_value)[source]

Bases: sconce.schedules.base.Schedule

A Schedule where the value begins with value <initial_value>, then changes linearly to <final_value>.

Parameters:
  • initial_value (float) – the initial value of the hyperparameter.
  • final_value (float) – the final value of the hyperparameter.
set_num_steps(num_steps)[source]

sconce.schedules.step module

class sconce.schedules.step.Step(initial_value, final_value, num_changes=1)[source]

Bases: sconce.schedules.base.Schedule

A Schedule where the value starts at <initial_value> and changes <num_changes> times over the course of <num_steps> to a final value of <final_value>.

Parameters:
  • initial_value (float) – the initial value of the hyperparameter.
  • final_value (float) – the final value of the hyperparameter.
  • num_changes (int, optional) – [1, <num_steps>-1] the number of times the hyperparameter’s value will change.

Note

The parameter <num_steps> is set during training based on the size of the batch_size and number of samples in the training_feed, and the batch_multiplier value.

set_num_steps(num_steps)[source]

sconce.schedules.triangle module

class sconce.schedules.triangle.Triangle(initial_value, peak_value, peak_fraction=0.5)[source]

Bases: sconce.schedules.base.Schedule

A Schedule where the value begins with value <initial_value>, then changes linearly to <peak_value>, and then back to <initial_value> by the end of <num_steps>. Peak value will occur after <peak_steps>.

Parameters:
  • initial_value (float) – the initial value of the hyperparameter.
  • peak_value (float) – the value of the hyperparameter at it’s peak (maximum or minimum).
  • peak_fraction (float, optional) – (0.0, 1.0) used to determine the number of steps before the hyperparameter’s value will become <peak_value>.
set_num_steps(num_steps)[source]