fireworks.queue package¶
Submodules¶
fireworks.queue.queue_adapter module¶
-
class
fireworks.queue.queue_adapter.
Command
(command)¶ Bases:
object
Helper class - run subprocess commands in a different thread with TIMEOUT option. From https://gist.github.com/kirpit/1306188 Based on jcollado’s solution: http://stackoverflow.com/questions/1191374/subprocess-with-timeout/4825933#4825933
-
__init__
(command)¶ initialize the object.
Parameters: command – command to run
-
command
= None¶
-
error
= ''¶
-
output
= ''¶
-
process
= None¶
-
run
(timeout=None, **kwargs)¶ Run the command.
Parameters: Returns: (status, output, error)
-
status
= None¶
-
-
class
fireworks.queue.queue_adapter.
QScriptTemplate
(template)¶ Bases:
string.Template
-
delimiter
= '$$'¶
-
pattern
= re.compile('\n \\$\\$(?:\n (?P<escaped>\\$\\$) | # Escape sequence of two delimiters\n (?P<named>(?-i:[_a-zA-Z][_a-zA-Z0-9]*)) | # delimiter and a Python identifier\n {(?P<braced>(?-i:, re.IGNORECASE|re.VERBOSE)¶
-
-
class
fireworks.queue.queue_adapter.
QueueAdapterBase
¶ Bases:
collections.defaultdict
,fireworks.utilities.fw_serializers.FWSerializable
The QueueAdapter is responsible for all interactions with a specific queue management system. This includes handling all details of queue script format as well as queue submission and
management.A user should extend this class with implementations that work on specific queue systems. Examples and implementations are in: fireworks/user_objects/queue_adapters.
Documentation on implementing queue adapters can be found on FireWorks home page, https://materialsproject.github.io/fireworks
-
defaults
= {}¶
-
classmethod
from_dict
(m_dict)¶
-
get_njobs_in_queue
(username=None)¶ Returns the number of jobs currently in the queue for the user.
Parameters: username (str) – the username of the jobs to count (default is to autodetect) Returns: (int) number of jobs in the queue
-
get_qlogger
(name)¶
-
get_script_str
(launch_dir)¶ returns a (multi-line) String representing the queue script, e.g. PBS script. Uses the template_file along with internal parameters to create the script.
Parameters: launch_dir (str) – The directory the job will be launched in Returns: (str) the queue script
-
q_name
= 'OVERRIDE_ME'¶
-
submit_cmd
= 'OVERRIDE_ME'¶
-
submit_to_queue
(script_file)¶ Submits the job to the queue and returns the job id.
Parameters: script_file – (str) name of the script file to use (String) Returns: (int) job_id
-
template_file
= 'OVERRIDE_ME'¶
-
to_dict
(*args, **kwargs)¶
-
fireworks.queue.queue_launcher module¶
-
fireworks.queue.queue_launcher.
launch_rocket_to_queue
(launchpad, fworker, qadapter, launcher_dir='.', reserve=False, strm_lvl='INFO', create_launcher_dir=False, fill_mode=False, fw_id=None)¶ Submit a single job to the queue.
Parameters: - launchpad (LaunchPad) –
- fworker (FWorker) –
- qadapter (QueueAdapterBase) –
- launcher_dir (str) – The directory where to submit the job
- reserve (bool) – Whether to queue in reservation mode
- strm_lvl (str) – level at which to stream log messages
- create_launcher_dir (bool) – Whether to create a subfolder launcher+timestamp, if needed
- fill_mode (bool) – whether to submit jobs even when there is nothing to run (only in non-reservation mode)
- fw_id (int) – specific fw_id to reserve (reservation mode only)
-
fireworks.queue.queue_launcher.
rapidfire
(launchpad, fworker, qadapter, launch_dir='.', nlaunches=0, njobs_queue=0, njobs_block=500, sleep_time=None, reserve=False, strm_lvl='INFO', timeout=None, fill_mode=False)¶ Submit many jobs to the queue.
Parameters: - launchpad (LaunchPad) –
- fworker (FWorker) –
- qadapter (QueueAdapterBase) –
- launch_dir (str) – directory where we want to write the blocks
- nlaunches (int) – total number of launches desired; “infinite” for loop, 0 for one round
- njobs_queue (int) – stops submitting jobs when njobs_queue jobs are in the queue, 0 for no limit
- njobs_block (int) – automatically write a new block when njobs_block jobs are in a single block
- sleep_time (int) – secs to sleep between rapidfire loop iterations
- reserve (bool) – Whether to queue in reservation mode
- strm_lvl (str) – level at which to stream log messages
- timeout (int) – # of seconds after which to stop the rapidfire process
- fill_mode (bool) – whether to submit jobs even when there is nothing to run (only in non-reservation mode)
-
fireworks.queue.queue_launcher.
setup_offline_job
(launchpad, fw, launch_id)¶