Python Code¶
Subpackages¶
Submodules¶
cis_interface.backwards module¶
This module allows for backward compatibility.
cis_interface.command_line module¶
cis_interface.config module¶
This module imports the configuration for cis_interface.
Todo
Remove reference to environment variables for accessing config options.
-
class
cis_interface.config.
CisConfigParser
(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False)[source]¶ Bases:
ConfigParser.ConfigParser
Config parser that returns None if option not provided on get.
-
cis_interface.config.
cfg_environment
(env=None, cfg=None)[source]¶ Set environment variables based on config options.
Parameters: - env (dict, optional) – Dictionary of environment variables that should be updated. Defaults to os.environ.
- cfg (
cis_interface.config.CisConfigParser
, optional) – Config parser with options that should be used to update the environment. Defaults tocis_interface.config.cis_cfg
.
-
cis_interface.config.
cfg_logging
(cfg=None)[source]¶ Set logging levels from config options.
Parameters: cfg ( cis_interface.config.CisConfigParser
, optional) – Config parser with options that should be used to update the environment. Defaults tocis_interface.config.cis_cfg
.
cis_interface.platform module¶
This module handle platform compatibility issues.
cis_interface.runner module¶
This module provides tools for running models using cis_interface.
-
class
cis_interface.runner.
CisRunner
(modelYmls, namespace, host=None, rank=0, cis_debug_level=None, rmq_debug_level=None, cis_debug_prefix=None)[source]¶ Bases:
cis_interface.tools.CisClass
This class handles the orchestration of starting the model and IO drivers, monitoring their progress, and cleaning up on exit.
Parameters: - modelYmls (list) – List of paths to yaml files specifying the models that should be run.
- namespace (str) – Name that should be used to uniquely identify any RMQ exchange.
- host (str, optional) – Name of the host that the models will be launched from. Defaults to None.
- rank (int, optional) – Rank of this set of models if run in parallel. Defaults to 0.
- cis_debug_level (str, optional) – Level for CiS debug messages. Defaults to environment variable ‘CIS_DEBUG’.
- rmq_debug_level (str, optional) – Level for RabbitMQ debug messages. Defaults to environment variable ‘RMQ_DEBUG’.
- cis_debug_prefix (str, optional) – Prefix for CiS debug messages. Defaults to namespace.
-
namespace
¶ str – Name that should be used to uniquely identify any RMQ exchange.
-
host
¶ str – Name of the host that the models will be launched from.
-
rank
¶ int – Rank of this set of models if run in parallel.
-
modeldrivers
¶ dict – Model drivers associated with this run.
-
inputdrivers
¶ dict – Input drivers associated with this run.
-
outputdrivers
¶ dict – Output drivers associated with this run.
-
serverdrivers
¶ dict – The addresses associated with different server drivers.
-
interrupt_time
¶ float – Time of last interrupt signal.
-
error_flag
¶ bool – True if one or more models raises an error.
..todo:: namespace, host, and rank do not seem strictly necessary.
-
all_drivers
¶ iterator – For all drivers.
-
closeChannels
(force_stop=False)[source]¶ Stop IO drivers and join the threads.
Parameters: force_stop (bool, optional) – If True, the terminate method is used to stop the drivers. Otherwise, the stop method is used. The stop method will try to exit gracefully while terminate will exit as quickly as possible. Defaults to False.
-
createDriver
(yml)[source]¶ Create a driver instance from the yaml information.
Parameters: yml (yaml) – Yaml object containing driver information. Returns: An instance of the specified driver. Return type: object
-
createInputDriver
(yml)[source]¶ Create an input driver instance from the yaml information.
Parameters: yml (yaml) – Yaml object containing driver information. Returns: An instance of the specified driver. Return type: object
-
createModelDriver
(yml)[source]¶ Create a model driver instance from the yaml information.
Parameters: yml (yaml) – Yaml object containing driver information. Returns: An instance of the specified driver. Return type: object
-
createOutputDriver
(yml)[source]¶ Create an output driver instance from the yaml information.
Parameters: yml (yaml) – Yaml object containing driver information. Returns: An instance of the specified driver. Return type: object
-
do_client_exits
(model)[source]¶ Perform exits for IO drivers associated with a client model.
Parameters: model (dict) – Dictionary of model parameters including any associated IO drivers.
-
do_model_exits
(model)[source]¶ Perform exits for IO drivers associated with a model.
Parameters: model (dict) – Dictionary of model parameters including any associated IO drivers.
-
io_drivers
(model=None)[source]¶ Return the input and output drivers for one or all models.
Parameters: model (str, optional) – Name of a model that I/O drivers should be returned for. Defaults to None and all I/O drivers are returned. Returns: Access to list of I/O drivers. Return type: iterator
-
loadDrivers
()[source]¶ Load all of the necessary drivers, doing the IO drivers first and adding IO driver environmental variables back tot he models.
-
cis_interface.runner.
get_runner
(models, **kwargs)[source]¶ Get runner for a set of models, getting run information from the environment.
Parameters: - models (list) – List of yaml files containing information on the models that should be run.
- **kwargs – Additonal keyword arguments are passed to CisRunner.
Returns: Runner for the provided models.
Return type: Raises: Exception
– If config option ‘namespace’ in ‘rmq’ section not set.
cis_interface.scanf module¶
Small scanf implementation.
Python has powerful regular expressions but sometimes they are totally overkill when you just want to parse a simple-formatted string. C programmers use the scanf-function for these tasks (see link below).
This implementation of scanf translates the simple scanf-format into regular expressions. Unlike C you can be sure that there are no buffer overflows possible.
- For more information see
- Original code from:
- http://code.activestate.com/recipes/502213-simple-scanf-implementation/
Modified original to make the %f more robust, as well as added %* modifier to skip fields.
Version: 1.3.3
- Releases:
- 1.0
- 2010-10-11
- Initial release
- 1.1
- 2010-10-13
- Changed regex from ‘match’ (only matches at beginning of line) to ‘search’ (matches anywhere in line)
- Bugfix - ignore cast for skipped fields
- 1.2
- 2013-05-30
- Added ‘collapseWhitespace’ flag (defaults to True) to take the search string and replace all whitespace with regex string to match repeated whitespace. This enables better matching in log files where the data has been formatted for easier reading. These cases have variable amounts of whitespace between the columns, depending on the number of characters in the data itself.
- 1.3
- 2016-01-18
- Add ‘extractdata’ function.
- 1.3.1
- 2016-06-23
- Release to PyPi, now including README.md
- Updates by Meagan Lang:
- 2018-04-12
- Added ability to parse components of field specifiers and cope with whitespace padding.
-
cis_interface.scanf.
scanf
(format, s=None, collapseWhitespace=True)[source]¶ - scanf supports the following formats:
- %c One character %5c 5 characters %d, %i int value %7d, %7i int value with length 7 %f float value %o octal value %X, %x hex value %s string terminated by whitespace
Examples: >>> scanf(“%s - %d errors, %d warnings”, “/usr/sbin/sendmail - 0 errors, 4 warnings”) (‘/usr/sbin/sendmail’, 0, 4) >>> scanf(“%o %x %d”, “0123 0x123 123”) (66, 291, 123)
If the parameter s is a file-like object, s.readline is called. If s is not specified, stdin is assumed.
The function returns a tuple of found values or None if the format does not match.
-
cis_interface.scanf.
scanf_compile
(format, collapseWhitespace=True)[source]¶ Translate the format into a regular expression
For example: >>> format_re, casts = _scanf_compile(‘%s - %d errors, %d warnings’) >>> print format_re.pattern (S+) - ([+-]?d+) errors, ([+-]?d+) warnings
Translated formats are cached for faster reuse
cis_interface.schema module¶
-
class
cis_interface.schema.
ComponentSchema
(schema_type, subtype_attr=None, **kwargs)[source]¶ Bases:
dict
Schema information for one component.
Parameters: -
append_rules
(new)[source]¶ Add rules from new class’s schema to this one.
Parameters: new (dict) – New schema to add.
-
class2subtype
¶ dict – Mapping from class to list of subtypes.
-
classes
¶ list – All available classes for this schema.
-
subtype2class
¶ dict – Mapping from subtype to class.
-
subtypes
¶ list – All subtypes for this schema type.
-
-
class
cis_interface.schema.
SchemaDumper
(stream, default_style=None, default_flow_style=None, canonical=None, indent=None, width=None, allow_unicode=None, line_break=None, encoding=None, explicit_start=None, explicit_end=None, version=None, tags=None)[source]¶ Bases:
yaml.dumper.Dumper
SafeDumper for schema that includes tuples and Schema classes.
-
yaml_representers
= {None: <unbound method SafeRepresenter.represent_undefined>, <class 'cis_interface.schema.SchemaRegistry'>: <unbound method SchemaDumper.represent_SchemaRegistry>, <class 'cis_interface.schema.ComponentSchema'>: <unbound method SchemaDumper.represent_ComponentSchema>, <type 'bool'>: <unbound method SafeRepresenter.represent_bool>, <type 'classobj'>: <unbound method Representer.represent_name>, <type 'complex'>: <unbound method Representer.represent_complex>, <type 'float'>: <unbound method SafeRepresenter.represent_float>, <type 'function'>: <unbound method Representer.represent_name>, <type 'int'>: <unbound method SafeRepresenter.represent_int>, <type 'list'>: <unbound method SafeRepresenter.represent_list>, <type 'long'>: <unbound method Representer.represent_long>, <type 'dict'>: <unbound method SafeRepresenter.represent_dict>, <type 'builtin_function_or_method'>: <unbound method Representer.represent_name>, <type 'module'>: <unbound method Representer.represent_module>, <type 'NoneType'>: <unbound method SafeRepresenter.represent_none>, <type 'set'>: <unbound method SafeRepresenter.represent_set>, <type 'str'>: <unbound method Representer.represent_str>, <type 'tuple'>: <unbound method SchemaDumper.represent_python_tuple>, <type 'type'>: <unbound method Representer.represent_name>, <type 'unicode'>: <unbound method Representer.represent_unicode>, <type 'datetime.datetime'>: <unbound method SafeRepresenter.represent_datetime>, <type 'datetime.date'>: <unbound method SafeRepresenter.represent_date>}¶
-
-
class
cis_interface.schema.
SchemaLoader
(stream)[source]¶ Bases:
yaml.loader.SafeLoader
SafeLoader for schema that includes tuples.
-
yaml_constructors
= {None: <unbound method SafeConstructor.construct_undefined>, u'tag:yaml.org,2002:binary': <unbound method SafeConstructor.construct_yaml_binary>, u'tag:yaml.org,2002:bool': <unbound method SafeConstructor.construct_yaml_bool>, u'tag:yaml.org,2002:float': <unbound method SafeConstructor.construct_yaml_float>, u'tag:yaml.org,2002:int': <unbound method SafeConstructor.construct_yaml_int>, u'tag:yaml.org,2002:map': <unbound method SafeConstructor.construct_yaml_map>, u'tag:yaml.org,2002:null': <unbound method SafeConstructor.construct_yaml_null>, u'tag:yaml.org,2002:omap': <unbound method SafeConstructor.construct_yaml_omap>, u'tag:yaml.org,2002:pairs': <unbound method SafeConstructor.construct_yaml_pairs>, 'tag:yaml.org,2002:python/tuple': <unbound method SchemaLoader.construct_python_tuple>, u'tag:yaml.org,2002:seq': <unbound method SafeConstructor.construct_yaml_seq>, u'tag:yaml.org,2002:set': <unbound method SafeConstructor.construct_yaml_set>, u'tag:yaml.org,2002:str': <unbound method SafeConstructor.construct_yaml_str>, u'tag:yaml.org,2002:timestamp': <unbound method SafeConstructor.construct_yaml_timestamp>}¶
-
-
class
cis_interface.schema.
SchemaRegistry
(registry=None, required=None)[source]¶ Bases:
dict
Registry of schema’s for different integration components.
Parameters: Raises: ValueError
– If registry is provided and one of the required components is missing.-
class2conntype
¶ dict – Mapping from connection class to comm classes & direction.
-
class2filetype
¶ dict – Mapping from communication class to filetype.
-
class2language
¶ dict – Mapping from ModelDriver class to programming language.
-
conntype2class
¶ dict – Mapping from comm classes & direction to connection class.
-
filetype2class
¶ dict – Mapping from filetype to communication class.
-
classmethod
from_file
(fname)[source]¶ Create a SchemaRegistry from a file.
Parameters: fname (str) – Full path to the file the schema should be loaded from.
-
language2class
¶ dict – Mapping from programming language to ModelDriver class.
-
load
(fname)[source]¶ Load schema from a file.
Parameters: fname (str) – Full path to the file the schema should be loaded from.
-
validator
¶ Compose complete schema for parsing yaml.
-
-
class
cis_interface.schema.
SchemaValidator
(*args, **kwargs)[source]¶ Bases:
cerberus.validator.Validator
Class for validating the schema.
-
cis_type_order
= ['list', 'string', 'integer', 'boolean', 'function']¶
-
coercers
= ('boolean', 'function', 'integer', 'list', 'string')¶
-
default_setters
= ()¶
-
normalization_rules
= {'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'purge_unknown': {'type': 'boolean'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}¶
-
rules
= {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'purge_unknown': {'type': 'boolean'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}¶
-
types_mapping
= {'binary': TypeDefinition(name='binary', included_types=(<type 'str'>, <type 'bytearray'>), excluded_types=()), 'boolean': TypeDefinition(name='boolean', included_types=(<type 'bool'>,), excluded_types=()), 'date': TypeDefinition(name='date', included_types=(<type 'datetime.date'>,), excluded_types=()), 'datetime': TypeDefinition(name='datetime', included_types=(<type 'datetime.datetime'>,), excluded_types=()), 'dict': TypeDefinition(name='dict', included_types=(<class '_abcoll.Mapping'>,), excluded_types=()), 'float': TypeDefinition(name='float', included_types=(<type 'float'>, (<type 'int'>, <type 'long'>)), excluded_types=()), 'function': TypeDefinition(name='function', included_types=<type 'function'>, excluded_types=()), 'integer': TypeDefinition(name='integer', included_types=((<type 'int'>, <type 'long'>),), excluded_types=()), 'list': TypeDefinition(name='list', included_types=(<class '_abcoll.Sequence'>,), excluded_types=(<type 'basestring'>,)), 'number': TypeDefinition(name='number', included_types=((<type 'int'>, <type 'long'>), <type 'float'>), excluded_types=(<type 'bool'>,)), 'set': TypeDefinition(name='set', included_types=(<type 'set'>,), excluded_types=()), 'string': TypeDefinition(name='string', included_types=<type 'basestring'>, excluded_types=())}¶
-
validation_rules
= {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}¶
-
validators
= ()¶
-
-
cis_interface.schema.
get_schema
(fname=None)[source]¶ Return the cis_interface schema for YAML options.
Parameters: fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname. Returns: cis_interface YAML options. Return type: dict
-
cis_interface.schema.
inherit_schema
(orig, key, value, **kwargs)[source]¶ Create an inherited schema, adding new value to accepted ones for dependencies.
Parameters: Returns: New schema.
Return type:
-
cis_interface.schema.
load_schema
(fname=None)[source]¶ Return the cis_interface schema for YAML options.
Parameters: fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname. Returns: cis_interface YAML options. Return type: dict
-
cis_interface.schema.
register_component
(component_class)[source]¶ Decorator for registering a class as a yaml component.
-
cis_interface.schema.
str_to_function
(value)[source]¶ Convert a string to a function.
Parameters: value (str, list) – String or list of strings, specifying function(s). The format should be “<package.module>:<function>” so that <function> can be imported from <package>. Returns: Callable function. Return type: func
cis_interface.tools module¶
This modules offers various tools.
-
class
cis_interface.tools.
CisClass
(name, uuid=None, working_dir=None, timeout=60.0, sleeptime=0.01, **kwargs)[source]¶ Bases:
logging.LoggerAdapter
Base class for CiS classes.
Parameters: - name (str) – Class name.
- uuid (str, optional) – Unique ID for this instance. Defaults to None and is assigned.
- working_dir (str, optional) – Working directory. If not provided, the current working directory is used.
- timeout (float, optional) – Maximum time (in seconds) that should be spent waiting on a process. Defaults to 60.
- sleeptime (float, optional) – Time that class should sleep for when sleep is called. Defaults to 0.01.
- **kwargs – Additional keyword arguments are assigned to the extra_kwargs dictionary.
-
name
¶ str – Class name.
-
uuid
¶ str – Unique ID for this instance.
-
sleeptime
¶ float – Time that class should sleep for when sleep called.
-
longsleep
¶ float – Time that the class will sleep for when waiting for longer tasks to complete (10x longer than sleeptime).
-
timeout
¶ float – Maximum time that should be spent waiting on a process.
-
working_dir
¶ str – Working directory.
-
errors
¶ list – List of errors.
-
extra_kwargs
¶ dict – Keyword arguments that were not parsed.
-
sched_out
¶ obj – Output from the last scheduled task with output.
-
logger
¶ logging.Logger – Logger object for this object.
-
suppress_special_debug
¶ bool – If True, special_debug log messages are suppressed.
-
as_str
(obj)[source]¶ Return str version of object if it is not already a string.
Parameters: obj (object) – Object that should be turned into a string. Returns: String version of provided object. Return type: str
-
check_timeout
(key=None, key_level=0)[source]¶ Check timeout for the calling function/method.
Parameters: - key (str, optional) – Key for timeout that should be checked. Defaults to None and is set by the calling class and function/method (See timeout_key).
- key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
Raises: KeyError
– If there is not a timeout registered for the specified key.
-
error
¶ Log an error level message.
-
exception
¶ Log an exception level message.
-
get_timeout_key
(key_level=0, key_suffix=None)[source]¶ Return a key for a given level in the stack, relative to the function calling get_timeout_key.
Parameters: - key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that is 2 steps higher in the stack. Higher values use classes and function/methods further up in the stack. Defaults to 0.
- key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
Returns: Key identifying calling object and method.
Return type:
-
name
Name of the class object.
-
print_encoded
(msg, *args, **kwargs)[source]¶ Print bytes to stdout, encoding if possible.
Parameters:
-
raise_error
(e)[source]¶ Raise an exception, logging it first.
Parameters: e (Exception) – Exception to raise. Raises: The provided exception.
-
sched_task
(t, func, args=[], kwargs={}, store_output=False)[source]¶ Schedule a task that will be executed after a certain time has elapsed.
Parameters: - t (float) – Number of seconds that should be waited before task is executed.
- func (object) – Function that should be executed.
- args (list, optional) – Arguments for the provided function. Defaults to [].
- kwargs (dict, optional) – Keyword arguments for the provided function. Defaults to {}.
- store_output (bool, optional) – If True, the output from the scheduled task is stored in self.sched_out. Otherwise, it is not stored. Defaults to False.
-
sleep
(t=None)[source]¶ Have the class sleep for some period of time.
Parameters: t (float, optional) – Time that class should sleep for. If not provided, the attribute ‘sleeptime’ is used.
-
special_debug
¶ Log debug level message contingent of supression flag.
-
start_timeout
(t=None, key=None, key_level=0, key_suffix=None)[source]¶ Start a timeout for the calling function/method.
Parameters: - t (float, optional) – Maximum time that the calling function should wait before timeing out. If not provided, the attribute ‘timeout’ is used.
- key (str, optional) – Key that should be associated with the timeout that is created. Defaults to None and is set by the calling class and function/method (See get_timeout_key).
- key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
- key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
Raises: KeyError
– If the key already exists.
-
stop_timeout
(key=None, key_level=0, key_suffix=None, quiet=False)[source]¶ Stop a timeout for the calling function method.
Parameters: - key (str, optional) – Key for timeout that should be stopped. Defaults to None and is set by the calling class and function/method (See timeout_key).
- key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
- key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
- quiet (bool, optional) – If True, error message on timeout exceeded will be debug log. Defaults to False.
Raises: KeyError
– If there is not a timeout registered for the specified key.
-
timeout_key
¶ str – Key identifying calling object and method.
-
class
cis_interface.tools.
CisPopen
(cmd_args, forward_signals=True, **kwargs)[source]¶ Bases:
subprocess.Popen
Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe. In addition this class overrides Popen.kill() to allow processes to be killed with CTRL_BREAK_EVENT on windows.
Parameters: - args (list, str) – Shell command or list of arguments that should be run.
- forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True.
- **kwargs – Additional keywords arguments are passed to Popen.
-
class
cis_interface.tools.
CisThread
(name=None, target=None, args=(), kwargs=None, daemon=False, group=None, **cis_kwargs)[source]¶ Bases:
threading.Thread
,cis_interface.tools.CisClass
Thread for CiS that tracks when the thread is started and joined.
-
lock
¶ threading.RLock – Lock for accessing the sockets from multiple threads.
-
start_event
¶ threading.Event – Event indicating that the thread was started.
-
terminate_event
¶ threading.Event – Event indicating that the thread should be terminated. The target must exit when this is set.
-
main_terminated
¶ bool – True if the main thread has terminated.
-
terminate
(no_wait=False)[source]¶ Set the terminate event and wait for the thread to stop.
Parameters: no_wait (bool, optional) – If True, terminate will not block until the thread stops. Defaults to False and blocks. Raises: AssertionError
– If no_wait is False and the thread has not stopped after the timeout.
-
wait
(timeout=None, key=None)[source]¶ Wait until thread finish to return using sleeps rather than blocking.
Parameters:
-
was_started
¶ bool – True if the thread was started. False otherwise.
-
was_terminated
¶ bool – True if the thread was terminated. False otherwise.
-
-
class
cis_interface.tools.
CisThreadLoop
(*args, **kwargs)[source]¶ Bases:
cis_interface.tools.CisThread
Thread that will run a loop until the terminate event is called.
-
on_main_terminated
(dont_break=False)[source]¶ Actions performed when 1st main terminated.
Parameters: dont_break (bool, optional) – If True, the break flag won’t be set. Defaults to False.
-
wait_for_loop
(timeout=None, key=None)[source]¶ Wait until thread enters loop to return using sleeps rather than blocking.
Parameters:
-
was_break
¶ bool – True if the break flag was set.
-
was_loop
¶ bool – True if the thread was loop. False otherwise.
-
-
class
cis_interface.tools.
TimeOut
(max_time, key=None)[source]¶ Bases:
object
Class for checking if a period of time has been elapsed.
Parameters: -
max_time
¶ float – Maximum period of time that should elapsed before ‘is_out’ returns True.
-
start_time
¶ float – Result of time.time() at start.
-
key
¶ str – Key that was used to register the timeout.
-
elapsed
¶ float – Total time that has elapsed since the start.
-
is_out
¶ bool – True if there is not any time remaining. False otherwise.
-
-
cis_interface.tools.
eval_kwarg
(x)[source]¶ If x is a string, eval it. Otherwise just return it.
Parameters: x (str, obj) – String to be evaluated as an object or an object. Returns: Result of evaluated string or the input object. Return type: obj
-
cis_interface.tools.
get_default_comm
()[source]¶ Get the default comm that should be used for message passing.
-
cis_interface.tools.
is_ipc_installed
()[source]¶ Determine if the IPC libraries are installed.
Returns: True if the IPC libraries are installed, False otherwise. Return type: bool
-
cis_interface.tools.
is_zmq_installed
(check_c=True)[source]¶ Determine if the libczmq & libzmq libraries are installed.
Returns: True if both libraries are installed, False otherwise. Return type: bool
-
cis_interface.tools.
locate_path
(fname, basedir='/')[source]¶ Find the full path to a file using where on Windows.
-
cis_interface.tools.
popen_nobuffer
(*args, **kwargs)[source]¶ Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe.
Parameters: - args (list, str) – Shell command or list of arguments that should be run.
- forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True.
- **kwargs – Additional keywords arguments are passed to Popen.
Returns: Process that was started.
Return type:
cis_interface.units module¶
cis_interface.yamlfile module¶
-
cis_interface.yamlfile.
bridge_io_drivers
(yml, iodict)[source]¶ Create connection from input/output driver.
Parameters: - dict – YAML entry for model output that should be modified.
- iodict (dict) – Log of all input/outputs.
-
cis_interface.yamlfile.
cdriver2filetype
(driver)[source]¶ Convert a connection driver to a file type.
Parameters: driver (str) – The name of the connection driver. Returns: The corresponding file type for the driver. Return type: str
-
cis_interface.yamlfile.
link_model_io
(existing)[source]¶ Link I/O drivers back to the models they communicate with.
Parameters: existing (dict) – Dictionary of existing components. Returns: Dictionary with I/O drivers added to models. Return type: dict
-
cis_interface.yamlfile.
load_yaml
(fname)[source]¶ Parse a yaml file defining a run.
Parameters: fname (str) – Path to the yaml file. Returns: Contents of yaml file. Return type: dict
-
cis_interface.yamlfile.
parse_component
(yml, ctype, existing=None)[source]¶ Parse a yaml entry for a component, adding it to the list of existing components.
Parameters: Raises: TypeError
– If yml is not a dictionary.ValueError
– If dtype is not ‘input’, ‘output’, ‘model’, or ‘connection’.ValueError
– If the component already exists.
Returns: All components identified.
Return type:
-
cis_interface.yamlfile.
parse_connection
(yml, existing)[source]¶ Parse a yaml entry for a connection between I/O channels.
Parameters: Raises: RuntimeError
– If the ‘input’ entry is not a model output or file.RuntimeError
– If neither the ‘input’ or ‘output’ entries correspond to model I/O channels.
Returns: Updated log of all entries.
Return type:
-
cis_interface.yamlfile.
parse_model
(yml, existing)[source]¶ Parse a yaml entry for a model.
Parameters: Returns: Updated log of all entries.
Return type:
-
cis_interface.yamlfile.
parse_yaml
(files)[source]¶ Parse list of yaml files.
Parameters: files (str, list) – Either the path to a single yaml file or a list of yaml files.
Raises: ValueError
– If the yml dictionary is missing a required keyword or has an invalid value.RuntimeError
– If one of the I/O channels is not initialized with driver information.
Returns: Dictionary of information parsed from the yamls.
Return type:
-
cis_interface.yamlfile.
prep_connection
(yml, iodict)[source]¶ Prepare yaml connection entry for parsing with schema.
Parameters:
-
cis_interface.yamlfile.
prep_model
(yml, iodict)[source]¶ Prepare yaml model entry for parsing with schema.
Parameters:
-
cis_interface.yamlfile.
prep_yaml
(files)[source]¶ Prepare yaml to be parsed by Cerberus using schema including covering backwards compatible options.
Parameters: files (str, list) – Either the path to a single yaml file or a list of yaml files. Returns: YAML ready to be parsed using schema. Return type: dict
Module contents¶
This package provides a framework for integrating models across languages such that they can be run simultaneously, passing input back and forth.