Python Code

Submodules

cis_interface.backwards module

This module allows for backward compatibility.

cis_interface.backwards.bytes2unicode(b)[source]

Convert from bytes/unicode/str to unicode.

Parameters:b (bytes, unicode, str) – Bytes to be converted into unicode.
Returns:String version of input.
Return type:unicode/str
Raises:TypeError – If supplied type cannot be converted to unicode.
cis_interface.backwards.unicode2bytes(s)[source]

Convert from bytes/unicode/str to a bytes object.

Parameters:s (str, bytes, unicode) – Object to convert to a bytes version.
Returns:Bytes version of input.
Return type:bytes
Raises:TypeError – If supplied type cannot be converted to bytes.

cis_interface.command_line module

cis_interface.command_line.cc_flags()[source]

Get the compiler flags necessary for including the interface library in a C or C++ program.

Returns:The necessary compiler flags and preprocessor definitions.
Return type:list
cis_interface.command_line.ciscc()[source]

Compile C/C++ program.

cis_interface.command_line.cisrun()[source]

Start a run.

cis_interface.command_line.ld_flags()[source]

Get the linker flags necessary for calling functions/classes from the interface library in a C or C++ program.

Returns:The necessary library linking flags.
Return type:list

cis_interface.config module

This module imports the configuration for cis_interface.

Todo

Remove reference to environment variables for accessing config options.

class cis_interface.config.CisConfigParser(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False)[source]

Bases: ConfigParser.ConfigParser

Config parser that returns None if option not provided on get.

get(section, option, default=None, **kwargs)[source]

Return None if the section/option does not exist.

Parameters:
  • section (str) – Name of section.
  • option (str) – Name of option in section.
  • default (obj, optional) – Value that should be returned if the section and/or option are not found or are an empty string. Defaults to None.
  • **kwargs – Additional keyword arguments are passed to the parent class’s get.
Returns:

String entry if the section & option exist, otherwise default.

Return type:

obj

cis_interface.config.cfg_environment(env=None, cfg=None)[source]

Set environment variables based on config options.

Parameters:
  • env (dict, optional) – Dictionary of environment variables that should be updated. Defaults to os.environ.
  • cfg (cis_interface.config.CisConfigParser, optional) – Config parser with options that should be used to update the environment. Defaults to cis_interface.config.cis_cfg.
cis_interface.config.cfg_logging(cfg=None)[source]

Set logging levels from config options.

Parameters:cfg (cis_interface.config.CisConfigParser, optional) – Config parser with options that should be used to update the environment. Defaults to cis_interface.config.cis_cfg.

cis_interface.platform module

This module handle platform compatibility issues.

cis_interface.runner module

This module provides tools for running models using cis_interface.

class cis_interface.runner.CisRunner(modelYmls, namespace, host=None, rank=0, cis_debug_level=None, rmq_debug_level=None, cis_debug_prefix=None)[source]

Bases: cis_interface.tools.CisClass

This class handles the orchestration of starting the model and IO drivers, monitoring their progress, and cleaning up on exit.

Parameters:
  • modelYmls (list) – List of paths to yaml files specifying the models that should be run.
  • namespace (str) – Name that should be used to uniquely identify any RMQ exchange.
  • host (str, optional) – Name of the host that the models will be launched from. Defaults to None.
  • rank (int, optional) – Rank of this set of models if run in parallel. Defaults to 0.
  • cis_debug_level (str, optional) – Level for CiS debug messages. Defaults to environment variable ‘CIS_DEBUG’.
  • rmq_debug_level (str, optional) – Level for RabbitMQ debug messages. Defaults to environment variable ‘RMQ_DEBUG’.
  • cis_debug_prefix (str, optional) – Prefix for CiS debug messages. Defaults to namespace.
namespace

str – Name that should be used to uniquely identify any RMQ exchange.

host

str – Name of the host that the models will be launched from.

rank

int – Rank of this set of models if run in parallel.

modeldrivers

dict – Model drivers associated with this run.

inputdrivers

dict – Input drivers associated with this run.

outputdrivers

dict – Output drivers associated with this run.

serverdrivers

dict – The addresses associated with different server drivers.

interrupt_time

float – Time of last interrupt signal.

error_flag

bool – True if one or more models raises an error.

..todo:: namespace, host, and rank do not seem strictly necessary.

all_drivers

iterator – For all drivers.

cleanup()[source]

Perform cleanup operations for all drivers.

closeChannels(force_stop=False)[source]

Stop IO drivers and join the threads.

Parameters:force_stop (bool, optional) – If True, the terminate method is used to stop the drivers. Otherwise, the stop method is used. The stop method will try to exit gracefully while terminate will exit as quickly as possible. Defaults to False.
createDriver(yml)[source]

Create a driver instance from the yaml information.

Parameters:yml (yaml) – Yaml object containing driver information.
Returns:An instance of the specified driver.
Return type:object
createInputDriver(yml)[source]

Create an input driver instance from the yaml information.

Parameters:yml (yaml) – Yaml object containing driver information.
Returns:An instance of the specified driver.
Return type:object
createModelDriver(yml)[source]

Create a model driver instance from the yaml information.

Parameters:yml (yaml) – Yaml object containing driver information.
Returns:An instance of the specified driver.
Return type:object
createOutputDriver(yml)[source]

Create an output driver instance from the yaml information.

Parameters:yml (yaml) – Yaml object containing driver information.
Returns:An instance of the specified driver.
Return type:object
do_client_exits(model)[source]

Perform exits for IO drivers associated with a client model.

Parameters:model (dict) – Dictionary of model parameters including any associated IO drivers.
do_model_exits(model)[source]

Perform exits for IO drivers associated with a model.

Parameters:model (dict) – Dictionary of model parameters including any associated IO drivers.
io_drivers(model=None)[source]

Return the input and output drivers for one or all models.

Parameters:model (str, optional) – Name of a model that I/O drivers should be returned for. Defaults to None and all I/O drivers are returned.
Returns:Access to list of I/O drivers.
Return type:iterator
loadDrivers()[source]

Load all of the necessary drivers, doing the IO drivers first and adding IO driver environmental variables back tot he models.

pprint(*args)[source]

Print with color.

printStatus()[source]

Print the status of all drivers, starting with the IO drivers.

reset_signal_handler()[source]

Reset signal handlers to old ones.

run(signal_handler=None)[source]

Run all of the models and wait for them to exit.

set_signal_handler(signal_handler=None)[source]

Set the signal handler.

Parameters:signal_handler (function, optional) – Function that should handle received SIGINT and SIGTERM signals. Defaults to self.signal_handler.
signal_handler(sig, frame)[source]

Terminate all drivers on interrrupt.

startDrivers()[source]

Start drivers, starting with the IO drivers.

terminate()[source]

Immediately stop all drivers, beginning with IO drivers.

waitModels()[source]

Wait for all model drivers to finish. When a model finishes, join the thread and perform exits for associated IO drivers.

cis_interface.runner.get_runner(models, **kwargs)[source]

Get runner for a set of models, getting run information from the environment.

Parameters:
  • models (list) – List of yaml files containing information on the models that should be run.
  • **kwargs – Additonal keyword arguments are passed to CisRunner.
Returns:

Runner for the provided models.

Return type:

CisRunner

Raises:

Exception – If config option ‘namespace’ in ‘rmq’ section not set.

cis_interface.scanf module

Small scanf implementation.

Python has powerful regular expressions but sometimes they are totally overkill when you just want to parse a simple-formatted string. C programmers use the scanf-function for these tasks (see link below).

This implementation of scanf translates the simple scanf-format into regular expressions. Unlike C you can be sure that there are no buffer overflows possible.

For more information see
Original code from:
http://code.activestate.com/recipes/502213-simple-scanf-implementation/

Modified original to make the %f more robust, as well as added %* modifier to skip fields.

Version: 1.3.3

Releases:
1.0
2010-10-11
  • Initial release
1.1
2010-10-13
  • Changed regex from ‘match’ (only matches at beginning of line) to ‘search’ (matches anywhere in line)
  • Bugfix - ignore cast for skipped fields
1.2
2013-05-30
  • Added ‘collapseWhitespace’ flag (defaults to True) to take the search string and replace all whitespace with regex string to match repeated whitespace. This enables better matching in log files where the data has been formatted for easier reading. These cases have variable amounts of whitespace between the columns, depending on the number of characters in the data itself.
1.3
2016-01-18
  • Add ‘extractdata’ function.
1.3.1
2016-06-23
  • Release to PyPi, now including README.md
Updates by Meagan Lang:
2018-04-12
  • Added ability to parse components of field specifiers and cope with whitespace padding.
cis_interface.scanf.scanf(format, s=None, collapseWhitespace=True)[source]
scanf supports the following formats:
%c One character %5c 5 characters %d, %i int value %7d, %7i int value with length 7 %f float value %o octal value %X, %x hex value %s string terminated by whitespace

Examples: >>> scanf(“%s - %d errors, %d warnings”, “/usr/sbin/sendmail - 0 errors, 4 warnings”) (‘/usr/sbin/sendmail’, 0, 4) >>> scanf(“%o %x %d”, “0123 0x123 123”) (66, 291, 123)

If the parameter s is a file-like object, s.readline is called. If s is not specified, stdin is assumed.

The function returns a tuple of found values or None if the format does not match.

cis_interface.scanf.scanf_compile(format, collapseWhitespace=True)[source]

Translate the format into a regular expression

For example: >>> format_re, casts = _scanf_compile(‘%s - %d errors, %d warnings’) >>> print format_re.pattern (S+) - ([+-]?d+) errors, ([+-]?d+) warnings

Translated formats are cached for faster reuse

cis_interface.schema module

class cis_interface.schema.ComponentSchema(schema_type, subtype_attr=None, **kwargs)[source]

Bases: dict

Schema information for one component.

Parameters:
  • schema_type (str) – The name of the component.
  • subtype_attr (str, optional) – The attribute that should be used to log subtypes. Defaults to None.
  • **kwargs – Additional keyword arguments are entries in the component schema.
append(comp_cls, subtype=None)[source]

Append component class to the schema.

Parameters:
  • comp_cls (class) – Component class that should be added.
  • subtype (str, tuple, optional) – Key used to identify the subtype of the component type. Defaults to subtype_attr if one was provided, otherwise the subtype will not be logged.
append_rules(new)[source]

Add rules from new class’s schema to this one.

Parameters:new (dict) – New schema to add.
class2subtype

dict – Mapping from class to list of subtypes.

classes

list – All available classes for this schema.

subtype2class

dict – Mapping from subtype to class.

subtypes

list – All subtypes for this schema type.

class cis_interface.schema.SchemaDumper(stream, default_style=None, default_flow_style=None, canonical=None, indent=None, width=None, allow_unicode=None, line_break=None, encoding=None, explicit_start=None, explicit_end=None, version=None, tags=None)[source]

Bases: yaml.dumper.Dumper

SafeDumper for schema that includes tuples and Schema classes.

represent_ComponentSchema(data)[source]
represent_SchemaRegistry(data)[source]
represent_python_tuple(data, **kwargs)[source]
yaml_representers = {None: <unbound method SafeRepresenter.represent_undefined>, <class 'cis_interface.schema.SchemaRegistry'>: <unbound method SchemaDumper.represent_SchemaRegistry>, <class 'cis_interface.schema.ComponentSchema'>: <unbound method SchemaDumper.represent_ComponentSchema>, <type 'bool'>: <unbound method SafeRepresenter.represent_bool>, <type 'classobj'>: <unbound method Representer.represent_name>, <type 'complex'>: <unbound method Representer.represent_complex>, <type 'float'>: <unbound method SafeRepresenter.represent_float>, <type 'function'>: <unbound method Representer.represent_name>, <type 'int'>: <unbound method SafeRepresenter.represent_int>, <type 'list'>: <unbound method SafeRepresenter.represent_list>, <type 'long'>: <unbound method Representer.represent_long>, <type 'dict'>: <unbound method SafeRepresenter.represent_dict>, <type 'builtin_function_or_method'>: <unbound method Representer.represent_name>, <type 'module'>: <unbound method Representer.represent_module>, <type 'NoneType'>: <unbound method SafeRepresenter.represent_none>, <type 'set'>: <unbound method SafeRepresenter.represent_set>, <type 'str'>: <unbound method Representer.represent_str>, <type 'tuple'>: <unbound method SchemaDumper.represent_python_tuple>, <type 'type'>: <unbound method Representer.represent_name>, <type 'unicode'>: <unbound method Representer.represent_unicode>, <type 'datetime.datetime'>: <unbound method SafeRepresenter.represent_datetime>, <type 'datetime.date'>: <unbound method SafeRepresenter.represent_date>}
class cis_interface.schema.SchemaLoader(stream)[source]

Bases: yaml.loader.SafeLoader

SafeLoader for schema that includes tuples.

construct_python_tuple(node)[source]
yaml_constructors = {None: <unbound method SafeConstructor.construct_undefined>, u'tag:yaml.org,2002:binary': <unbound method SafeConstructor.construct_yaml_binary>, u'tag:yaml.org,2002:bool': <unbound method SafeConstructor.construct_yaml_bool>, u'tag:yaml.org,2002:float': <unbound method SafeConstructor.construct_yaml_float>, u'tag:yaml.org,2002:int': <unbound method SafeConstructor.construct_yaml_int>, u'tag:yaml.org,2002:map': <unbound method SafeConstructor.construct_yaml_map>, u'tag:yaml.org,2002:null': <unbound method SafeConstructor.construct_yaml_null>, u'tag:yaml.org,2002:omap': <unbound method SafeConstructor.construct_yaml_omap>, u'tag:yaml.org,2002:pairs': <unbound method SafeConstructor.construct_yaml_pairs>, 'tag:yaml.org,2002:python/tuple': <unbound method SchemaLoader.construct_python_tuple>, u'tag:yaml.org,2002:seq': <unbound method SafeConstructor.construct_yaml_seq>, u'tag:yaml.org,2002:set': <unbound method SafeConstructor.construct_yaml_set>, u'tag:yaml.org,2002:str': <unbound method SafeConstructor.construct_yaml_str>, u'tag:yaml.org,2002:timestamp': <unbound method SafeConstructor.construct_yaml_timestamp>}
class cis_interface.schema.SchemaRegistry(registry=None, required=None)[source]

Bases: dict

Registry of schema’s for different integration components.

Parameters:
  • registry (dict, optional) – Dictionary of registered components. Defaults to None and the registry will be empty.
  • required (list, optional) – Components that are required. Defaults to [‘comm’, ‘file’, ‘model’, ‘connection’]. Ignored if registry is None.
Raises:

ValueError – If registry is provided and one of the required components is missing.

class2conntype

dict – Mapping from connection class to comm classes & direction.

class2filetype

dict – Mapping from communication class to filetype.

class2language

dict – Mapping from ModelDriver class to programming language.

conntype2class

dict – Mapping from comm classes & direction to connection class.

filetype2class

dict – Mapping from filetype to communication class.

classmethod from_file(fname)[source]

Create a SchemaRegistry from a file.

Parameters:fname (str) – Full path to the file the schema should be loaded from.
language2class

dict – Mapping from programming language to ModelDriver class.

load(fname)[source]

Load schema from a file.

Parameters:fname (str) – Full path to the file the schema should be loaded from.
save(fname)[source]

Save the schema to a file.

Parameters:
  • fname (str) – Full path to the file the schema should be saved to.
  • schema (dict) – cis_interface YAML options.
validator

Compose complete schema for parsing yaml.

class cis_interface.schema.SchemaValidator(*args, **kwargs)[source]

Bases: cerberus.validator.Validator

Class for validating the schema.

cis_type_order = ['list', 'string', 'integer', 'boolean', 'function']
coercers = ('boolean', 'function', 'integer', 'list', 'string')
default_setters = ()
normalization_rules = {'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'purge_unknown': {'type': 'boolean'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}
rules = {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'coerce': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'default': {'nullable': True}, 'default_setter': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'purge_unknown': {'type': 'boolean'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'rename': {'type': 'hashable'}, 'rename_handler': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}}, {'type': 'string', 'allowed': ('boolean', 'function', 'integer', 'list', 'string')}]}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}
types_mapping = {'binary': TypeDefinition(name='binary', included_types=(<type 'str'>, <type 'bytearray'>), excluded_types=()), 'boolean': TypeDefinition(name='boolean', included_types=(<type 'bool'>,), excluded_types=()), 'date': TypeDefinition(name='date', included_types=(<type 'datetime.date'>,), excluded_types=()), 'datetime': TypeDefinition(name='datetime', included_types=(<type 'datetime.datetime'>,), excluded_types=()), 'dict': TypeDefinition(name='dict', included_types=(<class '_abcoll.Mapping'>,), excluded_types=()), 'float': TypeDefinition(name='float', included_types=(<type 'float'>, (<type 'int'>, <type 'long'>)), excluded_types=()), 'function': TypeDefinition(name='function', included_types=<type 'function'>, excluded_types=()), 'integer': TypeDefinition(name='integer', included_types=((<type 'int'>, <type 'long'>),), excluded_types=()), 'list': TypeDefinition(name='list', included_types=(<class '_abcoll.Sequence'>,), excluded_types=(<type 'basestring'>,)), 'number': TypeDefinition(name='number', included_types=((<type 'int'>, <type 'long'>), <type 'float'>), excluded_types=(<type 'bool'>,)), 'set': TypeDefinition(name='set', included_types=(<type 'set'>,), excluded_types=()), 'string': TypeDefinition(name='string', included_types=<type 'basestring'>, excluded_types=())}
validation_rules = {'allof': {'type': 'list', 'logical': 'allof'}, 'allow_unknown': {'oneof': [{'type': 'boolean'}, {'validator': 'bulk_schema', 'type': ['dict', 'string']}]}, 'allowed': {'type': 'list'}, 'anyof': {'type': 'list', 'logical': 'anyof'}, 'dependencies': {'validator': 'dependencies', 'type': ('dict', 'hashable', 'list')}, 'empty': {'type': 'boolean'}, 'excludes': {'type': ('hashable', 'list'), 'schema': {'type': 'hashable'}}, 'forbidden': {'type': 'list'}, 'items': {'validator': 'items', 'type': 'list'}, 'keyschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}, 'max': {'nullable': False}, 'maxlength': {'type': 'integer'}, 'min': {'nullable': False}, 'minlength': {'type': 'integer'}, 'noneof': {'type': 'list', 'logical': 'noneof'}, 'nullable': {'type': 'boolean'}, 'oneof': {'type': 'list', 'logical': 'oneof'}, 'readonly': {'type': 'boolean'}, 'regex': {'type': 'string'}, 'required': {'type': 'boolean'}, 'schema': {'anyof': [{'validator': 'schema'}, {'validator': 'bulk_schema'}], 'type': ['dict', 'string']}, 'type': {'validator': 'type', 'type': ['string', 'list']}, 'validator': {'oneof': [{'type': 'callable'}, {'type': 'list', 'schema': {'oneof': [{'type': 'callable'}, {'type': 'string', 'allowed': ()}]}}, {'type': 'string', 'allowed': ()}]}, 'valueschema': {'forbidden': ['rename', 'rename_handler'], 'validator': 'bulk_schema', 'type': ['dict', 'string']}}
validators = ()
cis_interface.schema.clear_schema()[source]

Clear global schema.

cis_interface.schema.create_schema()[source]

Create a new schema from the registry.

cis_interface.schema.get_schema(fname=None)[source]

Return the cis_interface schema for YAML options.

Parameters:fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
Returns:cis_interface YAML options.
Return type:dict
cis_interface.schema.inherit_schema(orig, key, value, **kwargs)[source]

Create an inherited schema, adding new value to accepted ones for dependencies.

Parameters:
  • orig (dict) – Schema that will be inherited.
  • key (str) – Field that other fields are dependent on.
  • value (str) – New value for key that dependent fields should accept.
  • **kwargs – Additional keyword arguments will be added to the schema with dependency on the provided key/value pair.
Returns:

New schema.

Return type:

dict

cis_interface.schema.init_registry()[source]

Initialize the registries and schema.

cis_interface.schema.init_schema(fname=None)[source]

Initialize global schema.

cis_interface.schema.load_schema(fname=None)[source]

Return the cis_interface schema for YAML options.

Parameters:fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
Returns:cis_interface YAML options.
Return type:dict
cis_interface.schema.register_component(component_class)[source]

Decorator for registering a class as a yaml component.

cis_interface.schema.str_to_function(value)[source]

Convert a string to a function.

Parameters:value (str, list) – String or list of strings, specifying function(s). The format should be “<package.module>:<function>” so that <function> can be imported from <package>.
Returns:Callable function.
Return type:func

cis_interface.tools module

This modules offers various tools.

class cis_interface.tools.CisClass(name, uuid=None, working_dir=None, timeout=60.0, sleeptime=0.01, **kwargs)[source]

Bases: logging.LoggerAdapter

Base class for CiS classes.

Parameters:
  • name (str) – Class name.
  • uuid (str, optional) – Unique ID for this instance. Defaults to None and is assigned.
  • working_dir (str, optional) – Working directory. If not provided, the current working directory is used.
  • timeout (float, optional) – Maximum time (in seconds) that should be spent waiting on a process. Defaults to 60.
  • sleeptime (float, optional) – Time that class should sleep for when sleep is called. Defaults to 0.01.
  • **kwargs – Additional keyword arguments are assigned to the extra_kwargs dictionary.
name

str – Class name.

uuid

str – Unique ID for this instance.

sleeptime

float – Time that class should sleep for when sleep called.

longsleep

float – Time that the class will sleep for when waiting for longer tasks to complete (10x longer than sleeptime).

timeout

float – Maximum time that should be spent waiting on a process.

working_dir

str – Working directory.

errors

list – List of errors.

extra_kwargs

dict – Keyword arguments that were not parsed.

sched_out

obj – Output from the last scheduled task with output.

logger

logging.Logger – Logger object for this object.

suppress_special_debug

bool – If True, special_debug log messages are suppressed.

as_str(obj)[source]

Return str version of object if it is not already a string.

Parameters:obj (object) – Object that should be turned into a string.
Returns:String version of provided object.
Return type:str
check_timeout(key=None, key_level=0)[source]

Check timeout for the calling function/method.

Parameters:
  • key (str, optional) – Key for timeout that should be checked. Defaults to None and is set by the calling class and function/method (See timeout_key).
  • key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
Raises:

KeyError – If there is not a timeout registered for the specified key.

debug_log()[source]

Turn on debugging.

display(msg='', *args, **kwargs)[source]

Print a message, no log.

dummy_log(*args, **kwargs)[source]

Dummy log function that dosn’t do anything.

error

Log an error level message.

exception

Log an exception level message.

get_timeout_key(key_level=0, key_suffix=None)[source]

Return a key for a given level in the stack, relative to the function calling get_timeout_key.

Parameters:
  • key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that is 2 steps higher in the stack. Higher values use classes and function/methods further up in the stack. Defaults to 0.
  • key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
Returns:

Key identifying calling object and method.

Return type:

str

name

Name of the class object.

printStatus()[source]

Print the class status.

print_encoded(msg, *args, **kwargs)[source]

Print bytes to stdout, encoding if possible.

Parameters:
  • msg (str, bytes) – Message to print.
  • *args – Additional arguments are passed to print.
  • **kwargs – Additional keyword arguments are passed to print.
process(msg, kwargs)[source]

Process logging message.

raise_error(e)[source]

Raise an exception, logging it first.

Parameters:e (Exception) – Exception to raise.
Raises:The provided exception.
reset_log()[source]

Resetting logging to prior value.

sched_task(t, func, args=[], kwargs={}, store_output=False)[source]

Schedule a task that will be executed after a certain time has elapsed.

Parameters:
  • t (float) – Number of seconds that should be waited before task is executed.
  • func (object) – Function that should be executed.
  • args (list, optional) – Arguments for the provided function. Defaults to [].
  • kwargs (dict, optional) – Keyword arguments for the provided function. Defaults to {}.
  • store_output (bool, optional) – If True, the output from the scheduled task is stored in self.sched_out. Otherwise, it is not stored. Defaults to False.
sleep(t=None)[source]

Have the class sleep for some period of time.

Parameters:t (float, optional) – Time that class should sleep for. If not provided, the attribute ‘sleeptime’ is used.
special_debug

Log debug level message contingent of supression flag.

start_timeout(t=None, key=None, key_level=0, key_suffix=None)[source]

Start a timeout for the calling function/method.

Parameters:
  • t (float, optional) – Maximum time that the calling function should wait before timeing out. If not provided, the attribute ‘timeout’ is used.
  • key (str, optional) – Key that should be associated with the timeout that is created. Defaults to None and is set by the calling class and function/method (See get_timeout_key).
  • key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
  • key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
Raises:

KeyError – If the key already exists.

stop_timeout(key=None, key_level=0, key_suffix=None, quiet=False)[source]

Stop a timeout for the calling function method.

Parameters:
  • key (str, optional) – Key for timeout that should be stopped. Defaults to None and is set by the calling class and function/method (See timeout_key).
  • key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.
  • key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
  • quiet (bool, optional) – If True, error message on timeout exceeded will be debug log. Defaults to False.
Raises:

KeyError – If there is not a timeout registered for the specified key.

timeout_key

str – Key identifying calling object and method.

verbose_debug(*args, **kwargs)[source]

Log a verbose debug level message.

class cis_interface.tools.CisPopen(cmd_args, forward_signals=True, **kwargs)[source]

Bases: subprocess.Popen

Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe. In addition this class overrides Popen.kill() to allow processes to be killed with CTRL_BREAK_EVENT on windows.

Parameters:
  • args (list, str) – Shell command or list of arguments that should be run.
  • forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True.
  • **kwargs – Additional keywords arguments are passed to Popen.
kill(*args, **kwargs)[source]

On windows using CTRL_BREAK_EVENT to kill the process.

class cis_interface.tools.CisThread(name=None, target=None, args=(), kwargs=None, daemon=False, group=None, **cis_kwargs)[source]

Bases: threading.Thread, cis_interface.tools.CisClass

Thread for CiS that tracks when the thread is started and joined.

lock

threading.RLock – Lock for accessing the sockets from multiple threads.

start_event

threading.Event – Event indicating that the thread was started.

terminate_event

threading.Event – Event indicating that the thread should be terminated. The target must exit when this is set.

atexit()[source]

Actions performed when python exits.

before_start()[source]

Actions to perform on the main thread before starting the thread.

cleanup()[source]

Actions to perform to clean up the thread after it has stopped.

main_terminated

bool – True if the main thread has terminated.

run(*args, **kwargs)[source]

Continue running until terminate event set.

set_started_flag()[source]

Set the started flag for the thread to True.

set_terminated_flag()[source]

Set the terminated flag for the thread to True.

start(*args, **kwargs)[source]

Start thread and print info.

terminate(no_wait=False)[source]

Set the terminate event and wait for the thread to stop.

Parameters:no_wait (bool, optional) – If True, terminate will not block until the thread stops. Defaults to False and blocks.
Raises:AssertionError – If no_wait is False and the thread has not stopped after the timeout.
unset_started_flag()[source]

Set the started flag for the thread to False.

unset_terminated_flag()[source]

Set the terminated flag for the thread to False.

wait(timeout=None, key=None)[source]

Wait until thread finish to return using sleeps rather than blocking.

Parameters:
  • timeout (float, optional) – Maximum time that should be waited for the driver to finish. Defaults to None and is infinite.
  • key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.
was_started

bool – True if the thread was started. False otherwise.

was_terminated

bool – True if the thread was terminated. False otherwise.

class cis_interface.tools.CisThreadLoop(*args, **kwargs)[source]

Bases: cis_interface.tools.CisThread

Thread that will run a loop until the terminate event is called.

after_loop()[source]

Actions performed after the loop.

before_loop()[source]

Actions performed before the loop.

on_main_terminated(dont_break=False)[source]

Actions performed when 1st main terminated.

Parameters:dont_break (bool, optional) – If True, the break flag won’t be set. Defaults to False.
run(*args, **kwargs)[source]

Continue running until terminate event set.

run_loop(*args, **kwargs)[source]

Actions performed on each loop iteration.

set_break_flag()[source]

Set the break flag for the thread to True.

set_loop_flag()[source]

Set the loop flag for the thread to True.

terminate(*args, **kwargs)[source]

Also set break flag.

unset_break_flag()[source]

Set the break flag for the thread to False.

unset_loop_flag()[source]

Set the loop flag for the thread to False.

wait_for_loop(timeout=None, key=None)[source]

Wait until thread enters loop to return using sleeps rather than blocking.

Parameters:
  • timeout (float, optional) – Maximum time that should be waited for the thread to enter loop. Defaults to None and is infinite.
  • key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.
was_break

bool – True if the break flag was set.

was_loop

bool – True if the thread was loop. False otherwise.

class cis_interface.tools.TimeOut(max_time, key=None)[source]

Bases: object

Class for checking if a period of time has been elapsed.

Parameters:
  • max_time (float, bbol) – Maximum period of time that should elapse before ‘is_out’ returns True. If False, ‘is_out’ will never return True. Providing 0 indicates that ‘is_out’ should immediately return True.
  • key (str, optional) – Key that was used to register the timeout. Defaults to None.
max_time

float – Maximum period of time that should elapsed before ‘is_out’ returns True.

start_time

float – Result of time.time() at start.

key

str – Key that was used to register the timeout.

elapsed

float – Total time that has elapsed since the start.

is_out

bool – True if there is not any time remaining. False otherwise.

cis_interface.tools.check_locks()[source]

Check for locks in lock registry that are locked.

cis_interface.tools.check_sockets()[source]

Check registered sockets.

cis_interface.tools.check_threads()[source]

Check for threads that are still running.

cis_interface.tools.cis_atexit()[source]

Things to do at exit.

cis_interface.tools.eval_kwarg(x)[source]

If x is a string, eval it. Otherwise just return it.

Parameters:x (str, obj) – String to be evaluated as an object or an object.
Returns:Result of evaluated string or the input object.
Return type:obj
cis_interface.tools.get_CIS_MSG_MAX()[source]

Get the maximum message size for the default comm.

cis_interface.tools.get_default_comm()[source]

Get the default comm that should be used for message passing.

cis_interface.tools.is_ipc_installed()[source]

Determine if the IPC libraries are installed.

Returns:True if the IPC libraries are installed, False otherwise.
Return type:bool
cis_interface.tools.is_zmq_installed(check_c=True)[source]

Determine if the libczmq & libzmq libraries are installed.

Returns:True if both libraries are installed, False otherwise.
Return type:bool
cis_interface.tools.kill(pid, signum)[source]

Kill process by mapping signal number.

Parameters:
  • pid (int) – Process ID.
  • signum (int) – Signal that should be sent.
cis_interface.tools.locate_path(fname, basedir='/')[source]

Find the full path to a file using where on Windows.

cis_interface.tools.popen_nobuffer(*args, **kwargs)[source]

Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe.

Parameters:
  • args (list, str) – Shell command or list of arguments that should be run.
  • forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True.
  • **kwargs – Additional keywords arguments are passed to Popen.
Returns:

Process that was started.

Return type:

CisPopen

cis_interface.tools.print_encoded(msg, *args, **kwargs)[source]

Print bytes to stdout, encoding if possible.

Parameters:
  • msg (str, bytes) – Message to print.
  • *args – Additional arguments are passed to print.
  • **kwargs – Additional keyword arguments are passed to print.
cis_interface.tools.sleep(interval)[source]

Sleep for a specified number of seconds.

Parameters:interval (float) – Time in seconds that process should sleep.

cis_interface.units module

cis_interface.units.is_unit(ustr)[source]

Determine if a string is a valid unit.

Parameters:ustr – String representation to test.
Returns:True if the string is a valid unit. False otherwise.
Return type:bool

cis_interface.yamlfile module

cis_interface.yamlfile.bridge_io_drivers(yml, iodict)[source]

Create connection from input/output driver.

Parameters:
  • dict – YAML entry for model output that should be modified.
  • iodict (dict) – Log of all input/outputs.
cis_interface.yamlfile.cdriver2filetype(driver)[source]

Convert a connection driver to a file type.

Parameters:driver (str) – The name of the connection driver.
Returns:The corresponding file type for the driver.
Return type:str

Link I/O drivers back to the models they communicate with.

Parameters:existing (dict) – Dictionary of existing components.
Returns:Dictionary with I/O drivers added to models.
Return type:dict
cis_interface.yamlfile.load_yaml(fname)[source]

Parse a yaml file defining a run.

Parameters:fname (str) – Path to the yaml file.
Returns:Contents of yaml file.
Return type:dict
cis_interface.yamlfile.parse_component(yml, ctype, existing=None)[source]

Parse a yaml entry for a component, adding it to the list of existing components.

Parameters:
  • yml (dict) – YAML dictionary for a component.
  • ctype (str) – Component type. This can be ‘input’, ‘output’, ‘model’, or ‘connection’.
  • existing (dict, optional) – Dictionary of existing components. Defaults to empty dict.
Raises:
  • TypeError – If yml is not a dictionary.
  • ValueError – If dtype is not ‘input’, ‘output’, ‘model’, or ‘connection’.
  • ValueError – If the component already exists.
Returns:

All components identified.

Return type:

dict

cis_interface.yamlfile.parse_connection(yml, existing)[source]

Parse a yaml entry for a connection between I/O channels.

Parameters:
  • yml (dict) – YAML dictionary for a connection.
  • existing (dict) – Dictionary of existing components.
Raises:
  • RuntimeError – If the ‘input’ entry is not a model output or file.
  • RuntimeError – If neither the ‘input’ or ‘output’ entries correspond to model I/O channels.
Returns:

Updated log of all entries.

Return type:

dict

cis_interface.yamlfile.parse_model(yml, existing)[source]

Parse a yaml entry for a model.

Parameters:
  • yml (dict) – YAML dictionary for a model.
  • existing (dict) – Dictionary of existing components.
Returns:

Updated log of all entries.

Return type:

dict

cis_interface.yamlfile.parse_yaml(files)[source]

Parse list of yaml files.

Parameters:

files (str, list) – Either the path to a single yaml file or a list of yaml files.

Raises:
  • ValueError – If the yml dictionary is missing a required keyword or has an invalid value.
  • RuntimeError – If one of the I/O channels is not initialized with driver information.
Returns:

Dictionary of information parsed from the yamls.

Return type:

dict

cis_interface.yamlfile.prep_connection(yml, iodict)[source]

Prepare yaml connection entry for parsing with schema.

Parameters:
  • yml (dict) – YAML entry for connection that will be modified.
  • iodict (dict) – Log of all input/outputs.
cis_interface.yamlfile.prep_model(yml, iodict)[source]

Prepare yaml model entry for parsing with schema.

Parameters:
  • yml (dict) – YAML entry for model that will be modified.
  • iodict (dict) – Log of all input/outputs.
cis_interface.yamlfile.prep_yaml(files)[source]

Prepare yaml to be parsed by Cerberus using schema including covering backwards compatible options.

Parameters:files (str, list) – Either the path to a single yaml file or a list of yaml files.
Returns:YAML ready to be parsed using schema.
Return type:dict
cis_interface.yamlfile.rwmeth2filetype(rw_meth)[source]

Module contents

This package provides a framework for integrating models across languages such that they can be run simultaneously, passing input back and forth.