cis_interface.drivers package

Subpackages

Submodules

cis_interface.drivers.AsciiFileInputDriver module

class cis_interface.drivers.AsciiFileInputDriver.AsciiFileInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileInputDriver.FileInputDriver

Class that sends lines from an ASCII file.

Parameters:
  • name (str) – Name of the queue that messages should be sent to.
  • args (str or dict) – Path to the file that messages should be read from or dictionary containing the filepath and other keyword arguments to be passed to the created AsciiFile object.
  • comment (str, optional) – String that should be used to identify comments. Default set by AsciiFile.
  • newline (str, optional) – String that should be used to identify the end of a line. Default set by AsciiFile.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.AsciiFileOutputDriver module

class cis_interface.drivers.AsciiFileOutputDriver.AsciiFileOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileOutputDriver.FileOutputDriver

Class to handle output line by line to an ASCII file.

Parameters:
  • name (str) – Name of the output queue to receive messages from.
  • args (str or dict) – Path to the file that messages should be written to or dictionary containing the filepath and other keyword arguments to be passed to the created AsciiFile object.
  • comment (str, optional) – String that should be used to identify comments. Default set by AsciiFile.
  • newline (str, optional) – String that should be used to identify the end of a line. Default set by AsciiFile.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.AsciiTableInputDriver module

class cis_interface.drivers.AsciiTableInputDriver.AsciiTableInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.AsciiFileInputDriver.AsciiFileInputDriver

Class to handle input from an ASCII table.

Parameters:
  • name (str) – Name of the input queue to send messages to.
  • args (str or dict) – Path to the file that messages should be read from or dictionary containing the filepath and other keyword arguments to be passed to the created AsciiTable object.
  • format_str (str) – Format string that should be used to format output in the case that the io_mode is ‘w’ (write). It is not required if the io_mode is any other value.
  • field_names (list, optional) – List of column names. Defaults to None.
  • field_units (list, optional) – List of column units. Defaults to None.
  • use_astropy (bool, optional) – If True, astropy is used to determine a table’s format if it is installed. If False, a format string must be contained in the table. Defaults to False.
  • delimiter (str, optional) – String that should be used to separate columns. Default set by serialize._default_delimiter.
  • comment (str, optional) – String that should be used to identify comments. Default set by serialize._default_comment.
  • newline (str, optional) – String that should be used to identify the end of a line. Default set by serialize._default_newline.
  • as_array (bool, optional) – If True, the table contents are sent all at once as an array. Defaults to False.
  • **kwargs – Additional keyword arguments are passed to parent class.
update_serializer(msg)[source]

Update the serializer for the output comm based on input.

cis_interface.drivers.AsciiTableOutputDriver module

class cis_interface.drivers.AsciiTableOutputDriver.AsciiTableOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.AsciiFileOutputDriver.AsciiFileOutputDriver

Class to handle output of received messages to an ASCII table.

Parameters:
  • name (str) – Name of the output queue to receive messages from.
  • args (str or dict) – Path to the file that messages should be written to or dictionary containing the filepath and other keyword arguments to be passed to the created AsciiTable object.
  • format_str (str) – Format string that should be used to format output in the case that the io_mode is ‘w’ (write). It is not required if the io_mode is any other value.
  • field_names (list, optional) – List of column names. Defaults to None.
  • field_units (list, optional) – List of column units. Defaults to None.
  • use_astropy (bool, optional) – If True, astropy is used to determine a table’s format if it is installed. If False, a format string must be contained in the table. Defaults to False.
  • delimiter (str, optional) – String that should be used to separate columns. Default set by serialize._default_delimiter.
  • comment (str, optional) – String that should be used to identify comments. Default set by serialize._default_comment.
  • newline (str, optional) – String that should be used to identify the end of a line. Default set by serialize._default_newline.
  • as_array (bool, optional) – If True, the table contents are sent all at once as an array. Defaults to False.
  • **kwargs – Additional keyword arguments are passed to parent class.
update_serializer(msg)[source]

Update the serializer for the output comm based on input.

cis_interface.drivers.CMakeModelDriver module

class cis_interface.drivers.CMakeModelDriver.CMakeModelDriver(name, args, make_command=None, makedir=None, makefile=None, **kwargs)[source]

Bases: cis_interface.drivers.ModelDriver.ModelDriver

Class for running cmake file compiled drivers. Before running the cmake command, the necessary compiler & linker flags for the interface’s C/C++ library are stored the environment variables CISCCFLAGS and CISLDFLAGS respectively. These should be used in the make file to correctly compile with the interface’s C/C++ libraries.

Parameters:
  • name (str) – Driver name.
  • args (str, list) – Executable that should be created (make target) and any arguments for the executable.
  • make_command (str, optional) – Command that should be used for make. Defaults to ‘make’ on linux/osx and ‘nmake’ on windows.
  • makefile (str, optional) – Path to make file either relative to makedir or absolute. Defaults to Makefile.
  • makedir (str, optional) – Directory where make should be invoked from if it is not the same as the directory containing the makefile. Defaults to directory containing makefile if an absolute path is provided, otherwise self.workingDir.
  • **kwargs – Additional keyword arguments are passed to parent class.
compiled

bool – True if the compilation was successful, False otherwise.

target

str – Name of executable that should be created and called.

make_command

str – Command that should be used for make.

makedir

str – Directory where make should be invoked from.

makefile

str – Path to make file either relative to makedir or absolute.

Raises:RuntimeError – If neither the IPC or ZMQ C libraries are available.
cleanup()[source]

Remove compile executable.

make_target(target)[source]

Run the make command to make the target.

Parameters:target (str) – Target that should be made.
Raises:RuntimeError – If there is an error in running the make.
cis_interface.drivers.CMakeModelDriver.setup_environ(compile_flags=[], linker_flags=[])[source]

Set environment variables CISCCFLAGS and CISLDFLAGS.

Parameters:
  • compile_flags (list, optional) – Additional compile flags that should be set. Defaults to [].
  • linker_flags (list, optional) – Additional linker flags that should be set. Defaults to [].

cis_interface.drivers.ClientDriver module

class cis_interface.drivers.ClientDriver.ClientDriver(model_request_name, request_name=None, comm=None, comm_address=None, **kwargs)[source]

Bases: cis_interface.drivers.ClientRequestDriver.ClientRequestDriver

Alias for ClientRequestDriver.

cis_interface.drivers.ClientRequestDriver module

class cis_interface.drivers.ClientRequestDriver.ClientRequestDriver(model_request_name, request_name=None, comm=None, comm_address=None, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class for handling client side RPC type communication.

Parameters:
  • model_request_name (str) – The name of the channel used by the client model to send requests.
  • request_name (str, optional) – The name of the channel used to send requests to the server request driver. Defaults to model_request_name + ‘_SERVER’ if not set.
  • comm (str, optional) – The comm class that should be used to communicate with the server request driver. Defaults to tools.get_default_comm().
  • comm_address (str, optional) – Address for the server request driver. Defaults to None and a new address is generated.
  • **kwargs – Additional keyword arguments are passed to parent class.
comm

str – The comm class that should be used to communicate with the server request driver.

comm_address

str – Address for the server request driver.

response_drivers

list – Response drivers created for each request.

before_loop()[source]

Send client sign on to server response driver.

close_comm()[source]

Close response drivers.

close_response_drivers()[source]

Close response driver.

last_header

dict – Information contained in the header of the last message received from the client model.

model_response_address

str – The address of the channel used by the client model to receive responses.

printStatus(*args, **kwargs)[source]

Also print response drivers.

request_address

str – The address of the channel used to send requests to the server request driver.

request_id

str – Unique ID for the last message.

request_name

str – The name of the channel used to send requests to the server request driver.

send_eof()[source]

Send EOF message.

Returns:Success or failure of send.
Return type:bool
send_message(*args, **kwargs)[source]

Start a response driver for a request message and send message with header.

Parameters:
  • *args – Arguments are passed to parent class send_message.
  • **kwargs – Keyword arguments are passed to parent class send_message.
Returns:

Success or failure of send.

Return type:

bool

cis_interface.drivers.ClientResponseDriver module

class cis_interface.drivers.ClientResponseDriver.ClientResponseDriver(model_response_address, request_name=None, comm=None, msg_id=None, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class for handling client side RPC type communication.

Parameters:
  • model_response_address (str) – The address of the channel used by the client model to receive responses.
  • comm (str, optional) – The comm class that should be used to communicate with the server response driver. Defaults to tools.get_default_comm().
  • msg_id (str, optional) – ID associate with the request message this driver was created to respond to. Defaults to new unique ID.
  • **kwargs – Additional keyword arguments are passed to parent class.
comm

str – The comm class that should be used to communicate with the server response driver.

msg_id

str – ID associate with the request message this driver was created to respond to.

response_address

str – Address of response comm.

cis_interface.drivers.CommDriver module

class cis_interface.drivers.CommDriver.CommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.Driver.Driver

Base driver for any driver that does communication.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent and comm classes.
comm_name

str – Name of communication class.

comm

CommBase – Instance of communication class.

state

str – Description of the last operation performed by the driver.

numSent

int – The number of messages sent to the queue.

numReceived

int – The number of messages received from the queue.

cleanup()[source]

Ensure that the queues are removed.

close_comm()[source]

Close the queue.

do_terminate()[source]

Stop the CommDriver by closing the comm.

graceful_stop(timeout=None, **kwargs)[source]

Stop the CommDriver, first draining the message queue.

Parameters:
  • timeout (float, optional) – Max time that should be waited. Defaults to None and is set to attribute timeout. If 0, it will never timeout.
  • **kwargs – Additional keyword arguments are passed to the parent class’s graceful_stop method.
is_comm_closed

bool – Returns True if the connection is closed.

is_comm_open

bool – Returns True if the connection is open.

is_valid

bool – Returns True if the connection is open and the parent class is valid.

maxMsgSize

int – Maximum message size.

n_msg

int – The number of messages in the queue.

open_comm()[source]

Open the queue.

printStatus(beg_msg='', end_msg='')[source]

Print information on the status of the CommDriver.

Parameters:
  • beg_msg (str, optional) – Additional message to print at beginning.
  • end_msg (str, optional) – Additional message to print at end.
recv(*args, **kwargs)[source]

Receive a message smaller than maxMsgSize.

Parameters:
  • *args – All arguments are passed to comm recv method.
  • **kwargs – All keywords arguments are passed to comm recv method.
Returns:

The success or failure of receiving and the

received message.

Return type:

tuple (bool, str)

recv_nolimit(*args, **kwargs)[source]

Receive a message larger than maxMsgSize in multiple parts.

Parameters:
  • *args – All arguments are passed to comm recv_nolimit method.
  • **kwargs – All keywords arguments are passed to comm recv_nolimit method.
Returns:

The success or failure of receiving and the

received message.

Return type:

tuple (bool, str)

send(data, *args, **kwargs)[source]

Send a message smaller than maxMsgSize.

Parameters:
  • str – The message to be sent.
  • *args – All arguments are passed to comm send method.
  • **kwargs – All keywords arguments are passed to comm send method.
Returns:

Success or failure of send.

Return type:

bool

send_nolimit(data, *args, **kwargs)[source]

Send a message larger than maxMsgSize in multiple parts.

Parameters:
  • str – The message to be sent.
  • *args – All arguments are passed to comm send_nolimit method.
  • **kwargs – All keywords arguments are passed to comm send_nolimit method.
Returns:

Success or failure of send.

Return type:

bool

start()[source]

Open connection before running.

cis_interface.drivers.ConnectionDriver module

Module for funneling messages from one comm to another.

class cis_interface.drivers.ConnectionDriver.ConnectionDriver(name, icomm_kws=None, ocomm_kws=None, translator=None, timeout_send_1st=None, single_use=False, **kwargs)[source]

Bases: cis_interface.drivers.Driver.Driver

Class that continuously passes messages from one comm to another.

Parameters:
  • name (str) – Name that should be used to set names of input/output comms.
  • icomm_kws (dict, optional) – Keyword arguments for the input communicator.
  • ocomm_kws (dict, optional) – Keyword arguments for the output communicator.
  • translator (str, func, optional) – Function or string specifying function that should be used to translate messages from the input communicator before passing them to the output communicator. If a string, the format should be “<package.module>:<function>” so that <function> can be imported from <package>. Defaults to None and messages are passed directly.
  • timeout_send_1st (float, optional) – Time in seconds that should be waited before giving up on the first send. Defaults to self.timeout.
  • single_use (bool, optional) – If True, the driver will be stopped after one loop. Defaults to False.
  • **kwargs – Additonal keyword arguments are passed to the parent class.
icomm_kws

dict – Keyword arguments for the input communicator.

ocomm_kws

dict – Keyword arguments for the output communicator.

icomm

CommBase – Input communicator.

ocomm

CommBase – Output communicator.

nrecv

int – Number of messages received.

nproc

int – Number of messages processed.

nsent

int – Number of messages sent.

nskip

int – Number of messages skipped.

state

str – Descriptor of last action taken.

translator

func – Function that will be used to translate messages from the input communicator before passing them to the output communicator.

timeout_send_1st

float – Time in seconds that should be waited before giving up on the first send.

single_use

bool – If True, the driver will be stopped after one loop.

after_loop()[source]

Actions to perform after sending messages.

before_loop()[source]

Actions to perform prior to sending messages.

cleanup()[source]

Ensure that the communicators are closed.

close_comm()[source]

Close the communicators.

do_terminate()[source]

Stop the driver by closing the communicators.

drain_input(timeout=None)[source]

Drain messages from input comm.

drain_output(timeout=None)[source]

Drain messages from output comm.

graceful_stop(timeout=None, **kwargs)[source]

Stop the driver, first waiting for the input comm to be empty.

Parameters:
  • timeout (float, optional) – Max time that should be waited. Defaults to None and is set to attribute timeout.
  • **kwargs – Additional keyword arguments are passed to the parent class’s graceful_stop method.
is_comm_closed

bool – Returns True if both communicators are closed.

is_comm_open

bool – Returns True if both communicators are open.

is_valid

bool – Returns True if the connection is open and the parent class is valid.

n_msg

int – Number of messages waiting in input communicator.

on_eof()[source]

Actions to take when EOF received.

Returns:Value that should be returned by recv_message on EOF.
Return type:str, bool
on_message(msg)[source]

Process a message.

Parameters:msg (bytes, str) – Message to be processed.
Returns:Processed message.
Return type:bytes, str
on_model_exit()[source]

Drain input and then close it.

open_comm()[source]

Open the communicators.

printStatus(beg_msg='', end_msg='')[source]

Print information on the status of the ConnectionDriver.

Parameters:
  • beg_msg (str, optional) – Additional message to print at beginning.
  • end_msg (str, optional) – Additional message to print at end.
recv_message(**kwargs)[source]

Get a new message to send.

Parameters:**kwargs – Additional keyword arguments are passed to the appropriate recv method.
Returns:False if no more messages, message otherwise.
Return type:str, bool
run_loop()[source]

Run the driver. Continue looping over messages until there are not any left or the communication channel is closed.

send_eof()[source]

Send EOF message.

Returns:Success or failure of send.
Return type:bool
send_message(*args, **kwargs)[source]

Send a single message.

Parameters:
  • *args – Arguments are passed to the output comm send method.
  • *kwargs – Keyword arguments are passed to the output comm send method.
Returns:

Success or failure of send.

Return type:

bool

set_close_state(state)[source]

Set the close state if its not already set.

start()[source]

Open connection before running.

update_serializer(msg)[source]

Update the serializer for the output comm based on input.

wait_for_route(timeout=None)[source]

Wait until messages have been routed.

cis_interface.drivers.Driver module

class cis_interface.drivers.Driver.Driver(name, yml=None, env=None, comm_env=None, namespace=None, rank=None, **kwargs)[source]

Bases: cis_interface.tools.CisThreadLoop

Base class for all drivers.

Parameters:
  • name (str) – Driver name.
  • yml (dict, optional) – Dictionary of yaml specification options for this driver. Defaults to empty dict.
  • env (dict, optional) – Dictionary of environment variables that should be set when the driver starts. Defaults to {}.
  • comm_env (dict, optional) – Dictionary of environment variables for paired IO communication drivers. Defaults to {}.
  • namespace (str, optional) – Namespace for set of drivers running together. If not provided, the config option (‘rmq’, ‘namespace’) is used.
  • rank (int, optional) – Rank of the integration. Defaults to None.
name

str – Driver name.

env

dict – Dictionary of environment variables.

comm_env

dict – Dictionary of environment variables for paired IO communication drivers.

yml

dict – Dictionary of yaml specification options for this driver.

namespace

str – Namespace for set of drivers running together.

rank

int – Rank of the integration.

do_terminate()[source]

Actions that should stop the driver.

graceful_stop()[source]

Gracefully stop the driver.

is_valid

bool – True if the driver is functional.

on_model_exit()[source]

Processes that should be run when an associated model exits.

stop()[source]

Stop the driver.

terminate()[source]

Stop the driver, without attempting to allow it to finish.

cis_interface.drivers.FileInputDriver module

class cis_interface.drivers.FileInputDriver.FileInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class that sends messages read from a file.

Parameters:
  • name (str) – Name of the queue that messages should be sent to.
  • args (str) – Path to the file that messages should be read from.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.FileOutputDriver module

class cis_interface.drivers.FileOutputDriver.FileOutputDriver(name, path, in_temp=False, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class to handle output of received messages to a file.

Parameters:
  • name (str) – Name of the output queue to receive messages from.
  • path (str) – Path to the file that messages should be written to.
  • in_temp (bool, str, optional) – If True or ‘True’, the path will be considered relative to the temporary directory. Defaults to False.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.GCCModelDriver module

class cis_interface.drivers.GCCModelDriver.GCCModelDriver(name, args, cc=None, **kwargs)[source]

Bases: cis_interface.drivers.ModelDriver.ModelDriver

Class for running gcc compiled drivers.

Parameters:
  • name (str) – Driver name.
  • args (str or list) – Argument(s) for running the model on the command line. If the first element ends with ‘.c’, the driver attempts to compile the code with the necessary interface include directories. Additional arguments that start with ‘-I’ are included in the compile command. Others are assumed to be runtime arguments.
  • cc (str, optional) – C/C++ Compiler that should be used. Defaults to gcc for ‘.c’ files, and g++ for ‘.cpp’ or ‘.cc’ files on Linux or OSX. Defaults to cl on Windows.
  • **kwargs – Additional keyword arguments are passed to parent class.
Attributes (in additon to parent class’s):
compiled (bool): True if the compilation was succesful. False otherwise. cfile (str): Source file. cc (str): C/C++ Compiler that should be used. flags (list): List of compiler flags. efile (str): Compiled executable file.
Raises:
cleanup()[source]

Remove compile executable.

parse_arguments(args)[source]

Sort arguments based on their syntax. Arguments ending with ‘.c’ or ‘.cpp’ are considered source and the first one will be compiled to an executable. Arguments starting with ‘-L’ or ‘-l’ are treated as linker flags. Arguments starting with ‘-‘ are treated as compiler flags. Any arguments that do not fall into one of the categories will be treated as command line arguments for the compiled executable.

Parameters:args (list) – List of arguments provided.
Raises:RuntimeError – If there is not a valid source file in the argument list.
remove_products()[source]

Delete products produced during the compilation process.

cis_interface.drivers.GCCModelDriver.build_regex_win32()[source]

Build the regex_win32 library.

cis_interface.drivers.GCCModelDriver.do_compile(src, out=None, cc=None, ccflags=None, ldflags=None, working_dir=None)[source]

Compile a C/C++ program with necessary interface libraries.

Parameters:
  • src (list) – List of source files.
  • out (str, optional) – Path where compile executable should be saved. Defaults to name of source file without extension on linux/osx and with .exe extension on windows.
  • cc (str, optional) – Compiler command. Defaults to gcc/g++ on linux/osx and cl on windows.
  • ccflags (list, optional) – Compiler flags. Defaults to [].
  • ldflags (list, optional) – Linker flags. Defaults to [].
  • working_dir (str, optional) – Working directory that input file paths are relative to. Defaults to current working directory.
Returns:

Full path to the compiled executable.

Return type:

str

cis_interface.drivers.GCCModelDriver.get_flags()[source]

Get the necessary flags for compiling & linking with CiS libraries.

Returns:compile and linker flags.
Return type:tuple(list, list)
cis_interface.drivers.GCCModelDriver.get_ipc_flags()[source]

Get the necessary flags for compiling & linking with ipc libraries.

Returns:compile and linker flags.
Return type:tuple(list, list)
cis_interface.drivers.GCCModelDriver.get_zmq_flags()[source]

Get the necessary flags for compiling & linking with zmq libraries.

Returns:compile and linker flags.
Return type:tuple(list, list)

cis_interface.drivers.IODriver module

class cis_interface.drivers.IODriver.IODriver(name, suffix='', **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Base driver for any driver that requires access to a message queue.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • suffix (str, optional) – Suffix added to name to create the environment variable where the message queue key is stored. Defaults to ‘’.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.IPCCommDriver module

class cis_interface.drivers.IPCCommDriver.IPCCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for communication via IPC queues.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.
-

cis_interface.drivers.InputCommDriver module

class cis_interface.drivers.InputCommDriver.InputCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for input communication.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.InputDriver module

class cis_interface.drivers.InputDriver.InputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Driver for receiving input from another model’s comm via a local comm.

Parameters:
  • name (str) – The name of the local message comm that the driver should connect to.
  • args (str) – The name of the other message comme that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.MakeModelDriver module

class cis_interface.drivers.MakeModelDriver.MakeModelDriver(name, args, make_command=None, makedir=None, makefile=None, **kwargs)[source]

Bases: cis_interface.drivers.ModelDriver.ModelDriver

Class for running make file compiled drivers. Before running the make command, the necessary compiler & linker flags for the interface’s C/C++ library are stored the environment variables CISCCFLAGS and CISLDFLAGS respectively. These should be used in the make file to correctly compile with the interface’s C/C++ libraries.

Parameters:
  • name (str) – Driver name.
  • args (str, list) – Executable that should be created (make target) and any arguments for the executable.
  • make_command (str, optional) – Command that should be used for make. Defaults to ‘make’ on linux/osx and ‘nmake’ on windows.
  • makefile (str, optional) – Path to make file either relative to makedir or absolute. Defaults to Makefile.
  • makedir (str, optional) – Directory where make should be invoked from if it is not the same as the directory containing the makefile. Defaults to directory containing makefile if an absolute path is provided, otherwise self.workingDir.
  • **kwargs – Additional keyword arguments are passed to parent class.
compiled

bool – True if the compilation was successful, False otherwise.

target

str – Name of executable that should be created and called.

make_command

str – Command that should be used for make.

makedir

str – Directory where make should be invoked from.

makefile

str – Path to make file either relative to makedir or absolute.

Raises:RuntimeError – If neither the IPC or ZMQ C libraries are available.
cleanup()[source]

Remove compile executable.

make_target(target)[source]

Run the make command to make the target.

Parameters:target (str) – Target that should be made.
Raises:RuntimeError – If there is an error in running the make.
cis_interface.drivers.MakeModelDriver.setup_environ(compile_flags=[], linker_flags=[])[source]

Set environment variables CISCCFLAGS and CISLDFLAGS.

Parameters:
  • compile_flags (list, optional) – Additional compile flags that should be set. Defaults to [].
  • linker_flags (list, optional) – Additional linker flags that should be set. Defaults to [].

cis_interface.drivers.MatInputDriver module

class cis_interface.drivers.MatInputDriver.MatInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileInputDriver.FileInputDriver

Class that sends pickled dictionaries of matricies read from a .mat file.

Parameters:
  • name (str) – Name of the queue that messages should be sent to.
  • args (str) – Path to the .mat file that messages should be read from.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.MatOutputDriver module

class cis_interface.drivers.MatOutputDriver.MatOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileOutputDriver.FileOutputDriver

Class to handle output to .mat Matlab files.

Parameters:
  • name (str) – Name of the output queue to receive messages from.
  • args (str) – Path to the file that messages should be written to.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.MatlabModelDriver module

class cis_interface.drivers.MatlabModelDriver.MatlabModelDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.ModelDriver.ModelDriver

Base class for running Matlab models.

Parameters:
  • name (str) – Driver name.
  • args (str or list) – Argument(s) for running the model in matlab. Generally, this should be the full path to a Matlab script.
  • **kwargs – Additional keyword arguments are passed to parent class’s __init__ method.
started_matlab

bool – True if the driver had to start a new matlab engine. False otherwise.

screen_session

str – Screen session that Matlab was started in.

mlengine

object – Matlab engine used to run script.

mlsession

str – Name of the Matlab session that was started.

Raises:RuntimeError – If Matlab is not installed.
after_loop()[source]

Actions to perform after run_loop has finished. Mainly checking if there was an error and then handling it.

before_start()[source]

Actions to perform before the run loop.

cleanup()[source]

Close the Matlab session and engine.

run_loop()[source]

Loop to check if model is still running and forward output.

start_matlab()[source]

Start matlab session and connect to it.

class cis_interface.drivers.MatlabModelDriver.MatlabProcess(target, args, kwargs=None, name=None)[source]

Bases: cis_interface.tools.CisClass

Add features to mimic subprocess.Popen while running Matlab function asynchronously.

Parameters:
  • target (func) – Matlab function that should be called.
  • args (list, tuple) – Arguments that should be passed to target.
  • kwargs (dict, optional) – Keyword arguments that should be passed to target. Defaults to empty dict.
stdout

StringIO – File like string buffer that stdout from target will be written to.

stderr

StringIO – File like string buffer that stderr from target will be written to.

target

func – Matlab function that should be called.

args

list, tuple – Arguments that should be passed to target.

kwargs

dict – Keyword arguments that should be passed to target.

future

MatlabFutureResult – Future result from async function. This will be None until start is called.

Raises:RuntimeError – If Matlab is not installed.
is_alive()[source]

bool: Is the async call funning.

is_cancelled()[source]

bool: Was the async call cancelled or not.

is_done()[source]

bool: Is the async call still running.

is_started()[source]

bool: Has start been called.

kill(*args, **kwargs)[source]

Cancel the async call.

poll(*args, **kwargs)[source]

Fake poll.

print_output()[source]

Print output from stdout and stderr.

returncode

int – Return code.

start()[source]

Start asychronous call.

stderr_line

str – Output to stderr from function call.

stdout_line

str – Output to stdout from function call.

cis_interface.drivers.MatlabModelDriver.install_matlab_engine()[source]

Install the MATLAB engine API for Python.

cis_interface.drivers.MatlabModelDriver.locate_matlabroot()[source]

Find directory that servers as matlab root.

Returns:Full path to matlabroot directory.
Return type:str
cis_interface.drivers.MatlabModelDriver.start_matlab()[source]

Start a Matlab shared engine session inside a detached screen session.

Returns:Name of the screen session running matlab.
Return type:str
Raises:RuntimeError – If Matlab is not installed.
cis_interface.drivers.MatlabModelDriver.stop_matlab(screen_session, matlab_engine, matlab_session)[source]

Stop a Matlab shared engine session running inside a detached screen session.

Parameters:
  • screen_session (str) – Name of the screen session that the shared Matlab session was started in.
  • matlab_engine (MatlabEngine) – Matlab engine that should be stopped.
  • matlab_session (str) – Name of Matlab session that the Matlab engine is connected to.
Raises:

RuntimeError – If Matlab is not installed.

cis_interface.drivers.ModelDriver module

class cis_interface.drivers.ModelDriver.ModelDriver(name, args, is_server=False, client_of=[], with_strace=False, strace_flags=None, with_valgrind=False, valgrind_flags=None, model_index=0, **kwargs)[source]

Bases: cis_interface.drivers.Driver.Driver

Base class for Model drivers and for running executable based models.

Parameters:
  • name (str) – Driver name.
  • args (str or list) – Argument(s) for running the model on the command line. This should be a complete command including the necessary executable and command line arguments to that executable.
  • is_server (bool, optional) – If True, the model is assumed to be a server and an instance of cis_interface.drivers.ServerDriver is started. Defaults to False.
  • client_of (str, list, optional) – The names of ne or more servers that this model is a client of. Defaults to empty list.
  • with_strace (bool, optional) – If True, the command is run with strace (on Linux) or dtrace (on OSX). Defaults to False.
  • strace_flags (list, optional) – Flags to pass to strace (or dtrace). Defaults to [].
  • with_valgrind (bool, optional) – If True, the command is run with valgrind. Defaults to False.
  • valgrind_flags (list, optional) – Flags to pass to valgrind. Defaults to [].
  • model_index (int, optional) – Index of model in list of models being run. Defaults to 0.
  • **kwargs – Additional keyword arguments are passed to parent class.
args

list – Argument(s) for running the model on the command line.

process

cis_interface.tools.CisPopen – Process used to run the model.

is_server

bool – If True, the model is assumed to be a server and an instance of cis_interface.drivers.ServerDriver is started.

client_of

list – The names of server models that this model is a client of.

with_strace

bool – If True, the command is run with strace or dtrace.

strace_flags

list – Flags to pass to strace/dtrace.

with_valgrind

bool – If True, the command is run with valgrind.

valgrind_flags

list – Flags to pass to valgrind.

model_index

int – Index of model in list of models being run.

Raises:RuntimeError – If both with_strace and with_valgrind are True.
after_loop()[source]

Actions to perform after run_loop has finished. Mainly checking if there was an error and then handling it.

before_loop()[source]

Actions before loop.

before_start()[source]

Actions to perform before the run starts.

enqueue_output_loop()[source]

Keep passing lines to queue.

graceful_stop()[source]

Gracefully stop the driver.

kill_process()[source]

Kill the process running the model, checking return code.

model_process_complete

bool – Has the process finished or not. Returns True if the process has not started.

run_loop()[source]

Loop to check if model is still running and forward output.

set_env()[source]
wait_process(timeout=None, key=None, key_suffix=None)[source]

Wait for some amount of time for the process to finish.

Parameters:
  • timeout (float, optional) – Time (in seconds) that should be waited. Defaults to None and is infinite.
  • key (str, optional) – Key that should be used to register the timeout. Defaults to None and set based on the stack trace.
Returns:

True if the process completed. False otherwise.

Return type:

bool

cis_interface.drivers.OutputCommDriver module

class cis_interface.drivers.OutputCommDriver.OutputCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for output communication.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.OutputDriver module

class cis_interface.drivers.OutputDriver.OutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Driver for sending output to another model’s comm via a local comm.

Parameters:
  • name (str) – The name of the local message comm that the driver should connect to.
  • args (str) – The name of the other message comme that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.PandasFileInputDriver module

class cis_interface.drivers.PandasFileInputDriver.PandasFileInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileInputDriver.FileInputDriver

Class to handle input from a Pandas csv file.

Parameters:
  • name (str) – Name of the input queue to send messages to.
  • args (str or dict) – Path to the file that messages should be read from.
  • delimiter (str, optional) – String that should be used to separate columns. Defaults to ‘t’.
  • **kwargs – Additional keyword arguments are passed to parent class.
update_serializer(msg)[source]

Update the serializer for the output comm based on input.

cis_interface.drivers.PandasFileOutputDriver module

class cis_interface.drivers.PandasFileOutputDriver.PandasFileOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileOutputDriver.FileOutputDriver

Class to handle output of received messages to a Pandas csv file.

Parameters:
  • name (str) – Name of the output queue to receive messages from.
  • args (str or dict) – Path to the file that messages should be written to.
  • delimiter (str, optional) – String that should be used to separate columns. Defaults to ‘t’.
  • **kwargs – Additional keyword arguments are passed to parent class.
update_serializer(msg)[source]

Update the serializer for the output comm based on input.

cis_interface.drivers.PickleFileInputDriver module

class cis_interface.drivers.PickleFileInputDriver.PickleFileInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileInputDriver.FileInputDriver

Class that sends messages read from a file.

Parameters:
  • name (str) – Name of the queue that messages should be sent to.
  • args (str) – Path to the file that messages should be read from.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.PickleFileOutputDriver module

class cis_interface.drivers.PickleFileOutputDriver.PickleFileOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.FileOutputDriver.FileOutputDriver

Class that writes received messages to a file.

Parameters:
  • name (str) – Name of the queue that messages should be sent to.
  • args (str) – Path to the file that messages should be read from.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.PythonModelDriver module

class cis_interface.drivers.PythonModelDriver.PythonModelDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.ModelDriver.ModelDriver

Class for running Python models.

Parameters:
  • name (str) – Driver name.
  • args (str or list) – Argument(s) for running the model on the command after the call to python.
  • **kwargs – Additional keyword arguments are passed to parent class’s __init__ method.

cis_interface.drivers.RMQAsyncClientDriver module

class cis_interface.drivers.RMQAsyncClientDriver.RMQAsyncClientDriver(*args, **kwargs)[source]

Bases: cis_interface.drivers.ClientDriver.ClientDriver

Class for handling an RMQAsync client.

Parameters:
  • *args – Arguments are passed to parent class.
  • **kwargs – Keyword arguments are passed to parent class.

cis_interface.drivers.RMQAsyncCommDriver module

class cis_interface.drivers.RMQAsyncCommDriver.RMQAsyncCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for communication via RMQAsync queues.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.RMQAsyncServerDriver module

class cis_interface.drivers.RMQAsyncServerDriver.RMQAsyncServerDriver(*args, **kwargs)[source]

Bases: cis_interface.drivers.ServerDriver.ServerDriver

Class for handling an RMQAsync server.

Parameters:
  • *args – Arguments are passed to parent class.
  • **kwargs – Keyword arguments are passed to parent class.

cis_interface.drivers.RMQClientDriver module

class cis_interface.drivers.RMQClientDriver.RMQClientDriver(*args, **kwargs)[source]

Bases: cis_interface.drivers.ClientDriver.ClientDriver

Class for handling an RMQ client.

Parameters:
  • *args – Arguments are passed to parent class.
  • **kwargs – Keyword arguments are passed to parent class.

cis_interface.drivers.RMQCommDriver module

class cis_interface.drivers.RMQCommDriver.RMQCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for communication via RMQ queues.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.
-

cis_interface.drivers.RMQInputDriver module

Module for receiving input from a RabbitMQ server.

class cis_interface.drivers.RMQInputDriver.RMQInputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.InputDriver.InputDriver

Driver for receiving input from a RabbitMQ server via a local comm.

Parameters:
  • name (str) – The name of the local message queue that the driver should connect to.
  • args (str) – The name of the RabbitMQ message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.RMQOutputDriver module

Module for sending output to a RabbitMQ server.

class cis_interface.drivers.RMQOutputDriver.RMQOutputDriver(name, args, **kwargs)[source]

Bases: cis_interface.drivers.OutputDriver.OutputDriver

Driver for sending output to a RabbitMQ server.

Parameters:
  • name (str) – The name of the local message queue that the driver should connect to.
  • args (str) – The name of the RabbitMQ message queue that the driver connect to.
  • **kwargs – Additional keyword arguments are passed to parent class.

cis_interface.drivers.RMQServerDriver module

class cis_interface.drivers.RMQServerDriver.RMQServerDriver(*args, **kwargs)[source]

Bases: cis_interface.drivers.ServerDriver.ServerDriver

Class for handling an RMQ server.

Parameters:
  • *args – Arguments are passed to parent class.
  • **kwargs – Keyword arguments are passed to parent class.

cis_interface.drivers.RPCCommDriver module

class cis_interface.drivers.RPCCommDriver.RPCCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for communication via RPC.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.

cis_interface.drivers.RPCDriver module

class cis_interface.drivers.RPCDriver.RPCDriver(name, args=None, **kwargs)[source]

Bases: cis_interface.drivers.Driver.Driver

Base class for any driver that requires to access to input & output queues for RPC type functionality.

Parameters:
  • name (str) – The name of the message queue set that the driver should connect to. (name + “_IN” and name + “_OUT”)
  • **kwargs – Additional keyword arguments are passed to parent class.
icomm

cis_interface.drivers.CommDriver – Driver for the input message queue.

ocomm

cis_interface.drivers.CommDriver – Driver for the output message ueue.

cleanup()[source]

Perform cleanup for IPC drivers.

close_comms()[source]

Close the IPC comms.

comms_open

bool – True if both input and output comms open.

do_terminate()[source]

Terminate input/output comm drivers.

graceful_stop()[source]

Allow the IPC comms to terminate gracefully.

is_valid

bool – True if both comms are open and parent class is valid.

n_msg_in

int – The number of messages in the input comm.

n_msg_out

int – The number of messages in the output comm.

on_model_exit()[source]

Actions to perform when the associated model driver is finished.

printStatus()[source]

Print information on the status of the driver.

recv(*args, **kwargs)[source]

Receive message smaller than maxMsgSize from the input comm.

Parameters:
  • *args – Arguments are passed to input comm recv method.
  • **kwargs – Keyword arguments are passed to input comm recv method.
Returns:

Success or failure of recv and the received

message.

Return type:

tuple (bool, str)

recv_nolimit(*args, **kwargs)[source]

Receive message larger than maxMsgSize from the output comm.

Parameters:
  • *args – Arguments are passed to input comm recv_nolimit method.
  • **kwargs – Keyword arguments are passed to input comm recv_nolimit method.
Returns:

Success or failure of recv and the received

message.

Return type:

tuple (bool, str)

send(data, *args, **kwargs)[source]

Send message smaller than maxMsgSize to the output comm.

Parameters:
  • data (str) – Message to be sent.
  • *args – Arguments are passed to output comm send method.
  • **kwargs – Keyword arguments are passed to output comm send method.
Returns:

Succes or failure of send.

Return type:

bool

send_nolimit(data, *args, **kwargs)[source]

Send message larger than maxMsgSize to the input comm.

Parameters:
  • data (str) – Message to be sent.
  • *args – Arguments are passed to output comm send_nolimit method.
  • **kwargs – Keyword arguments are passed to output comm send_nolimit method.
Returns:

Succes or failure of send.

Return type:

bool

start()[source]

Run the input/output comm drivers.

cis_interface.drivers.ServerDriver module

class cis_interface.drivers.ServerDriver.ServerDriver(model_request_name, request_name=None, comm=None, comm_address=None, **kwargs)[source]

Bases: cis_interface.drivers.ServerRequestDriver.ServerRequestDriver

Alias for ServerRequestDriver.

cis_interface.drivers.ServerRequestDriver module

class cis_interface.drivers.ServerRequestDriver.ServerRequestDriver(model_request_name, request_name=None, comm=None, comm_address=None, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class for handling server side RPC type communication.

Parameters:
  • model_request_name (str) – The name of the channel used by the server model to send requests.
  • request_name (str, optional) – The name of the channel that should be used to receive requests from the client request driver. Defaults to model_request_name + ‘_SERVER’ if not set.
  • comm (str, optional) – The comm class that should be used to communicate with the client request driver. Defaults to tools.get_default_comm().
  • comm_address (str, optional) – Address for the client request driver. Defaults to None and a new address is generated.
  • **kwargs – Additional keyword arguments are passed to parent class.
comm

str – The comm class that should be used to communicate with the server driver. Defaults to tools.get_default_comm().

comm_address

str – Address for the client request driver.

response_drivers

list – Response drivers created for each request.

nclients

int – Number of clients signed on.

close_comm()[source]

Close response drivers.

close_response_drivers()[source]

Close response drivers.

last_header

dict – Information contained in the header of the last message received from the client model.

on_client_exit()[source]

Close input comm to stop the loop.

on_eof()[source]

On EOF, decrement number of clients. Only send EOF if the number of clients drops to 0.

on_message(msg)[source]

Process a message checking to see if it is a client signing on.

Parameters:msg (bytes, str) – Message to be processed.
Returns:Processed message.
Return type:bytes, str
printStatus(*args, **kwargs)[source]

Also print response drivers.

request_id

str – Unique ID for the last message.

request_name

str – The name of the channel used to receive requests from the client request driver.

response_address

str – The address of the channel used by the server response driver to send responses.

send_eof()[source]

Send EOF message.

Returns:Success or failure of send.
Return type:bool
send_message(*args, **kwargs)[source]

Send a single message.

Parameters:
  • *args – Arguments are passed to parent class send_message.
  • **kwargs – Keyword arguments are passed to parent class send_message.
Returns:

Success or failure of send.

Return type:

bool

cis_interface.drivers.ServerResponseDriver module

class cis_interface.drivers.ServerResponseDriver.ServerResponseDriver(response_address, comm=None, msg_id=None, request_name=None, **kwargs)[source]

Bases: cis_interface.drivers.ConnectionDriver.ConnectionDriver

Class for handling server side RPC type communication.

Parameters:
  • response_address (str) – The address of the channel used to send responses to the client response driver.
  • comm (str, optional) – The comm class that should be used to communicate with the server resposne driver. Defaults to tools.get_default_comm().
  • msg_id (str, optional) – ID associate with the request message this driver was created to respond to. Defaults to new unique ID.
  • **kwargs – Additional keyword arguments are passed to parent class.
comm

str – The comm class that should be used to communicate with the server driver. Defaults to tools.get_default_comm().

msg_id

str – ID associate with the request message this driver was created to respond to.

model_response_address

str – The address of the channel used by the server model to send responses.

model_response_name

str – The name of the channel used by the server model to send responses.

response_address

str – The address of the channel used to send responses to the client response driver.

cis_interface.drivers.ZMQCommDriver module

class cis_interface.drivers.ZMQCommDriver.ZMQCommDriver(name, **kwargs)[source]

Bases: cis_interface.drivers.CommDriver.CommDriver

Driver for communication via ZMQ sockets.

Parameters:
  • name (str) – The name of the message queue that the driver should connect to.
  • **kwargs – Additional keyword arguments are passed to the parent class.
-

Module contents

IO and Model drivers.

cis_interface.drivers.import_driver(driver=None)[source]

Dynamically import a driver based on a string.

Parameters:driver (str) – Name of the driver that should be imported.
cis_interface.drivers.create_driver(driver=None, name=None, args=None, **kwargs)[source]

Dynamically create a driver based on a string and other driver properties.

Parameters:
  • driver (str) – Name of the driver that should be created.
  • name (str) – Name to give the driver.
  • args (object, optional) – Second argument for drivers which take a minimum of two arguments. If None, the driver is assumed to take a minimum of one argument. Defaults to None.
  • **kwargs – Additional keyword arguments are passed to the driver class.
Returns:

Instance of the requested driver.

Return type:

object