cis_interface.communication package¶
Submodules¶
cis_interface.communication.AsciiFileComm module¶
-
class
cis_interface.communication.AsciiFileComm.
AsciiFileComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.FileComm.FileComm
Class for handling I/O from/to a file on disk.
Parameters: -
comment
¶ str – String indicating a comment.
-
cis_interface.communication.AsciiMapComm module¶
-
class
cis_interface.communication.AsciiMapComm.
AsciiMapComm
(name, **kwargs)[source]¶ Bases:
cis_interface.communication.FileComm.FileComm
Class for handling I/O from/to a ASCII map on disk.
Parameters: - name (str) – The environment variable where file path is stored.
- **kwargs – Additional keywords arguments are passed to parent class.
cis_interface.communication.AsciiTableComm module¶
-
class
cis_interface.communication.AsciiTableComm.
AsciiTableComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.AsciiFileComm.AsciiFileComm
Class for handling I/O from/to a file on disk.
Parameters: - name (str) – The environment variable where communication address is stored.
- delimiter (str, optional) – String that should be used to separate columns. If not provided and format_str is not set prior to I/O, this defaults to whitespace.
- use_astropy (bool, optional) – If True and the astropy package is installed, it will be used to read/write the table. Defaults to False.
- **kwargs – Additional keywords arguments are passed to parent class.
-
advance_in_series
(*args, **kwargs)[source]¶ Advance to a certain file in a series.
Parameters: index (int, optional) – Index of file in the series that should be moved to. Defaults to None and call will advance to the next file in the series. Returns: True if the file was advanced in the series, False otherwise. Return type: bool
-
change_position
(file_pos, series_index=None, header_was_read=None, header_was_written=None)[source]¶ Change the position in the file/series.
Parameters: - file_pos (int) – Position that should be moved to in the file.
- series_index (int, optinal) – Index of the file in the series that should be moved to. Defaults to None and will be set to the current series index.
- header_was_read (bool, optional) – Status of if header has been read or not. Defaults to None and will be set to the current value.
- header_was_written (bool, optional) – Status of if header has been written or not. Defaults to None and will be set to the current value.
cis_interface.communication.AsyncComm module¶
-
class
cis_interface.communication.AsyncComm.
AsyncComm
(name, dont_backlog=False, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for handling asynchronous I/O.
Parameters: -
dont_backlog
¶ bool – If True, the backlog will not be started and all messages will be sent/received directly to/from the comm.
-
backlog_send_ready
¶ threading.Event – Event set when there is a message in the send backlog.
-
backlog_recv_ready
¶ threading.Event – Event set when there is a message in the recv backlog.
-
add_backlog_recv
(msg)[source]¶ Add a message to the backlog of received messages.
Parameters: msg (str) – Received message that should be backlogged.
-
add_backlog_send
(msg, **kwargs)[source]¶ Add a message to the backlog of messages to be sent.
Parameters: - msg (str) – Message that should be backlogged for sending.
- **kwargs – Additional keyword arguments are added along with the message.
-
backlog_recv
¶ list – Messages that have been received.
-
backlog_send
¶ list – Messages that should be sent.
-
backlog_thread
¶ tools.CisThread – Thread that will handle sinding or receiving backlogged messages.
-
is_confirmed_recv
¶ bool – True if all received messages have been confirmed.
-
is_confirmed_send
¶ bool – True if all sent messages have been confirmed.
-
is_open
¶ bool – True if the backlog is open.
-
is_open_backlog
¶ bool – True if the backlog thread is running.
-
is_open_direct
¶ bool – True if the direct comm is not None.
-
n_msg_backlog
¶ int – Number of messages in the backlog.
-
n_msg_backlog_recv
¶ int – Number of messages in the receive backlog.
-
n_msg_backlog_send
¶ int – Number of messages in the send backlog.
-
n_msg_direct
¶ int – Number of messages currently being routed.
-
n_msg_direct_recv
¶ int – Number of messages currently being routed in recv.
-
n_msg_direct_send
¶ int – Number of messages currently being routed in send.
-
n_msg_recv
¶ int – Number of messages in the receive backlog.
-
n_msg_recv_drain
¶ int – Number of messages in the receive backlog and direct comm.
-
n_msg_send
¶ int – Number of messages in the send backlog.
-
n_msg_send_drain
¶ int – Number of messages in the send backlog and direct comm.
-
pop_backlog_recv
()[source]¶ Pop a message from the front of the recv backlog.
Returns: First backlogged recv message. Return type: str
-
cis_interface.communication.ClientComm module¶
-
class
cis_interface.communication.ClientComm.
ClientComm
(name, request_comm=None, response_kwargs=None, dont_open=False, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for handling Client side communication.
Parameters: - name (str) – The environment variable where communication address is stored.
- request_comm (str, optional) – Comm class that should be used for the request comm. Defaults to None.
- response_kwargs (dict, optional) – Keyword arguments for the response comm. Defaults to empty dict.
- **kwargs – Additional keywords arguments are passed to the output comm.
-
response_kwargs
¶ dict – Keyword arguments for the response comm.
-
icomm
¶ dict – Response comms keyed to the ID of the associated request.
-
icomm_order
¶ list – Response comm keys in the order or the requests.
-
ocomm
¶ Comm – Request comm.
-
call
(*args, **kwargs)[source]¶ Do RPC call. The request message is sent to the output comm and the response is received from the input comm.
Parameters: - *args – Arguments are passed to output comm send method.
- **kwargs – Keyword arguments are passed to output comm send method
Returns: Output from input comm recv method.
Return type: obj
-
is_closed
¶ bool – True if the connection is closed.
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_send
¶ int – The number of messages in the connection.
-
n_msg_send_drain
¶ int – The number of outgoing messages in the connection to drain.
-
classmethod
new_comm_kwargs
(name, request_comm=None, **kwargs)[source]¶ Initialize communication with new comms.
Parameters:
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
opp_comms
¶ dict – Name/address pairs for opposite comms.
-
recv
(*args, **kwargs)[source]¶ Receive a message from the input comm and open a new response comm for output using address from the header.
Parameters: - *args – Arguments are passed to input comm recv method.
- **kwargs – Keyword arguments are passed to input comm recv method.
Returns: Output from input comm recv method.
Return type: obj
-
send
(*args, **kwargs)[source]¶ Create a response comm and then send a message to the output comm with the response address in the header.
Parameters: - *args – Arguments are passed to output comm send method.
- **kwargs – Keyword arguments are passed to output comm send method.
Returns: Output from output comm send method.
Return type: obj
cis_interface.communication.CommBase module¶
-
class
cis_interface.communication.CommBase.
CommBase
(name, address=None, direction='send', dont_open=False, is_interface=False, recv_timeout=0.0, close_on_eof_recv=True, close_on_eof_send=False, single_use=False, reverse_names=False, no_suffix=False, is_client=False, is_response_client=False, is_server=False, is_response_server=False, recv_converter=None, send_converter=None, comm=None, matlab=False, **kwargs)[source]¶ Bases:
cis_interface.tools.CisClass
Class for handling I/O.
Parameters: - name (str) – The environment variable where communication address is stored.
- address (str, optional) – Communication info. Default to None and address is taken from the environment variable.
- direction (str, optional) – The direction that messages should flow through the connection. ‘send’ if the connection will send messages, ‘recv’ if the connecton will receive messages. Defaults to ‘send’.
- serializer (DefaultSerialize, optional) – Class with serialize and deserialize methods that should be used to process sent and received messages. Defaults to None and is constructed using provided ‘serializer_kwargs’.
- serializer_kwargs (dict, optional) – Keyword arguments that should be passed to DefaultSerialize to create serializer. Defaults to {}.
- format_str (str, optional) – String that should be used to format/parse messages. Default to None.
- dont_open (bool, optional) – If True, the connection will not be opened. Defaults to False.
- is_interface (bool, optional) – Set to True if this comm is a Python interface binding. Defaults to False.
- recv_timeout (float, optional) – Time that should be waited for an incoming message before returning None. Defaults to 0 (no wait). A value of False indicates that recv should block.
- close_on_eof_recv (bool, optional) – If True, the comm will be closed when it receives an end-of-file messages. Otherwise, it will remain open. Defaults to True.
- close_on_eof_send (bool, optional) – If True, the comm will be closed after it sends an end-of-file messages. Otherwise, it will remain open. Defaults to False.
- single_use (bool, optional) – If True, the comm will only be used to send/recv a single message. Defaults to False.
- reverse_names (bool, optional) – If True, the suffix added to the comm with be reversed. Defaults to False.
- no_suffix (bool, optional) – If True, no directional suffix will be added to the comm name. Defaults to False.
- is_client (bool, optional) – If True, the comm is one of many potential clients that will be sending messages to one or more servers. Defaults to False.
- is_response_client (bool, optional) – If True, the comm is a client-side response comm. Defaults to False.
- is_server (bool, optional) – If True, the commis one of many potential servers that will be receiving messages from one or more clients. Defaults to False.
- is_response_server (bool, optional) – If True, the comm is a server-side response comm. Defaults to False.
- recv_converter (func, optional) – Converter that should be used on received objects. Defaults to None.
- send_converter (func, optional) – Converter that should be used on sent objects. Defaults to None.
- comm (str, optional) – The comm that should be created. This only serves as a check that the correct class is being created. Defaults to None.
- matlab (bool, optional) – True if the comm will be accessed by Matlab code. Defaults to False.
- **kwargs – Additional keywords arguments are passed to parent class.
-
name
¶ str – The environment variable where communication address is stored.
-
address
¶ str – Communication info.
-
direction
¶ str – The direction that messages should flow through the connection.
-
serializer
¶ DefaultSerialize – Object that will be used to serialize/deserialize messages to/from python objects.
-
is_interface
¶ bool – True if this comm is a Python interface binding.
-
recv_timeout
¶ float – Time that should be waited for an incoming message before returning None.
-
close_on_eof_recv
¶ bool – If True, the comm will be closed when it receives an end-of-file messages. Otherwise, it will remain open.
-
close_on_eof_send
¶ bool – If True, the comm will be closed after it sends an end-of-file messages. Otherwise, it will remain open.
-
single_use
¶ bool – If True, the comm will only be used to send/recv a single message.
-
is_client
¶ bool – If True, the comm is one of many potential clients that will be sending messages to one or more servers.
-
is_response_client
¶ bool – If True, the comm is a client-side response comm.
-
is_server
¶ bool – If True, the comm is one of many potential servers that will be receiving messages from one or more clients.
-
is_response_server
¶ bool – If True, the comm is a server-side response comm.
-
is_file
¶ bool – True if the comm accesses a file.
-
recv_converter
¶ func – Converter that should be used on received objects.
-
send_converter
¶ func – Converter that should be used on sent objects.
-
matlab
¶ bool – True if the comm will be accessed by Matlab code.
Raises: RuntimeError
– If the comm class is not installed.RuntimeError
– If there is not an environment variable with the specified name.ValueError
– If directions is not ‘send’ or ‘recv’.
-
add_work_comm
(comm)[source]¶ Add work comm to dict.
Parameters: comm (Comm) – Comm that should be added. Raises: KeyError
– If there is already a comm associated with the key.
-
chunk_message
(msg)[source]¶ Yield chunks of message of size maxMsgSize
Parameters: msg (str, bytes) – Raw message bytes to be chunked. Returns: Chunks of message. Return type: str
-
close_in_thread
(no_wait=False, timeout=None)[source]¶ In a new thread, close the comm when it is empty.
Parameters:
-
comm_class
¶ str – Name of communication class.
-
create_work_comm
(work_comm_name=None, **kwargs)[source]¶ Create a temporary work comm.
Parameters: - work_comm_name (str, optional) – Name that should be used for the work comm. If not provided, one is created from the header id and the comm class.
- **kwargs – Keyword arguments for new_comm that should override work_comm_kwargs.
Returns: Work comm.
Return type: Comm
-
create_work_comm_kwargs
¶ dict – Keyword arguments for a new work comm.
-
drain_messages
(direction=None, timeout=None, variable=None)[source]¶ Sleep while waiting for messages to be drained.
-
empty_msg
¶ str – Empty message.
-
empty_obj_recv
¶ obj – Empty message object.
-
eof_msg
¶ str – Message indicating EOF.
-
get_work_comm
(header, **kwargs)[source]¶ Get temporary work comm, creating as necessary.
Parameters: - header (dict) – Information that will be sent in the message header to the work comm.
- **kwargs – Additional keyword arguments are passed to header2workcomm.
Returns: Work comm.
Return type: Comm
-
get_work_comm_kwargs
¶ dict – Keyword arguments for an existing work comm.
-
header2workcomm
(header, work_comm_name=None, **kwargs)[source]¶ Get a work comm based on header info.
Parameters: - header (dict) – Information that will be sent in the message header to the work comm.
- work_comm_name (str, optional) – Name that should be used for the work comm. If not provided, one is created from the header id and the comm class.
- **kwargs – Additional keyword arguments are added to the returned dictionary.
Returns: Work comm.
Return type:
-
is_closed
¶ bool – True if the connection is closed.
-
is_confirmed
¶ bool – True if all messages have been confirmed.
-
is_confirmed_recv
¶ bool – True if all received messages have been confirmed.
-
is_confirmed_send
¶ bool – True if all sent messages have been confirmed.
-
is_empty_recv
(msg)[source]¶ Check if a received message object is empty.
Parameters: msg (obj) – Message object. Returns: True if the object is empty, False otherwise. Return type: bool
-
is_eof
(msg)[source]¶ Determine if a message is an EOF.
Parameters: msg (obj) – Message object to be tested. Returns: True if the message indicates an EOF, False otherwise. Return type: bool
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg
¶ int – The number of messages in the connection.
-
n_msg_recv
¶ int – The number of incoming messages in the connection.
-
n_msg_recv_drain
¶ int – The number of incoming messages in the connection to drain.
-
n_msg_send
¶ int – The number of outgoing messages in the connection.
-
n_msg_send_drain
¶ int – The number of outgoing messages in the connection to drain.
-
new_server
(srv_address)[source]¶ Create a new server.
Parameters: srv_address (str) – Address of server comm.
-
on_recv
(s_msg, second_pass=False)[source]¶ Process raw received message including handling deserializing message and handling EOF.
Parameters: Returns: - Success or failure, processed message, and
header information.
Return type:
-
on_recv_eof
()[source]¶ Actions to perform when EOF received.
Returns: Flag that should be returned for EOF. Return type: bool
-
on_send
(msg, header_kwargs=None)[source]¶ Process message to be sent including handling serializing message and handling EOF.
Parameters: - msg (obj) – Message to be sent
- header_kwargs (dict, optional) – Keyword arguments that should be added to the header.
Returns: - Truth of if message should be sent, raw
bytes message to send, and header info contained in the message.
Return type:
-
on_send_eof
()[source]¶ Actions to perform when EOF being sent.
Returns: True if EOF message should be sent, False otherwise. Return type: bool
-
opp_address
¶ str – Address for opposite comm.
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
opp_comms
¶ dict – Name/address pairs for opposite comms.
-
recv
(*args, **kwargs)[source]¶ Receive a message.
Parameters: - *args – All arguments are passed to comm _recv method.
- **kwargs – All keywords arguments are passed to comm _recv method.
Returns: - Success or failure of receive and received
message.
Return type:
-
recv_dict
(*args, **kwargs)[source]¶ Return a received message as a dictionary of fields. If there are not any fields specified, the fields will have the form ‘f0’, ‘f1’, ‘f2’, …
Parameters: - *args – Arguments are passed to recv.
- **kwargs – Keyword arguments are passed to recv.
Returns: - Success/failure of receive and a dictionar of
message fields.
Return type: Raises:
-
recv_multipart
(*args, **kwargs)[source]¶ Receive a multipart message. If a message is received without a header, it assumed to be complete. Otherwise, the message is received in parts from a worker comm initialized by the sender.
Parameters: - *args – All arguments are passed to comm _recv method.
- **kwargs – All keywords arguments are passed to comm _recv method.
Returns: - Success or failure of receive and received
message.
Return type:
-
remove_work_comm
(key, in_thread=False, linger=False)[source]¶ Close and remove a work comm.
Parameters:
-
send
(*args, **kwargs)[source]¶ Send a message.
Parameters: - *args – All arguments are assumed to be part of the message.
- **kwargs – All keywords arguments are passed to comm _send method.
Returns: Success or failure of send.
Return type:
-
send_dict
(args_dict, field_order=None, **kwargs)[source]¶ Send a message with fields specified in the input dictionary.
Parameters: - args_dict (dict) – Dictionary with fields specifying output fields.
- field_order (list, optional) – List of fields in the order they should be passed to send. If not provided, the fields from the serializer are used. If the serializer dosn’t have field names an error will be raised.
- **kwargs – Additiona keyword arguments are passed to send.
Returns: Success/failure of send.
Return type: Raises: RuntimeError
– If the field order can not be determined.
-
send_eof
(*args, **kwargs)[source]¶ Send the EOF message as a short message.
Parameters: - *args – All arguments are passed to comm send.
- **kwargs – All keywords arguments are passed to comm send.
Returns: Success or failure of send.
Return type:
-
send_multipart
(msg, header_kwargs=None, **kwargs)[source]¶ Send a multipart message. If the message is smaller than maxMsgSize, it is sent using _send, otherwise it is sent to a worker comm using _send_multipart_worker.
Parameters: - msg (obj) – Message to be sent.
- header_kwargs (dict, optional) – Keyword arguments that should be added to the header.
- **kwargs – Additional keyword arguments are passed to _send or _send_multipart_worker.
Returns: Success or failure of send.
Return type:
-
server_exists
(srv_address)[source]¶ Determine if a server exists.
Parameters: srv_address (str) – Address of server comm. Returns: - True if a server with the provided address exists, False
- otherwise.
Return type: bool
-
class
cis_interface.communication.CommBase.
CommServer
(srv_address, cli_address=None, **kwargs)[source]¶ Bases:
cis_interface.tools.CisThreadLoop
Basic server object to keep track of clients.
-
cli_count
¶ int – Number of clients that have connected to this server.
-
-
class
cis_interface.communication.CommBase.
CommThreadLoop
(comm, name=None, suffix='CommThread', **kwargs)[source]¶ Bases:
cis_interface.tools.CisThreadLoop
Thread loop for comms to ensure cleanup.
Parameters: - comm (CommBase) – Comm class that thread is for.
- name (str, optional) – Name for the thread. If not provided, one is created by combining the comm name and the provided suffix.
- suffix (str, optional) – Suffix that should be added to comm name to name the thread. Defaults to ‘CommThread’.
- **kwargs – Additional keyword arguments are passed to the parent class.
-
comm
¶ CommBase – Comm class that thread is for.
-
cis_interface.communication.CommBase.
cleanup_comms
(comm_class, close_func=None)[source]¶ Clean up comms of a certain type.
Parameters: comm_class (str) – Comm class that should be cleaned up. Returns: Number of comms closed. Return type: int
-
cis_interface.communication.CommBase.
get_comm_registry
(comm_class)[source]¶ Get the comm registry for a comm class.
Parameters: comm_class (str) – Comm class to get registry for. Returns: Dictionary of registered comm objects. Return type: dict
-
cis_interface.communication.CommBase.
is_registered
(comm_class, key)[source]¶ Determine if a comm object has been registered under the specified key.
Parameters:
-
cis_interface.communication.CommBase.
register_comm
(comm_class, key, value)[source]¶ Add a comm object to the global registry.
Parameters:
cis_interface.communication.ErrorComm module¶
cis_interface.communication.FileComm module¶
-
class
cis_interface.communication.FileComm.
FileComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for handling I/O from/to a file on disk.
Parameters: - name (str) – The environment variable where communication address is stored.
- read_meth (str, optional) – Method that should be used to read data from the file. Defaults to ‘read’. Ignored if direction is ‘send’.
- append (bool, optional) – If True and writing, file is openned in append mode. Defaults to False.
- in_temp (bool, optional) – If True, the path will be considered relative to the platform temporary directory. Defaults to False.
- open_as_binary (bool, optional) – If True, the file is opened in binary mode. Defaults to True.
- newline (str, optional) – String indicating a new line. Defaults to serialize._default_newline.
- is_series (bool, optional) – If True, input/output will be done to a series of files. If reading, each file will be processed until the end is reached. If writing, each output will be to a new file in the series. The addressed is assumed to contain a format for the index of the file. Defaults to False.
- **kwargs – Additional keywords arguments are passed to parent class.
-
fd
¶ file – File that should be read/written.
-
read_meth
¶ str – Method that should be used to read data from the file.
-
append
¶ bool – If True and writing, file is openned in append mode.
-
in_temp
¶ bool – If True, the path will be considered relative to the platform temporary directory.
-
open_as_binary
¶ bool – If True, the file is opened in binary mode.
-
newline
¶ str – String indicating a new line.
-
is_series
¶ bool – If True, input/output will be done to a series of files. If reading, each file will be processed until the end is reached. If writing, each output will be to a new file in the series.
-
platform_newline
¶ str – String indicating a newline on the current platform.
Raises: ValueError
– If the read_meth is not one of the supported values.-
advance_in_file
(file_pos)[source]¶ Advance to a certain position in the current file.
Parameters: file_pos (int) – Position that should be moved to in the current. file.
-
advance_in_series
(series_index=None)[source]¶ Advance to a certain file in a series.
Parameters: series_index (int, optional) – Index of file in the series that should be moved to. Defaults to None and call will advance to the next file in the series. Returns: True if the file was advanced in the series, False otherwise. Return type: bool
-
change_position
(file_pos, series_index=None)[source]¶ Change the position in the file/series.
Parameters:
-
current_address
¶ str – Address of file currently being used.
-
fd
Associated file identifier.
-
get_series_address
(index=None)[source]¶ Get the address of a file in the series.
Parameters: index (int, optional) – Index in series to get address for. Defaults to None and the current index is used. Returns: Address for the file in the series. Return type: str
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_recv
¶ int – The number of messages in the file.
-
on_send_eof
()[source]¶ Close file when EOF to be sent.
Returns: False so that message not sent. Return type: bool
-
open_mode
¶ str – Mode that should be used to open the file.
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
registry_key
¶ str – String used to register the socket.
-
remaining_bytes
¶ int – Remaining bytes in the file.
cis_interface.communication.ForkComm module¶
-
class
cis_interface.communication.ForkComm.
ForkComm
(name, comm=None, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for receiving/sending messages from/to multiple comms.
Parameters: -
comm_list
¶ list – Comms included in this fork.
-
curr_comm_index
¶ int – Index comm that next receive will be from.
-
curr_comm
¶ CommBase – Current comm.
-
is_confirmed_recv
¶ bool – True if all received messages have been confirmed.
-
is_confirmed_send
¶ bool – True if all sent messages have been confirmed.
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_recv
¶ int – The number of incoming messages in the connection.
-
n_msg_recv_drain
¶ int – The number of incoming messages in the connection to drain.
-
n_msg_send
¶ int – The number of outgoing messages in the connection.
-
n_msg_send_drain
¶ int – The number of outgoing messages in the connection to drain.
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
opp_comms
¶ dict – Name/address pairs for opposite comms.
-
cis_interface.communication.IPCComm module¶
-
class
cis_interface.communication.IPCComm.
IPCComm
(name, dont_backlog=False, **kwargs)[source]¶ Bases:
cis_interface.communication.AsyncComm.AsyncComm
Class for handling I/O via IPC message queues.
-
q
¶ sysv_ipc.MessageQueue
– Message queue.
-
is_open_direct
¶ bool – True if the queue is not None.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_direct_recv
¶ int – Number of messages in the queue to recv.
-
n_msg_direct_send
¶ int – Number of messages in the queue to send.
-
-
class
cis_interface.communication.IPCComm.
IPCServer
(srv_address, cli_address=None, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommServer
IPC server object for cleaning up server queue.
-
cis_interface.communication.IPCComm.
get_queue
(qid=None)[source]¶ Create or return a sysv_ipc.MessageQueue and register it.
Parameters: qid (int, optional) – If provided, ID for existing queue that should be returned. Defaults to None and a new queue is returned. Returns: Message queue. Return type: sysv_ipc.MessageQueue
-
cis_interface.communication.IPCComm.
ipc_queues
()[source]¶ Get a list of active IPC queues.
Returns: List of IPC queues. Return type: list
-
cis_interface.communication.IPCComm.
ipcrm
(options=[])[source]¶ Remove IPC constructs using the ipcrm command.
Parameters: options (list) – List of flags that should be used. Defaults to an empty list.
-
cis_interface.communication.IPCComm.
ipcrm_queues
(queue_keys=None)[source]¶ Delete existing IPC queues.
Parameters: queue_keys (list, str, optional) – A list of keys for queues that should be removed. Defaults to all existing queues.
cis_interface.communication.ObjFileComm module¶
-
class
cis_interface.communication.ObjFileComm.
ObjFileComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.PlyFileComm.PlyFileComm
Class for handling I/O from/to a .obj file on disk.
Parameters: - name (str) – The environment variable where communication address is stored.
- **kwargs – Additional keywords arguments are passed to parent class.
cis_interface.communication.PandasFileComm module¶
-
class
cis_interface.communication.PandasFileComm.
PandasFileComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.FileComm.FileComm
Class for handling I/O from/to a pandas csv file on disk.
Parameters: -
send_dict
(args_dict, field_order=None, **kwargs)[source]¶ Send a message with fields specified in the input dictionary.
Parameters: - args_dict (dict) – Dictionary with fields specifying output fields.
- field_order (list, optional) – List of fields in the order they should be passed to send. If not provided, the fields from the serializer are used. If the serializer dosn’t have field names an error will be raised.
- **kwargs – Additiona keyword arguments are passed to send.
Returns: Success/failure of send.
Return type: Raises: RuntimeError
– If the field order can not be determined.
-
cis_interface.communication.PickleFileComm module¶
-
class
cis_interface.communication.PickleFileComm.
PickleFileComm
(name, **kwargs)[source]¶ Bases:
cis_interface.communication.FileComm.FileComm
Class for handling I/O from/to a pickled file on disk.
Parameters: - name (str) – The environment variable where file path is stored.
- **kwargs – Additional keywords arguments are passed to parent class.
cis_interface.communication.PlyFileComm module¶
-
class
cis_interface.communication.PlyFileComm.
PlyFileComm
(*args, **kwargs)[source]¶ Bases:
cis_interface.communication.FileComm.FileComm
Class for handling I/O from/to a .ply file on disk.
Parameters: - name (str) – The environment variable where communication address is stored.
- **kwargs – Additional keywords arguments are passed to parent class.
cis_interface.communication.RMQAsyncComm module¶
-
class
cis_interface.communication.RMQAsyncComm.
RMQAsyncComm
(name, dont_backlog=False, **kwargs)[source]¶ Bases:
cis_interface.communication.RMQComm.RMQComm
Class for handling asynchronous RabbitMQ communications. It is not recommended to use this class as it can leave hanging threads if not closed propertly. The normal RMQComm will cover most use cases.
Parameters: -
times_connected
¶ int – Number of times that this connections has been established.
-
rmq_thread
¶ tools.CisThread – Thread used to run IO loop.
-
channel_open
¶ bool – True if connection ready for messages, False otherwise.
-
channel_stable
¶ bool – True if the connection ready for messages and not about to close. False otherwise.
-
n_msg_direct_recv
¶ int – Number of messages in the queue.
-
on_bindok
(unused_frame)[source]¶ Actions to perform once the queue is succesfully bound. Start consuming messages.
-
on_cancelok
(unused_frame)[source]¶ Actions to perform after succesfully cancelling consumption. Closes the channel.
-
on_channel_closed
(channel, reply_code, reply_text)[source]¶ Actions to perform when the channel is closed. Close the connection.
-
on_channel_open
(channel)[source]¶ Actions to perform after a channel is opened. Add the channel close callback and setup the exchange.
-
on_connection_closed
(connection, reply_code, reply_text)[source]¶ Actions that must be taken when the connection is closed. Set the channel to None. If the connection is meant to be closing, stop the IO loop. Otherwise, wait 5 seconds and try to reconnect.
-
on_connection_open
(connection)[source]¶ Actions that must be taken when the connection is opened. Add the close connection callback and open the RabbitMQ channel.
-
on_connection_open_error
(unused_connection)[source]¶ Actions that must be taken when the connection fails to open.
-
on_exchange_declareok
(unused_frame)[source]¶ Actions to perform once an exchange is succesfully declared. Set up the queue.
-
on_queue_declareok
(method_frame)[source]¶ Actions to perform once the queue is succesfully declared. Bind the queue.
-
rmq_lock
¶ Lock associated with RMQ ioloop thread.
-
cis_interface.communication.RMQComm module¶
-
class
cis_interface.communication.RMQComm.
RMQComm
(name, dont_backlog=False, **kwargs)[source]¶ Bases:
cis_interface.communication.AsyncComm.AsyncComm
Class for handling basic RabbitMQ communications.
-
connection
¶ pika.Connection
– RabbitMQ connection.
-
channel
¶ pika.Channel
– RabbitMQ channel.
Raises: RuntimeError
– If a connection cannot be established.-
create_work_comm_kwargs
¶ dict – Keyword arguments for a new work comm.
-
exchange
¶ str – AMQP exchange.
-
get_work_comm_kwargs
¶ dict – Keyword arguments for an existing work comm.
-
is_confirmed_recv
¶ bool – True if all received messages have been confirmed.
-
is_confirmed_send
¶ bool – True if all sent messages have been confirmed.
-
is_open_direct
¶ bool – True if the connection and channel are open.
-
n_msg_direct_recv
¶ int – Number of messages in the queue.
-
n_msg_direct_send
¶ int – Number of messages in the queue.
-
classmethod
new_comm_kwargs
(name, user=None, password=None, host=None, virtual_host=None, port=None, exchange=None, queue='', **kwargs)[source]¶ Initialize communication with new connection.
Parameters: - name (str) – Name of new connection.
- user (str, optional) – RabbitMQ server username. Defaults to config option ‘user’ in section ‘rmq’ if it exists and ‘guest’ if it does not.
- password (str, optional) – RabbitMQ server password. Defaults to config option ‘password’ in section ‘rmq’ if it exists and ‘guest’ if it does not.
- host (str, optional) – RabbitMQ server host. Defaults to config option ‘host’ in section ‘rmq’ if it exists and ‘localhost’ if it does not. If ‘localhost’, the output of socket.gethostname() is used.
- virtual_host (str, optional) – RabbitMQ server virtual host. Defaults to config option ‘vhost’ in section ‘rmq’ if it exists and ‘/’ if it does not.
- port (str, optional) – Port on host to use. Defaults to config option ‘port’ in section ‘rmq’ if it exists and ‘5672’ if it does not.
- exchange (str, optional) – RabbitMQ exchange. Defaults to config option ‘namespace’ in section ‘rmq’ if it exits and ‘’ if it does not.
- queue (str, optional) – Name of the queue that messages will be send to or received from. If an empty string, the queue will be a random string and exclusive to a receiving comm. Defaults to ‘’.
- **kwargs – Additional keywords arguments are returned as keyword arguments for the new comm.
Returns: Arguments and keyword arguments for new comm.
Return type:
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
queue
¶ str – AMQP queue.
-
url
¶ str – AMQP server address.
-
-
class
cis_interface.communication.RMQComm.
RMQServer
(srv_address, cli_address=None, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommServer
RMQ server object for cleaning up server connections.
-
cis_interface.communication.RMQComm.
check_rmq_server
(url=None, **kwargs)[source]¶ Check that connection to a RabbitMQ server is possible.
Parameters: - url (str, optional) – Address of RMQ server that includes all the necessary information. If this is provided, the remaining arguments are ignored. Defaults to None and the connection parameters are taken from the other arguments.
- username (str, optional) – RMQ server username. Defaults to config value.
- password (str, optional) – RMQ server password. Defaults to config value.
- host (str, optional) – RMQ hostname or IP Address to connect to. Defaults to config value.
- port (str, optional) – RMQ server TCP port to connect to. Defaults to config value.
- vhost (str, optional) – RMQ virtual host to use. Defaults to config value.
Returns: - True if connection to RabbitMQ server is possible, False
otherwise.
Return type:
cis_interface.communication.RPCComm module¶
-
class
cis_interface.communication.RPCComm.
RPCComm
(name, address=None, comm=None, icomm_kwargs=None, ocomm_kwargs=None, dont_open=False, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for handling RPC I/O.
Parameters: - name (str) – The environment variable where communication address is stored.
- address (str, optional) – Communication info. Default to None and address is taken from the environment variable.
- comm (str, optional) – The comm that should be created. This only serves as a check that the correct class is being created. Defaults to None.
- icomm_kwargs (dict, optional) – Keyword arguments for the input comm. Defaults to empty dict.
- ocomm_kwargs (dict, optional) – Keyword arguments for the output comm. Defaults to empty dict.
- dont_open (bool, optional) – If True, the connection will not be opened. Defaults to False.
- **kwargs – Additional keywords arguments are passed to parent class as well as the input and output comms.
-
icomm
¶ Comm – Input comm.
-
ocomm
¶ Comm – Output comm.
-
call
(*args, **kwargs)[source]¶ Do RPC call. The request message is sent to the output comm and the response is received from the input comm.
Parameters: - *args – Arguments are passed to output comm send method.
- **kwargs – Keyword arguments are passed to output comm send method
Returns: Output from input comm recv method.
Return type: obj
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_recv
¶ int – The number of messages in the input comm.
-
n_msg_send
¶ int – The number of messages in the output comm.
-
classmethod
new_comm_kwargs
(name, address=None, icomm_name=None, ocomm_name=None, icomm_comm=None, ocomm_comm=None, icomm_kwargs=None, ocomm_kwargs=None, **kwargs)[source]¶ Initialize communication with new comms.
Parameters: - name (str) – Name for new comm.
- address (str, optional) – Communication info. Default to None and address is taken from combination of input/output comm addresses that are provided or generated.
- icomm_name (str, optional) – Name for new input comm. Defaults to name + ‘_IN’. This will be overriden if ‘name’ is in icomm_kwargs.
- ocomm_name (str, optional) – Name for new output comm. Defaults to name + ‘_OUT’. This will be overriden if ‘name’ is in ocomm_kwargs.
- icomm_comm (str, optional) – Name of class for new input comm. Defaults to None. This will be overrriden if ‘comm’ is in icomm_kwargs.
- ocomm_comm (str, optional) – Name of class for new output comm. Defaults to None. This will be overrriden if ‘comm’ is in ocomm_kwargs.
- icomm_kwargs (dict, optional) – Keyword arguments for the icomm_comm new_comm_kwargs class method.
- ocomm_kwargs (dict, optional) – Keyword arguments for the ocomm_comm new_comm_kwargs class method.
-
opp_address
¶ str – Address for opposite RPC comm.
-
opp_comms
¶ dict – Name/address pairs for opposite comms.
-
recv
(*args, **kwargs)[source]¶ Receive a message from the input comm.
Parameters: - *args – Arguments are passed to input comm recv method.
- **kwargs – Keyword arguments are passed to input comm recv method.
Returns: Output from input comm recv method.
Return type: obj
cis_interface.communication.ServerComm module¶
-
class
cis_interface.communication.ServerComm.
ServerComm
(name, request_comm=None, response_kwargs=None, dont_open=False, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommBase
Class for handling Server side communication.
Parameters: - name (str) – The environment variable where communication address is stored.
- request_comm (str, optional) – Comm class that should be used for the request comm. Defaults to None.
- response_kwargs (dict, optional) – Keyword arguments for the response comm. Defaults to empty dict.
- **kwargs – Additional keywords arguments are passed to the input comm.
-
response_kwargs
¶ dict – Keyword arguments for the response comm.
-
icomm
¶ Comm – Request comm.
-
ocomm
¶ Comm – Response comm for last request.
-
is_closed
¶ bool – True if the connection is closed.
-
is_open
¶ bool – True if the connection is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_recv
¶ int – The number of messages in the connection.
-
n_msg_recv_drain
¶ int – The number of messages in the connection to drain.
-
classmethod
new_comm_kwargs
(name, request_comm=None, **kwargs)[source]¶ Initialize communication with new comms.
Parameters:
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
opp_comms
¶ dict – Name/address pairs for opposite comms.
-
recv
(*args, **kwargs)[source]¶ Receive a message from the input comm and open a new response comm for output using address from the header.
Parameters: - *args – Arguments are passed to input comm recv method.
- **kwargs – Keyword arguments are passed to input comm recv method.
Returns: Output from input comm recv method.
Return type: obj
cis_interface.communication.ZMQComm module¶
-
class
cis_interface.communication.ZMQComm.
ZMQComm
(name, dont_backlog=False, **kwargs)[source]¶ Bases:
cis_interface.communication.AsyncComm.AsyncComm
Class for handling I/O using ZeroMQ sockets.
Parameters: - name (str) – The environment variable where the socket address is stored.
- context (zmq.Context, optional) – ZeroMQ context that should be used. Defaults to None and the global context is used.
- socket_type (str, optional) – The type of socket that should be created. Defaults to _default_socket_type. See zmq for all options.
- socket_action (str, optional) – The action that the socket should perform. Defaults to action based on the direction (‘connect’ for ‘recv’, ‘bind’ for ‘send’.)
- topic_filter (str, optional) – Message filter to use when subscribing. This is only used for ‘SUB’ socket types. Defaults to ‘’ which is all messages.
- dealer_identity (str, optional) – Identity that should be used to route messages to a dealer socket. Defaults to ‘0’.
- **kwargs – Additional keyword arguments are passed to CommBase.
-
context
¶ zmq.Context – ZeroMQ context that will be used.
-
socket
¶ zmq.Socket – ZeroMQ socket.
-
socket_type_name
¶ str – The type of socket that should be created.
-
socket_type
¶ int – ZeroMQ socket type.
-
socket_action
¶ str, optional – The action that the socket should perform.
-
topic_filter
¶ str – Message filter to use when subscribing.
-
dealer_identity
¶ str – Identity that should be used to route messages to a dealer socket.
-
address_param
¶ dict – Address parameters.
-
check_reply_socket_recv
(msg)[source]¶ Check incoming message for reply address.
Parameters: msg (str) – Incoming message to check. Returns: Messages with reply address removed if present. Return type: str
-
check_reply_socket_send
(msg)[source]¶ Append reply socket address if it
Parameters: msg (str) – Message that will be piggy backed on. Returns: Message with reply address if it has not been sent. Return type: str
-
create_work_comm_kwargs
¶ dict – Keyword arguments for a new work comm.
-
get_work_comm_kwargs
¶ dict – Keyword arguments for an existing work comm.
-
header2workcomm
(header, **kwargs)[source]¶ Get a work comm based on header info.
Parameters: - header (dict) – Information that will be sent in the message header to the work comm.
- **kwargs – Additional keyword arguments are passed to the parent method.
Returns: Work comm.
Return type:
-
host
¶ str – Host that socket is connected to.
-
is_confirmed_recv
¶ bool – True if all received messages have been confirmed.
-
is_message
(flags)[source]¶ Poll the socket for a message.
Parameters: flags (int) – ZMQ poll flags. Returns: True if there is a message matching the flags, False otherwise. Return type: bool
-
is_open_direct
¶ bool – True if the socket is open.
-
maxMsgSize
¶ int – Maximum size of a single message that should be sent.
-
n_msg_direct_recv
¶ int – Number of messages currently being routed from recv.
-
n_msg_direct_send
¶ int – Number of messages currently being routed.
-
classmethod
new_comm_kwargs
(name, protocol=None, host=None, port=None, **kwargs)[source]¶ Initialize communication with new queue.
Parameters: - name (str) – Name of new socket.
- protocol (str, optional) – The protocol that should be used. Defaults to None and is set to _default_protocol. See zmq for details.
- host (str, optional) – The host that should be used. Invalid for ‘inproc’ protocol. Defaults to ‘localhost’.
- port (int, optional) – The port used. Invalid for ‘inproc’ protocol. Defaults to None and a random port is choosen.
- **kwargs – Additional keywords arguments are returned as keyword arguments for the new comm.
Returns: Arguments and keyword arguments for new socket.
Return type:
-
on_send
(msg, header_kwargs=None)[source]¶ Process message to be sent including handling serializing message and handling EOF.
Parameters: - msg (obj) – Message to be sent
- header_kwargs (dict, optional) – Keyword arguments that should be added to the header.
Returns: - Truth of if message should be sent, raw
bytes message to send, and header info contained in the message.
Return type:
-
on_send_eof
()[source]¶ Actions to perform when EOF being sent.
Returns: True if EOF message should be sent, False otherwise. Return type: bool
-
opp_address
¶ str – Address for opposite comm.
-
opp_comm_kwargs
()[source]¶ Get keyword arguments to initialize communication with opposite comm object.
Returns: Keyword arguments for opposite comm object. Return type: dict
-
port
¶ str – Port that socket is connected to.
-
protocol
¶ str – Protocol that socket uses.
-
registry_key
¶ str – String used to register the socket.
-
class
cis_interface.communication.ZMQComm.
ZMQProxy
(srv_address, context=None, retry_timeout=-1, nretry=1, **kwargs)[source]¶ Bases:
cis_interface.communication.CommBase.CommServer
Start a proxy in a new thread for a server address. A client-side address will be randomly generated.
Parameters: - srv_address (str) – Address that should face the server(s).
- context (zmq.Context, optional) – ZeroMQ context that should be used. Defaults to None and the global context is used.
- protocol (str, optional) – Protocol that should be used for the sockets. Defaults to None and is set to _default_protocol.
- host (str, optional) – Host for socket address. Defaults to ‘localhost’.
- retry_timeout (float, optional) – Time (in seconds) that should be waited before retrying to bind the sockets to the addresses. If negative, a retry will not be attempted and an error will be raised. Defaults to -1.
- nretry (int, optional) – Number of times to try binding the sockets to the addresses. Defaults to 1.
- **kwargs – Additional keyword arguments are passed to the parent class.
-
srv_address
¶ str – Address that faces the server(s).
-
cli_address
¶ str – Address that faces the client(s).
-
context
¶ zmq.Context – ZeroMQ context that will be used.
-
srv_socket
¶ zmq.Socket – Socket facing client(s).
-
cli_socket
¶ zmq.Socket – Socket facing server(s).
-
cli_count
¶ int – Number of clients that have connected to this proxy.
-
cis_interface.communication.ZMQComm.
bind_socket
(socket, address, retry_timeout=-1, nretry=1)[source]¶ Bind a socket to an address, getting a random port as necessary.
Parameters: - socket (zmq.Socket) – Socket that should be bound.
- address (str) – Address that socket should be bound to.
- retry_timeout (float, optional) – Time (in seconds) that should be waited before retrying to bind the socket to the address. If negative, a retry will not be attempted and an error will be raised. Defaults to -1.
- nretry (int, optional) – Number of times to try binding the socket to the addresses. Defaults to 1.
Returns: - Address that socket was bound to, including random port if one
was used.
Return type:
-
cis_interface.communication.ZMQComm.
format_address
(protocol, host, port=None)[source]¶ Format an address based on its parts.
Parameters: Returns: Complete address.
Return type: Raises: ValueError
– If the protocol is not recognized.
-
cis_interface.communication.ZMQComm.
get_ipc_host
()[source]¶ Get an IPC host using uuid.
Returns: File path for IPC transport created using uuid. Return type: str
-
cis_interface.communication.ZMQComm.
get_socket_type_mate
(t_in)[source]¶ Find the counterpart socket type.
Parameters: t_in (str) – Socket type. Returns: Counterpart socket type. Return type: str Raises: ValueError
– If t_in is not a recognized socket type.
-
cis_interface.communication.ZMQComm.
parse_address
(address)[source]¶ Split an address into its parts.
Parameters: address (str) – Address to be split.
Returns: Parameters extracted from the address.
Return type: Raises: ValueError
– If the address dosn’t contain ‘://’.ValueError
– If the protocol is not supported.
Module contents¶
-
cis_interface.communication.
new_comm
(name, comm=None, **kwargs)[source]¶ Return a new communicator, creating necessary components for communication (queues, sockets, channels, etc.).
Parameters: Returns: Communicator of given class.
Return type: Comm
-
cis_interface.communication.
get_comm
(name, comm=None, new_comm_class=None, **kwargs)[source]¶ Return communicator for existing comm components.
Parameters: Returns: Communicator of given class.
Return type: Comm
-
cis_interface.communication.
get_comm_class
(comm=None)[source]¶ Return a communication class given it’s name.
Parameters: comm (str, optional) – Name of communicator class. Defaults to tools.get_default_comm() if not provided. Returns: Communicator class. Return type: class