midgard.dev

midgard.dev.__marked_for_deletion_cache

Midgard library module for caching

Description:

Adds caching to properties on classes, using the recipe explained in Python Cookbook, 3rd ed. Recipe 8.10.

Also, adds a cached property that may depend on other data, with the possibility to reset the cache if the dependencies change.

dependent_property

Full name: midgard.dev.__marked_for_deletion_cache.dependent_property

Signature: ()

Decorator for cached properties that can be reset when dependencies change

forget_dependent_values()

Full name: midgard.dev.__marked_for_deletion_cache.forget_dependent_values

Signature: (obj:object, *dependencies:str) -> None

Reset cache when given dependencies change

property

Full name: midgard.dev.__marked_for_deletion_cache.property

Signature: (fget:Callable) -> None

Cached property, see Python Cookbook, 3rd ed. Recipe 8.10.

register_dependencies

Full name: midgard.dev.__marked_for_deletion_cache.register_dependencies

Signature: ()

Metaclass for registering dependencies using dot notation

midgard.dev.__marked_for_deletion_optional

Midgard library module for handling optional dependencies

Description:

Import dependencies that are only necessary for specific parts of Midgard. Using this module will delay raising an ImportError until the dependency is actually used. This means that if one for instance only wants to run a GNSS analysis (or only use a Rinex-parser) installing special libraries only used for VLBI is not necessary.

Examples:

The optional import is typically used as follows::

from midgard.lib import optional
netCDF4 = optional.optional_import('netCDF4')

Delete this?

EmptyStringMock

Full name: midgard.dev.__marked_for_deletion_optional.EmptyStringMock

Signature: (name:str, raise_error:bool=True, attrs:Union[Dict[str, Any], NoneType]=None, error_msg:Union[str, NoneType]=None) -> None

A mock object whose properties are all empty strings

SimpleMock

Full name: midgard.dev.__marked_for_deletion_optional.SimpleMock

Signature: (name:str, raise_error:bool=True, attrs:Union[Dict[str, Any], NoneType]=None, error_msg:Union[str, NoneType]=None) -> None

Class that can stand in for any other object

The SimpleMock is used to stand in for any library that can not be imported. The mock object simply returns itself whenever it is called, or any attributes are looked up on the object. This is done, to avoid ImportErrors when a library is imported, but never used (for instance if a plugin is loaded but never called).

Instead the ImportError is raised when the SimpleMock is used in any way. The ImportError will only be raised once for any SimpleMock-object (which is only important if the ImportError is caught and the program carries on).

The exception is if any attributes (attrs) are explicitly defined on the mock. No exception is raised if those attributes are looked up.

optional_import()

Full name: midgard.dev.__marked_for_deletion_optional.optional_import

Signature: (module_name:str, raise_error:bool=True, mock_cls:type=<class 'midgard.dev.__marked_for_deletion_optional.SimpleMock'>, attrs:Union[Dict[str, Any], NoneType]=None) -> Union[Any, midgard.dev.__marked_for_deletion_optional.SimpleMock]

Try to import an optional module

If the module does not exist, a SimpleMock-object is returned instead. If this SimpleMock-object is later used, an ImportError will be raised then (if raise_error is True, which is default).

Args:

Returns:

Imported module object, or a SimpleMock-object if the module can not be imported.

midgard.dev.console

Simpler dealing with the console

Description:

Utilities for using the console. Mainly wrappers around other libraries to make them easier and more intuitive to use.

Size of console: The two functions lines() and columns() report the current size of the console.

Textwrapping: The function fill() can be used to rewrap a text-string so that it fits inside the console.

Examples:

>>> from midgard.dev import console
>>> console.columns()  # doctest: +SKIP
86

>>> print(console.fill(a_very_long_string))  # doctest: +SKIP
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras tempus eleifend feugiat.
Maecenas vitae posuere metus. Sed sit amet fermentum velit. Aenean vitae turpis at
risus sollicitudin fringilla in in nisi. Maecenas vitae ante libero. Aenean ut eros
consequat, ornare erat at, tempus arcu. Suspendisse velit leo, eleifend eget mi non,
vehicula ultricies erat. Vestibulum id nisi eget nisl venenatis dignissim. Duis cursus
quam dui, vel hendrerit nibh lacinia id.

>>> print(console.color.Fore.YELLOW + console.color.Back.BLUE + 'I am YELLOW text on BLUE backdrop!')  # doctest: +SKIP
I am YELLOW text on a BLUE background!

columns()

Full name: midgard.dev.console.columns

Signature: () -> int

The width of the console

Returns:

The width of the console in characters.

dedent()

Full name: midgard.dev.console.dedent

Signature: (text:str, num_spaces:Union[int, NoneType]=None) -> str

Wrapper around textwrap.dedent

Dedents at most num_spaces. If num_spaces is not specified, dedents as much as possible.

Args:

Returns:

Dedented string.

fill()

Full name: midgard.dev.console.fill

Signature: (text:str, *, width:Union[int, NoneType]=None, hanging:Union[int, NoneType]=None, **tw_args:Any) -> str

Wrapper around textwrap.fill

The tw_args are passed on to textwrap.fill. See textwrap.TextWrapper for available keyword arguments.

The default value for width is console.columns(), while the new argument hanging, if defined, will try to set (although not override) the textwrap-arguments initial_indent and subsequent_indent to create a hanging indent (no indent on the first line) of hanging spaces.

Args:

Returns:

Wrapped string.

indent()

Full name: midgard.dev.console.indent

Signature: (text:str, num_spaces:int, **tw_args:Any) -> str

Wrapper around textwrap.indent

The tw_args are passed on to textwrap.indent.

Args:

Returns:

Indented string.

lines()

Full name: midgard.dev.console.lines

Signature: () -> int

The height of the console

Returns:

The heigth of the console in characters.

num_leading_spaces()

Full name: midgard.dev.console.num_leading_spaces

Signature: (text:str, space_char:str=' ') -> int

Count number of leading spaces in a string

Args:

Returns:

Number of leading spaces.

midgard.dev.exceptions

Definition of Midgard-specific exceptions

Description:

Custom exceptions used by Midgard for more specific error messages and handling.

FieldDoesNotExistError

Full name: midgard.dev.exceptions.FieldDoesNotExistError

Signature: ()

FieldExistsError

Full name: midgard.dev.exceptions.FieldExistsError

Signature: ()

InitializationError

Full name: midgard.dev.exceptions.InitializationError

Signature: ()

MidgardException

Full name: midgard.dev.exceptions.MidgardException

Signature: ()

MidgardExit

Full name: midgard.dev.exceptions.MidgardExit

Signature: ()

MissingConfigurationError

Full name: midgard.dev.exceptions.MissingConfigurationError

Signature: ()

MissingDataError

Full name: midgard.dev.exceptions.MissingDataError

Signature: ()

MissingEntryError

Full name: midgard.dev.exceptions.MissingEntryError

Signature: ()

MissingSectionError

Full name: midgard.dev.exceptions.MissingSectionError

Signature: ()

ParserError

Full name: midgard.dev.exceptions.ParserError

Signature: ()

TimerNotRunning

Full name: midgard.dev.exceptions.TimerNotRunning

Signature: ()

TimerRunning

Full name: midgard.dev.exceptions.TimerRunning

Signature: ()

UnitError

Full name: midgard.dev.exceptions.UnitError

Signature: ()

UnknownConstantError

Full name: midgard.dev.exceptions.UnknownConstantError

Signature: ()

UnknownConversionError

Full name: midgard.dev.exceptions.UnknownConversionError

Signature: ()

UnknownEnumError

Full name: midgard.dev.exceptions.UnknownEnumError

Signature: ()

UnknownPackageError

Full name: midgard.dev.exceptions.UnknownPackageError

Signature: ()

UnknownPluginError

Full name: midgard.dev.exceptions.UnknownPluginError

Signature: ()

UnknownSystemError

Full name: midgard.dev.exceptions.UnknownSystemError

Signature: ()

midgard.dev.library

Python wrapper around C-libraries

Description:

Loads a C-library. If a library is missing, a mock library is returned. If this mock is used for anything, a warning will be printed. This is done to avoid dependencies to all the C/C++-libraries for Python programs only using some of them.

SimpleMock

Full name: midgard.dev.library.SimpleMock

Signature: (name, raise_error=True)

Class that can stand in for any other object

The SimpleMock is used to stand in for any library that can not be imported. The mock object simply returns itself whenever it is called, or any attributes are looked up on the object. This is done, to avoid ImportErrors when a library is imported, but never used (typically because a plugin is loaded but never called).

Instead the ImportError is raised when the SimpleMock is used in any way. The ImportError will only be raised once for any SimpleMock-object (which is only important if the ImportError is caught and the program carries on).

load_name()

Full name: midgard.dev.library.load_name

Signature: (library_name, func_specs=None, name_patterns=None)

Load the given shared C-library

See load_path for an explanation of the func_specs and name_patterns-arguments.

Args:

library_name (String): The name of the library. func_specs (Dict): Specification of types in lib (see load_path). name_patterns (List): Name mangling patterns (see load_path).

Returns:

ctypes.CDLL: Representation of the shared library.

load_path()

Full name: midgard.dev.library.load_path

Signature: (library_path, func_specs=None, name_patterns=None)

Load the given shared C-library

The optional func_specs-dictionary can be used to specify argument and return types of functions in the library (see the ctypes documentation for information about argtypes and restype). The dictionary should be on the form::

func_spec = {'func_1': dict(func_name='name_of_func_1_in_lib',
                            argtypes=[ ... argtypes of func_1 ... ],
                            restype=... restype of func_1 ...),
             'func_2': ...
            }

If the library is not found, a mock library is returned instead. The mock library will print a warning if it is used.

For some libraries, name mangling is used and this might be different depending on operating system and how the library is compiled. For instance, in a Fortran library the function Test might be represented as __Test on a Windows system and test_ (with lower-case t) on a Linux system. This can be handled by providing a list of possible patterns. The above example can be handled by::

name_patterns = ('__{func_name}', '{func_name_lower}_')

In this case, each function in func_specs is looked up by testing each pattern in turn until a match is found.

Args:

library_path (String): The path to the library. func_specs (Dict): Specification of types in library (see above). name_patterns (List): Name mangling patterns (see above).

Returns:

ctypes.CDLL: Representation of the shared library.

midgard.dev.log

Midgard library module for logging

Description:

This module provides simple logging inside Midgard.

To use it, you must first add a an active logger. This is typically done using one of the init-functions: init() or file_init().

To write a log message, simply call one of midgard.log.debug, midgard.log.info, midgard.log.warn, midgard.log.error or midgard.log.fatal with a log message.

To add a different logger, you should subclass the Logger abstract class.

Example:

>>> from midgard.dev import log
>>> log.init("info", prefix="My prefix")
>>> n, m = 5, 3
>>> log.info(f"Calculating the inverse of a {n:>2d}x{m:<2d} matrix")
INFO  [My prefix] Calculating the inverse of a  5x3  matrix

ConsoleLogger

Full name: midgard.dev.log.ConsoleLogger

Signature: (log_level:Union[str, NoneType]=None, prefix:str='', use_command_line:bool=True) -> None

Log to the console, the log level can also be set using command line parameters

FileLogger

Full name: midgard.dev.log.FileLogger

Signature: (file_path:Union[str, pathlib.Path], log_level:Union[str, NoneType]=None, prefix:str='', rotation:Union[int, NoneType]=None) -> None

Log to a file, the log files can be rotated so that older files are kept

Logger

Full name: midgard.dev.log.Logger

Signature: (log_level:Union[str, NoneType]=None, prefix:str='')

Abstract class that can be specialized to create new loggers

all()

Full name: midgard.dev.log.all

Signature: (log_text:str, *, level:str='all') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

blank()

Full name: midgard.dev.log.blank

Signature: () -> None

Log blank line

debug()

Full name: midgard.dev.log.debug

Signature: (log_text:str, *, level:str='debug') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

error()

Full name: midgard.dev.log.error

Signature: (log_text:str, *, level:str='error') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

fatal()

Full name: midgard.dev.log.fatal

Signature: (log_text:str, *, level:str='fatal') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

file_init

Full name: midgard.dev.log.file_init

Signature: (file_path:Union[str, pathlib.Path], log_level:Union[str, NoneType]=None, prefix:str='', rotation:Union[int, NoneType]=None) -> None

Log to a file, the log files can be rotated so that older files are kept

info()

Full name: midgard.dev.log.info

Signature: (log_text:str, *, level:str='info') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

init

Full name: midgard.dev.log.init

Signature: (log_level:Union[str, NoneType]=None, prefix:str='', use_command_line:bool=True) -> None

Log to the console, the log level can also be set using command line parameters

log()

Full name: midgard.dev.log.log

Signature: (log_text:str, level:str) -> None

Log text at the given level

none()

Full name: midgard.dev.log.none

Signature: (log_text:str, *, level:str='none') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

Full name: midgard.dev.log.print_file

Signature: (log_path:Union[str, pathlib.Path], log_level:str='info', print_func:Callable[[str], NoneType]=<built-in function print>) -> None

Print a log file with colors, stripping away any item below log_level

warn()

Full name: midgard.dev.log.warn

Signature: (log_text:str, *, level:str='warn') -> None

partial(func, args, *keywords) - new function with partial application of the given arguments and keywords.

midgard.dev.plugins

Set up a plug-in architecture for Midgard

Description:

In order to be able to add models, parsers, data sources etc without needing to hardcode names, but rather pick them from configuration files, we use a simple plug-in architecture. The plug-in mechanism is based on the different plug-ins registering themselves using the register decorator:

from midgard.dev import plugins

@plugins.register
def simple_model(*args, **kwargs):
    ...

Plug-ins are registered based on the name of the module (file) they are defined in, as well as the package (directory) which contains them. Typically all plug-ins of a given type are collected in a package, e.g. models, techniques, parsers, etc. To list all plug-ins in a package use names:

> from midgard.dev import plugins
> plugins.names('midgard.models')
['model_one', 'model_three', 'model_two']

If the optional parameter config_key is given, then only plug-ins listed in the corresponding section in the current configuration file is listed. For instance, if the configuration file contains a line saying

ham_models = model_three, model_one

then we can list only the ham_models as follows:

> from midgard.dev import plugins
> plugins.names('midgard.models', config_key='ham_models')
['model_one', 'model_three']

Note that the plug-ins by default are sorted alphabetically.

To run the plug-ins, use either call_all or call_one. The former calls all plug-ins and returns a dictionary containing the result from each plug-in. As with names the optional parameter config_key may be given:

> from midgard.dev import plugins
> plugins.call_all('midgard.models', config_key='ham_models', arg_to_plugin='hello')
{'model_three': <result from model_three>, 'model_one': <result from model_one>}

Arguments to the plug-ins should be passed as named arguments to call_all.

Similarly, one plug-in may be called explicitly using call_one:

> from midgard.dev import plugins
> plugins.call_one('midgard.models', plugin_name='model_one', arg_to_plugin='hello')
<result from model_one>

There may be more than one function in each plug-in that is decorated by register. In this case, the default behaviour is that only the first function will be called. To call the other registered functions one should use the list_parts function to get a list of these functions and call them explicitly using the part optional parameter to call_one:

> from midgard.dev import plugins
> plugins.list_parts('midgard.techniques', plugin_name='vlbi')
['read', 'edit', 'calculate', 'estimate', 'write_result'])
> for part in plugins.list_parts('midgard.techniques', plugin_name='vlbi'):
...   plugins.call_one('midgard.techniques', plugin_name='vlbi', part=part, ...)

Plugin

Full name: midgard.dev.plugins.Plugin

Signature: (name:str, function:Callable, file_path:pathlib.Path, sort_value:int)

Information about a plug-in

Args:

add_alias()

Full name: midgard.dev.plugins.add_alias

Signature: (package_name:str, alias:str) -> None

Add alias to plug-in package

This allows one package of plug-ins to be spread over several directories

Args:

call()

Full name: midgard.dev.plugins.call

Signature: (package_name:str, plugin_name:str, part:Union[str, NoneType]=None, prefix:Union[str, NoneType]=None, plugin_logger:Union[Callable[[str], NoneType], NoneType]=None, **plugin_args:Any) -> Any

Call one plug-in

Args:

Returns:

Return value of the plug-in.

call_all()

Full name: midgard.dev.plugins.call_all

Signature: (package_name:str, plugins:Union[List[str], NoneType]=None, part:Union[str, NoneType]=None, prefix:Union[str, NoneType]=None, plugin_logger:Union[Callable[[str], NoneType], NoneType]=None, **plugin_args:Any) -> Dict[str, Any]

Call all plug-ins in a package

If plugins is given, it should be a list of names of plug-ins. If a plug-in listed in the plugins-list or in the config file does not exist, an UnknownPluginError is raised.

If plugins is not given, all available plugins will be called. Do note, however, that this will import all python files in the package.

Args:

Returns:

Dictionary of all results from the plug-ins.

doc()

Full name: midgard.dev.plugins.doc

Signature: (package_name:str, plugin_name:str, part:Union[str, NoneType]=None, prefix:Union[str, NoneType]=None, long_doc:bool=True, include_details:bool=False, use_module:bool=False) -> str

Document one plug-in

If the plug-in is not part of the package an UnknownPluginError is raised.

If there are several functions registered in a plug-in and part is not specified, then the first function registered in the plug-in will be documented.

Args:

Returns:

Documentation of the plug-in.

doc_all()

Full name: midgard.dev.plugins.doc_all

Signature: (package_name:str, plugins:Union[Iterable[str], NoneType]=None, prefix:Union[str, NoneType]=None, long_doc:bool=True, include_details:bool=False, use_module:bool=False) -> Dict[str, str]

Call all plug-ins in a package

If plugins is given, it should be a list of names of plug-ins. If a plug-in listed in the plugins-list does not exist, an UnknownPluginError is raised.

If plugins is not given, all available plugins will be called. Do note, however, that this will import all python files in the package.

Args:

Returns:

Dictionary of all doc-strings from the plug-ins.

exists()

Full name: midgard.dev.plugins.exists

Signature: (package_name:str, plugin_name:str) -> bool

Check whether or not a plug-in exists in a package

Tries to import the given plug-in.

Args:

Returns:

True if plug-in exists, False otherwise.

get()

Full name: midgard.dev.plugins.get

Signature: (package_name:str, plugin_name:str, part:Union[str, NoneType]=None, prefix:Union[str, NoneType]=None) -> midgard.dev.plugins.Plugin

Get a specific plugin-object

If the plug-in is not part of the package an UnknownPluginError is raised.

If there are several functions registered in a plug-in and part is not specified, then the first function registered in the plug-in will be called.

Args:

Returns:

Plugin-namedtuple representing the plug-in.

load()

Full name: midgard.dev.plugins.load

Signature: (package_name:str, plugin_name:str, prefix:Union[str, NoneType]=None) -> str

Load one plug-in from a package

First tries to load the plugin with the given name. If that fails, it tries to load {prefix}_{plugin_name} instead.

Args:

Returns:

Actual name of plug-in (with or without prefix).

names()

Full name: midgard.dev.plugins.names

Signature: (package_name:str, plugins:Union[Iterable[str], NoneType]=None, prefix:Union[str, NoneType]=None) -> List[str]

List plug-ins in a package

If plugins is given, it should be a list of names of plug-ins. If a plug-in listed in the plugins-list does not exist, an UnknownPluginError is raised.

If plugins is not given, all available plugins will be listed. Do note, however, that this will import all python files in the package.

Args:

Returns:

List of strings with names of plug-ins.

parts()

Full name: midgard.dev.plugins.parts

Signature: (package_name:str, plugin_name:str, prefix:Union[str, NoneType]=None) -> List[str]

List all parts of one plug-in

Args:

Returns:

register()

Full name: midgard.dev.plugins.register

Signature: (func:Callable, name:Union[str, NoneType]=None, sort_value:int=0) -> Callable

Decorator used to register a plug-in

Plug-ins are registered based on the name of the module (file) they are defined in, as well as the package (directory) which contains them. Typically all plug-ins of a given type are collected in a package, e.g. models, techniques, parsers, etc. The path to the source code file is also stored. This is used to be able to add the source code as a dependency file when the plug-in is called.

If name is given, the plug-in is registered based on this name instead of the name of the module. The name of the module is still registered as a part that can be used to distinguish between similar plug-ins in different files (see for instance how session is used in midgard.pipelines).

Args:

Returns:

The function that is being registered.

register_named()

Full name: midgard.dev.plugins.register_named

Signature: (name:str) -> Callable

Decorator used to register a named plug-in

This allows for overriding the name used to register the plug-in. See register for more details.

Args:

Returns:

Decorator that registers a named function.

register_ordered()

Full name: midgard.dev.plugins.register_ordered

Signature: (sort_value:int) -> Callable

Decorator used to register a plug-in with a specific sort order

The sort value should be a number. Lower numbers are sorted first, higher numbers last. Plug-ins without an explicit sort_order gets the sort value of 0.

Args:

Returns:

Decorator that registers an ordered function.

signature()

Full name: midgard.dev.plugins.signature

Signature: (package_name:str, plugin_name:str, part:Union[str, NoneType]=None, prefix:Union[str, NoneType]=None) -> inspect.Signature

Get signature of a plug-in

If the plug-in is not part of the package an UnknownPluginError is raised.

If there are several functions registered in a plug-in and part is not specified, then the first function registered in the plug-in will be documented.

Args:

Returns:

Signature of the plugin

midgard.dev.profiler

Add a profiler when running

Supports several profilers including cprofile, line_profiler, memprof and memory_profiler.

CProfile

Full name: midgard.dev.profiler.CProfile

Signature: ()

cprofile is used for profiling the whole program

LineProfiler

Full name: midgard.dev.profiler.LineProfiler

Signature: ()

line_profiler is used to profile one or a few functions in detail

Profiler

Full name: midgard.dev.profiler.Profiler

Signature: ()

Base class for profilers

midgard.dev.timer

Class for timing the running time of functions and code blocks

Description:

The dev.timer can be used to log the running time of functions and general code blocks. Typically, you will import the Timer-class from within the module:

from midgard.dev.timer import Timer

The Timer can then be used in three different ways:

  1. As a decorator to time one function:

    @Timer('The time to execute some_function was')
    def some_function(some_argument, some_other_argument=some_value):
        pass
    
  2. As a context manager together with with to time a code block:

    with Timer('Finish doing stuff in', logger=logger.debug) as t:
        do_something()
        do_something_else()
    
  3. With explicit start- and end-statements:

    t = Timer()
    t.start()
    do_something()
    do_something_else()
    t.end()
    

As can be seen in the examples above, Timer() may be called with several optional parameters, including the text to report when the timer ends and which logger is used to report the timing. See Timer.__init__ for more details.

AccumulatedTimer

Full name: midgard.dev.timer.AccumulatedTimer

Signature: (text:str='Elapsed time:', fmt:str='.4f', logger:Union[Callable[[str], NoneType], NoneType]=functools.partial(<function log at 0x7f800f4530d0>, level='info')) -> None

Timer

Full name: midgard.dev.timer.Timer

Signature: (text:str='Elapsed time:', fmt:str='.4f', logger:Union[Callable[[str], NoneType], NoneType]=functools.partial(<function log at 0x7f800f4530d0>, level='info')) -> None

Class for timing running time of functions and code blocks.