oceanum.datamesh.Connector

class oceanum.datamesh.Connector(token=None, service='https://datamesh.oceanum.io', gateway=None)[source]

Datamesh connector class.

All datamesh operations are methods of this class

Attributes

host

Datamesh host

Methods

__init__(token=None, service='https://datamesh.oceanum.io', gateway=None)[source]

Datamesh connector constructor

Parameters
  • token (string) – Your datamesh access token. Defaults to os.environ.get(“DATAMESH_TOKEN”, None).

  • service (string, optional) – URL of datamesh service. Defaults to os.environ.get(“DATAMESH_SERVICE”, “https://datamesh.oceanum.io”).

  • gateway (string, optional) – URL of gateway service. Defaults to os.environ.get(“DATAMESH_GATEWAY”, “https://gateway.<datamesh_service_domain>”).

Raises

ValueError – Missing or invalid arguments

get_catalog(filter={})[source]

Get datamesh catalog

Parameters

filter (dict, optional) – Set of filters to apply. Defaults to {}.

Returns

A datamesh catalog instance

Return type

oceanum.datamesh.Catalog

async get_catalog_async(filter={})[source]

Get datamesh catalog asynchronously

Parameters
  • filter (dict, optional) – Set of filters to apply. Defaults to {}.

  • loop – event loop. default=None will use asyncio.get_running_loop()

  • executorconcurrent.futures.Executor instance. default=None will use the default executor

Returns

A datamesh catalog instance

Return type

Coroutine<oceanum.datamesh.Catalog>

get_datasource(datasource_id)[source]

Get a Datasource instance from the datamesh. This does not load the actual data.

Parameters

datasource_id (string) – Unique datasource id

Returns

A datasource instance

Return type

oceanum.datamesh.Datasource

Raises

DatameshConnectError – Datasource cannot be found or is not authorized for the datamesh key

async get_datasource_async(datasource_id)[source]

Get a Datasource instance from the datamesh asynchronously. This does not load the actual data.

Parameters
  • datasource_id (string) – Unique datasource id

  • loop – event loop. default=None will use asyncio.get_running_loop()

  • executorconcurrent.futures.Executor instance. default=None will use the default executor

Returns

A datasource instance

Return type

Coroutine<oceanum.datamesh.Datasource>

Raises

DatameshConnectError – Datasource cannot be found or is not authorized for the datamesh key

load_datasource(datasource_id, use_dask=True)[source]

Load a datasource into the work environment

Parameters
  • datasource_id (string) – Unique datasource id

  • use_dask (bool, optional) – Load datasource as a dask enabled datasource if possible. Defaults to True.

Returns

The datasource container

Return type

Union[pandas.DataFrame, geopandas.GeoDataFrame, xarray.Dataset]

async load_datasource_async(datasource_id, use_dask=True)[source]

Load a datasource asynchronously into the work environment

Parameters
  • datasource_id (string) – Unique datasource id

  • use_dask (bool, optional) – Load datasource as a dask enabled datasource if possible. Defaults to True.

  • loop – event loop. default=None will use asyncio.get_running_loop()

  • executorconcurrent.futures.Executor instance. default=None will use the default executor

Returns

The datasource container

Return type

coroutine<Union[pandas.DataFrame, geopandas.GeoDataFrame, xarray.Dataset]>

query(query)[source]

Make a datamesh query

Parameters

query (Union[oceanum.datamesh.Query, dict]) – Datamesh query as a query object or a valid query dictionary

Returns

The datasource container

Return type

Union[pandas.DataFrame, geopandas.GeoDataFrame, xarray.Dataset]

async query_async(query)[source]

Make a datamesh query asynchronously

Parameters
  • query (Union[oceanum.datamesh.Query, dict]) – Datamesh query as a query object or a valid query dictionary

  • loop – event loop. default=None will use asyncio.get_running_loop()

  • executorconcurrent.futures.Executor instance. default=None will use the default executor

Returns

The datasource container

Return type

Coroutine<Union[pandas.DataFrame, geopandas.GeoDataFrame, xarray.Dataset]>