API Reference

intake_elasticsearch.ESSeqPlugin() Plugin for ElasticSearch to sequence reader
intake_elasticsearch.ESTablePlugin() Plugin for ElasticSearch to pandas reader
intake_elasticsearch.elasticsearch_table.ElasticSearchTableSource(…) Data source which executes arbitrary queries on ElasticSearch
intake_elasticsearch.elasticsearch_seq.ElasticSearchSeqSource(…) Data source which executes arbitrary queries on ElasticSearch
class intake_elasticsearch.ESSeqPlugin[source]

Plugin for ElasticSearch to sequence reader

Methods

open(query, **kwargs) Create ElasticSearchSource instance
separate_base_kwargs  
open(query, **kwargs)[source]

Create ElasticSearchSource instance

Parameters:
query : str
Query string (lucene syntax or JSON text)
qargs: dict
Set of modifiers to apply to the query (https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search)
kwargs (dict):
Additional parameters to pass to ElasticSearch init.
class intake_elasticsearch.ESTablePlugin[source]

Plugin for ElasticSearch to pandas reader

Methods

open(query, **kwargs) Create ElasticSearchSource instance
separate_base_kwargs  
class intake_elasticsearch.elasticsearch_table.ElasticSearchTableSource(*args, **kwargs)[source]

Data source which executes arbitrary queries on ElasticSearch

This is the tabular reader: will return dataframes. Nested return items will become dict-like objects in the output.

Parameters:
query: str

Query to execute. Can either be in Lucene single-line format, or a JSON structured query (presented as text)

qargs: dict

Further parameters to pass to the query, such as set of indexes to consider, filtering, ordering. See http://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search

es_kwargs: dict

Settings for the ES connection, e.g., a simple local connection may be {'host': 'localhost', 'port': 9200}. Other keywords to the Plugin that end up here and are material:

scroll: str

how long the query is live for, default '100m'

size: int

the paging size when downloading, default 1000.

metadata: dict

Extra information for this source.

Attributes:
plot

Methods

close() Close open resources corresponding to this data source.
discover() Open resource and populate the source attributes.
read() Load entire dataset into a container and return it
read_chunked() Return iterator over container fragments of data source
read_partition(i) Return a (offset_tuple, container) corresponding to i-th partition.
to_dask() Make single-partition lazy dask data-frame
to_dask()[source]

Make single-partition lazy dask data-frame

class intake_elasticsearch.elasticsearch_seq.ElasticSearchSeqSource(query, qargs, es_kwargs, metadata)[source]

Data source which executes arbitrary queries on ElasticSearch

This is the tabular reader: will return dataframes. Nested return items will become dict-like objects in the output.

Parameters:
query: str

Query to execute. Can either be in Lucene single-line format, or a JSON structured query (presented as text)

qargs: dict

Further parameters to pass to the query, such as set of indexes to consider, filtering, ordering. See http://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search

es_kwargs: dict

Settings for the ES connection, e.g., a simple local connection may be {'host': 'localhost', 'port': 9200}. Other keywords to the Plugin that end up here and are material:

scroll: str

how long the query is live for, default '100m'

size: int

the paging size when downloading, default 1000.

metadata: dict

Extra information for this source.

Attributes:
plot

Methods

close() Close open resources corresponding to this data source.
discover() Open resource and populate the source attributes.
read() Load entire dataset into a container and return it
read_chunked() Return iterator over container fragments of data source
read_partition(i) Return a (offset_tuple, container) corresponding to i-th partition.
to_dask() Return a dask container for this data source