dataloader module#

akerbp.mlpet.dataloader.DataLoader()

A helper class that performs the data loading part of processing MLPet data.

class akerbp.mlpet.dataloader.DataLoader[source]#

Bases: object

A helper class that performs the data loading part of processing MLPet data. This is an internal class only. It is strictly to be used as a super of the Dataset class.

save_df_to_cls(df: pandas.core.frame.DataFrame) → pandas.core.frame.DataFrame[source]#

Simple wrapper function to save a df to the class instance

Parameters: df (DataFrame) – Dataframe to be saved to class instance
Returns: Returns the passed dataframe.
Return type: DataFrame

load_from_cdf(client: cognite.client._cognite_client.CogniteClient, metadata: Dict[str, str], save_as: str = '') → pandas.core.frame.DataFrame[source]#

Retrieves data from CDF for the provided metadata config

Parameters

client (CogniteClient) – The CDF client object to retrieve data from
metadata (dict) – The metadata config to pass to the CDF client
save_as (str) – If wanting to save the retrieved data, a filepath can be passed to this arg and the data will be pickled at the provided filepath.

Returns

Returns the data retrieved from CDF.

Return type

DataFrame

load_from_las(filepaths: List[str], **kwargs: Any) → pandas.core.frame.DataFrame[source]#

Loads data from las file(s)

Parameters: filepaths (list of strings) – paths to las files
Returns: Returns the data loaded from the provided las files.
Return type: DataFrame

load_from_csv(filepath: str, **kwargs: Any) → pandas.core.frame.DataFrame[source]#

Loads data from csv files

Parameters: filepath (string) – path to csv file
Returns: Returns the data loaded from the provided csv file.
Return type: DataFrame

load_from_pickle(filepath: str, **kwargs: Any) → pandas.core.frame.DataFrame[source]#

Loads data from pickle files

Parameters: filepath (string) – path to pickle file
Returns: Returns the data loaded from the provided csv file.
Return type: DataFrame

load_from_dict(data_dict: Dict[str, Any], **kwargs: Any) → pandas.core.frame.DataFrame[source]#

Loads data from a dictionary

Parameters: data_dict (dict) – dictionary with data
Returns: Returns the data loaded from the provided dictionary.
Return type: DataFrame