dataloader module#

akerbp.mlpet.dataloader.DataLoader()

A helper class that performs the data loading part of processing MLPet data.

class akerbp.mlpet.dataloader.DataLoader[source]#

Bases: object

A helper class that performs the data loading part of processing MLPet data. This is an internal class only. It is strictly to be used as a super of the Dataset class.

save_df_to_cls(df: pandas.core.frame.DataFrame) pandas.core.frame.DataFrame[source]#

Simple wrapper function to save a df to the class instance

Parameters

df (DataFrame) – Dataframe to be saved to class instance

Returns

Returns the passed dataframe.

Return type

DataFrame

load_from_cdf(client: cognite.client._cognite_client.CogniteClient, metadata: Dict[str, str], save_as: str = '') pandas.core.frame.DataFrame[source]#

Retrieves data from CDF for the provided metadata config

Parameters
  • client (CogniteClient) – The CDF client object to retrieve data from

  • metadata (dict) – The metadata config to pass to the CDF client

  • save_as (str) – If wanting to save the retrieved data, a filepath can be passed to this arg and the data will be pickled at the provided filepath.

Returns

Returns the data retrieved from CDF.

Return type

DataFrame

load_from_las(filepaths: List[str], **kwargs: Any) pandas.core.frame.DataFrame[source]#

Loads data from las file(s)

Parameters

filepaths (list of strings) – paths to las files

Returns

Returns the data loaded from the provided las files.

Return type

DataFrame

load_from_csv(filepath: str, **kwargs: Any) pandas.core.frame.DataFrame[source]#

Loads data from csv files

Parameters

filepath (string) – path to csv file

Returns

Returns the data loaded from the provided csv file.

Return type

DataFrame

load_from_pickle(filepath: str, **kwargs: Any) pandas.core.frame.DataFrame[source]#

Loads data from pickle files

Parameters

filepath (string) – path to pickle file

Returns

Returns the data loaded from the provided csv file.

Return type

DataFrame

load_from_dict(data_dict: Dict[str, Any], **kwargs: Any) pandas.core.frame.DataFrame[source]#

Loads data from a dictionary

Parameters

data_dict (dict) – dictionary with data

Returns

Returns the data loaded from the provided dictionary.

Return type

DataFrame