kedro.io.SQLTableDataSet

class kedro.io.SQLTableDataSet(table_name, credentials, load_args=None, save_args=None)[source]

SQLTableDataSet loads data from a SQL table and saves a pandas dataframe to a table. It uses pandas.DataFrame internally, so it supports all allowed pandas options on read_sql_table and to_sql methods. However, it modifies the save parameters and stores the data with no index. This is designed to make load and save methods symmetric.

Example:

from kedro.io import SQLTableDataSet
import pandas as pd

data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5],
                     'col3': [5, 6]})
table_name="table_a"
credentials = {
         con: "postgresql://scott:tiger@localhost/test"
}
data_set = SQLTableDataSet(table_name=table_name,
                           credentials=credentials)

data_set.save(data)
reloaded = data_set.load()

assert data.equals(reloaded)
__init__(table_name, credentials, load_args=None, save_args=None)[source]

Creates a new SQLTableDataSet.

Parameters:
  • table_name (str) – The table name to load or save data to. It overwrites name in save_args and table_name parameters in load_args.
  • credentials (Dict[str, Any]) – A dictionary with a SQLAlchemy connection string. Users are supposed to provide the connection string ‘con’ through credentials. It overwrites con parameter in load_args and save_args in case it is provided.
  • load_args (Optional[Dict[str, Any]]) – Provided to underlying pandas read_sql_table function along with the connection string. To find all supported arguments, see here: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html
  • save_args (Optional[Dict[str, Any]]) – Provided to underlying pandas to_sql function along with the connection string. To find all supported arguments, see here: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_sql.html It has index=False in the default parameters.
Raises:

DataSetError – When either table_name or con is empty.

Return type:

None

Methods

__init__(table_name, credentials[, …]) Creates a new SQLTableDataSet.
exists() Checks whether a data set’s output already exists by calling the provided _exists() method.
from_config(name, config[, load_version, …]) Create a data set instance using the configuration provided.
load() Loads data by delegation to the provided load method.
save(data) Saves data by delegation to the provided save method.