SQLAlchemy 0.4 Documentation

Multiple Pages | One Page
Version: 0.4 Last Updated: 08/12/07 12:43:00

module sqlalchemy.engine

defines the basic components used to interface DBAPI modules with higher-level statement-construction, connection-management, execution and result contexts. The primary "entry point" class into this package is the Engine.

The package is represented among several individual modules, including:

base.py
Defines interface classes and some implementation classes which comprise the basic components used to interface between a DBAPI, constructed and plain-text statements, connections, transactions, and results.
default.py
Contains default implementations of some of the components defined in base.py. All current database dialects use the classes in default.py as base classes for their own database-specific implementations.
strategies.py
the mechanics of constructing Engine objects are represented here. Defines the EngineStrategy class which represents how to go from arguments specified to the create_engine() function, to a fully constructed Engine, including initialization of connection pooling, dialects, and specific subclasses of Engine.
threadlocal.py
the TLEngine class is defined here, which is a subclass of the generic Engine and tracks Connection and Transaction objects against the identity of the current thread. This allows certain programming patterns based around the concept of a "thread-local connection" to be possible. The TLEngine is created by using the "threadlocal" engine strategy in conjunction with the create_engine() function.
url.py
Defines the URL class which represents the individual components of a string URL passed to create_engine(). Also defines a basic module-loading strategy for the dialect specifier within a URL.

Module Functions

def create_engine(*args, **kwargs)

Create a new Engine instance.

The standard method of specifying the engine is via URL as the first positional argument, to indicate the appropriate database dialect and connection arguments, with additional keyword arguments sent as options to the dialect and resulting Engine.

The URL is a string in the form dialect://user:password@host/dbname[?key=value..], where dialect is a name such as mysql, oracle, postgres, etc. Alternatively, the URL can be an instance of sqlalchemy.engine.url.URL.

**kwargs represents options to be sent to the Engine itself as well as the components of the Engine, including the Dialect, the ConnectionProvider, and the Pool. A list of common options is as follows:

poolclass
a subclass of sqlalchemy.pool.Pool which will be used to instantiate a connection pool.
pool
an instance of sqlalchemy.pool.DBProxy or sqlalchemy.pool.Pool to be used as the underlying source for connections (DBProxy/Pool is described in the previous section). This argument supercedes "poolclass".
echo
Defaults to False: if True, the Engine will log all statements as well as a repr() of their parameter lists to the engines logger, which defaults to sys.stdout. A Engine instances' echo data member can be modified at any time to turn logging on and off. If set to the string 'debug', result rows will be printed to the standard output as well.
logger
Defaults to None: a file-like object where logging output can be sent, if echo is set to True. This defaults to sys.stdout.
encoding
Defaults to 'utf-8': the encoding to be used when encoding/decoding Unicode strings.
convert_unicode
Defaults to False: true if unicode conversion should be applied to all str types.
module
Defaults to None: this is a reference to a DBAPI2 module to be used instead of the engine's default module. For Postgres, the default is psycopg2, or psycopg1 if 2 cannot be found. For Oracle, its cx_Oracle. For mysql, MySQLdb.
strategy

allows alternate Engine implementations to take effect. Current implementations include plain and threadlocal. The default used by this function is plain.

plain provides support for a Connection object which can be used to execute SQL queries with a specific underlying DBAPI connection.

threadlocal is similar to plain except that it adds support for a thread-local connection and transaction context, which allows a group of engine operations to participate using the same underlying connection and transaction without the need for explicitly passing a single Connection.

def engine_descriptors()

Provide a listing of all the database implementations supported.

This data is provided as a list of dictionaries, where each dictionary contains the following key/value pairs:

name
the name of the engine, suitable for use in the create_engine function
description
a plain description of the engine.
arguments
a dictionary describing the name and description of each parameter used to connect to this engine's underlying DBAPI.

This function is meant for usage in automated configuration tools that wish to query the user for database and connection information.

class BufferedColumnResultProxy(ResultProxy)

ResultProxy that loads all columns into memory each time fetchone() is called. If fetchmany() or fetchall() are called, the full grid of results is fetched. This is to operate with databases where result rows contain "live" results that fall out of scope unless explicitly fetched. Currently this includes just cx_Oracle LOB objects, but this behavior is known to exist in other DBAPIs as well (Pygresql, currently unsupported).

def fetchall(self)
def fetchmany(self, size=None)
back to section top

class BufferedRowResultProxy(ResultProxy)

ResultProxy that buffers the contents of a selection of rows before fetchone() is called. This is to allow the results of cursor.description to be available immediately, when interfacing with a DBAPI that requires rows to be consumed before this information is available (currently psycopg2, when used with server-side cursors).

The pre-fetching behavior fetches only one row initially, and then grows its buffer size by a fixed amount with each successive need for additional rows up to a size of 100.

back to section top

class Compiled(object)

Represent a compiled SQL expression.

The __str__ method of the Compiled object should produce the actual text of the statement. Compiled objects are specific to their underlying database dialect, and also may or may not be specific to the columns referenced within a particular set of bind parameters. In no case should the Compiled object be dependent on the actual values of those bind parameters, even though it may reference those values as defaults.

def __init__(self, dialect, statement, parameters, bind=None)

Construct a new Compiled object.

statement
ClauseElement to be compiled.
parameters
Optional dictionary indicating a set of bind parameters specified with this Compiled object. These parameters are the default values corresponding to the ClauseElement's _BindParamClauses when the Compiled is executed. In the case of an INSERT or UPDATE statement, these parameters will also result in the creation of new _BindParamClause objects for each key and will also affect the generated column list in an INSERT statement and the SET clauses of an UPDATE statement. The keys of the parameter dictionary can either be the string names of columns or _ColumnClause objects.
bind
Optional Engine or Connection to compile this statement against.
def compile(self)

Produce the internal string representation of this element.

def construct_params(self, params)

Return the bind params for this compiled object.

params is a dict of string/object pairs whos values will override bind values compiled in to the statement.

def execute(self, *multiparams, **params)

Execute this compiled object.

def get_params(*args, **kwargs)

Deprecated. Use construct_params(). (supports unicode names)

def scalar(self, *multiparams, **params)

Execute this compiled object and return the result's scalar value.

back to section top

class Connectable(object)

Interface for an object that can provide an Engine and a Connection object which correponds to that Engine.

def contextual_connect(self)

Return a Connection object which may be part of an ongoing context.

def create(self, entity, **kwargs)

Create a table or index given an appropriate schema object.

dialect = property()

Dialect which this Connectable is associated with.

def drop(self, entity, **kwargs)

Drop a table or index given an appropriate schema object.

engine = property()

The Engine which this Connectable is associated with.

def execute(self, object, *multiparams, **params)
back to section top

class Connection(Connectable)

Represent a single DBAPI connection returned from the underlying connection pool.

Provides execution support for string-based SQL statements as well as ClauseElement, Compiled and DefaultGenerator objects. Provides a begin method to return Transaction objects.

The Connection object is not threadsafe.

def __init__(self, engine, connection=None, close_with_result=False, _branch=False)

Construct a new Connection.

def begin(self, nested=False)
def begin_nested(self)
def begin_twophase(self, xid=None)
def close(self)
def commit_prepared(self, xid, recover=False)
def compiler(self, statement, parameters, **kwargs)
def connect(self)

connect() is implemented to return self so that an incoming Engine or Connection object can be treated similarly.

connection = property()

The underlying DBAPI connection managed by this Connection.

def contextual_connect(self, **kwargs)

contextual_connect() is implemented to return self so that an incoming Engine or Connection object can be treated similarly.

def create(self, entity, **kwargs)

Create a Table or Index given an appropriate Schema object.

def default_schema_name(self)
def detach(self)

detach the underlying DBAPI connection from its connection pool.

This Connection instance will remain useable. When closed, the DBAPI connection will be literally closed and not returned to its pool. The pool will typically create a new connection to replace it, once requested.

This method can be used to insulate the rest of an application from a modified state on a connection (such as a transaction isolation level or similar).

dialect = property()

Dialect used by this Connection.

def drop(self, entity, **kwargs)

Drop a Table or Index given an appropriate Schema object.

engine = property()

The Engine with which this Connection is associated.

def execute(self, object, *multiparams, **params)
def in_transaction(self)
def invalidate(self)

invalidate the underying DBAPI connection and immediately close this Connection.

The underlying DBAPI connection is literally closed (if possible), and is discarded. Its source connection pool will typically create a new connection to replace it, once requested.

properties = property()

A set of per-DBAPI connection properties.

def recover_twophase(self)
def reflecttable(self, table, include_columns=None)

Reflect the columns in the given string table name from the database.

def rollback_prepared(self, xid, recover=False)
def run_callable(self, callable_)
def scalar(self, object, *multiparams, **params)
should_close_with_result = property()

Indicates if this Connection should be closed when a corresponding ResultProxy is closed; this is essentially an auto-release mode.

back to section top

class DefaultRunner(SchemaVisitor)

A visitor which accepts ColumnDefault objects, produces the dialect-specific SQL corresponding to their execution, and executes the SQL, returning the result value.

DefaultRunners are used internally by Engines and Dialects. Specific database modules should provide their own subclasses of DefaultRunner to allow database-specific behavior.

def __init__(self, context)

Construct a new DefaultRunner.

dialect = property()
def exec_default_sql(self, default)
def get_column_default(self, column)
def get_column_onupdate(self, column)
def visit_column_default(self, default)
def visit_column_onupdate(self, onupdate)
def visit_passive_default(self, default)

Do nothing.

Passive defaults by definition return None on the app side, and are post-fetched to get the DB-side value.

def visit_sequence(self, seq)

Do nothing.

Sequences are not supported by default.

back to section top

class Dialect(object)

Define the behavior of a specific database/DBAPI.

Any aspect of metadata definition, SQL query generation, execution, result-set handling, or anything else which varies between databases is defined under the general category of the Dialect. The Dialect acts as a factory for other database-specific object implementations including ExecutionContext, Compiled, DefaultGenerator, and TypeEngine.

All Dialects implement the following attributes:

positional
True if the paramstyle for this Dialect is positional
paramstyle
The paramstyle to be used (some DBAPIs support multiple paramstyles)
convert_unicode
True if unicode conversion should be applied to all str types
encoding
type of encoding to use for unicode, usually defaults to 'utf-8'
def compile(self, clauseelement, parameters=None)

Compile the given ClauseElement using this Dialect.

Returns Compiled. A convenience method which flips around the compile() call on ClauseElement.

def compiler(self, statement, parameters)

Return a Compiled object for the given statement/parameters.

The returned object is usually a subclass of ANSICompiler.

def create_connect_args(self, url)

Build DBAPI compatible connection arguments.

Given a URL object, returns a tuple consisting of a *args/**kwargs suitable to send directly to the dbapi's connect function.

def create_execution_context(self, connection, compiled=None, compiled_parameters=None, statement=None, parameters=None)

Return a new ExecutionContext object.

def create_xid(self)

create a two-phase transaction ID.

this id will be passed to do_begin_twophase(), do_rollback_twophase(), do_commit_twophase(). its format is unspecified.

def dbapi_type_map(self)

return a mapping of DBAPI type objects present in this Dialect's DBAPI mapped to TypeEngine implementations used by the dialect.

This is used to apply types to result sets based on the DBAPI types present in cursor.description; it only takes effect for result sets against textual statements where no explicit typemap was present. Constructed SQL statements always have type information explicitly embedded.

def defaultrunner(self, execution_context)

Return a SchemaVisitor instance that can execute defaults.

execution_context
a ExecutionContext to use for statement execution
def do_begin(self, connection)

Provide an implementation of connection.begin(), given a DBAPI connection.

def do_begin_twophase(self, connection, xid)

Begin a two phase transaction on the given connection.

def do_commit(self, connection)

Provide an implementation of connection.commit(), given a DBAPI connection.

def do_commit_twophase(self, connection, xid, is_prepared=True, recover=False)

Commit a two phase transaction on the given connection.

def do_execute(self, cursor, statement, parameters)

Provide an implementation of cursor.execute(statement, parameters).

def do_executemany(self, cursor, statement, parameters)

Provide an implementation of cursor.executemany(statement, parameters).

def do_prepare_twophase(self, connection, xid)

Prepare a two phase transaction on the given connection.

def do_recover_twophase(self, connection)

Recover list of uncommited prepared two phase transaction identifiers on the given connection.

def do_release_savepoint(self, connection, name)

Release the named savepoint on a SQL Alchemy connection.

def do_rollback(self, connection)

Provide an implementation of connection.rollback(), given a DBAPI connection.

def do_rollback_to_savepoint(self, connection, name)

Rollback a SQL Alchemy connection to the named savepoint.

def do_rollback_twophase(self, connection, xid, is_prepared=True, recover=False)

Rollback a two phase transaction on the given connection.

def do_savepoint(self, connection, name)

Create a savepoint with the given name on a SQLAlchemy connection.

def get_default_schema_name(self, connection)

Return the string name of the currently selected schema given a Connection.

def has_sequence(self, connection, sequence_name)

Check the existence of a particular sequence in the database.

Given a Connection object and a string sequence_name, return True if the given sequence exists in the database, False otherwise.

def has_table(self, connection, table_name, schema=None)

Check the existence of a particular table in the database.

Given a Connection object and a string table_name, return True if the given table (possibly within the specified schema) exists in the database, False otherwise.

def is_disconnect(self, e)

Return True if the given DBAPI error indicates an invalid connection

def max_identifier_length(self)

Return the maximum length of identifier names.

Return None if no limit.

def oid_column_name(self, column)

Return the oid column name for this dialect, or None if the dialect can't/won't support OID/ROWID.

The Column instance which represents OID for the query being compiled is passed, so that the dialect can inspect the column and its parent selectable to determine if OID/ROWID is not selected for a particular selectable (i.e. oracle doesnt support ROWID for UNION, GROUP BY, DISTINCT, etc.)

def reflecttable(self, connection, table, include_columns=None)

Load table description from the database.

Given a Connection and a Table object, reflect its columns and properties from the database. If include_columns (a list or set) is specified, limit the autoload to the given column names.

def schemadropper(self, connection, **kwargs)

Return a SchemaVisitor instance that can drop schemas.

connection
a Connection to use for statement execution

schemadropper() is called via the drop() method on Table, Index, and others.

def schemagenerator(self, connection, **kwargs)

Return a SchemaVisitor instance that can generate schemas.

connection
a Connection to use for statement execution

schemagenerator() is called via the create() method on Table, Index, and others.

def server_version_info(self, connection)

Return a tuple of the database's version number.

def supports_alter(self)

return True if the database supports ALTER TABLE.

def supports_sane_rowcount(self)

Indicate whether the dialect properly implements rowcount for UPDATE and DELETE statements.

This was needed for MySQL which had non-standard behavior of rowcount, but this issue has since been resolved.

def supports_unicode_statements(self)

indicate whether the DBAPI can receive SQL statements as Python unicode strings

def type_descriptor(self, typeobj)

Transform the given TypeEngine instance from generic to database-specific.

Subclasses will usually use the adapt_type() method in the types module to make this job easy.

back to section top

class Engine(Connectable)

Connects a Pool, a Dialect and a CompilerFactory together to provide a default implementation of SchemaEngine.

def __init__(self, pool, dialect, url, echo=None)

Construct a new Engine.

def compiler(self, statement, parameters, **kwargs)
def connect(self, **kwargs)

Return a newly allocated Connection object.

def contextual_connect(self, close_with_result=False, **kwargs)

Return a Connection object which may be newly allocated, or may be part of some ongoing context.

This Connection is meant to be used by the various "auto-connecting" operations.

def create(self, entity, connection=None, **kwargs)

Create a table or index within this engine's database connection given a schema.Table object.

dialect = property()

the Dialect in use by this engine.

def dispose(self)
def drop(self, entity, connection=None, **kwargs)

Drop a table or index within this engine's database connection given a schema.Table object.

echo = property()

when True, enable log output for this element.

This has the effect of setting the Python logging level for the namespace of this element's class and object reference. A value of boolean True indicates that the loglevel logging.INFO will be set for the logger, whereas the string value debug will set the loglevel to logging.DEBUG.

engine = property()
def execute(self, statement, *multiparams, **params)
func = property()
def has_table(self, table_name, schema=None)
def log(self, msg)

Log a message using this SQLEngine's logger stream.

name = property()

String name of the Dialect in use by this Engine.

def raw_connection(self)

Return a DBAPI connection.

def reflecttable(self, table, connection=None, include_columns=None)

Given a Table object, reflects its columns and properties from the database.

def run_callable(self, callable_, connection=None, *args, **kwargs)
def scalar(self, statement, *multiparams, **params)
def table_names(self, schema=None, connection=None)

Return a list of all table names available in the database.

schema:
Optional, retrieve names from a non-default schema.
connection:
Optional, use a specified connection. Default is the contextual_connect for this Engine.
def text(self, text, *args, **kwargs)

Return a sql.text() object for performing literal queries.

def transaction(self, callable_, connection=None, *args, **kwargs)

Execute the given function within a transaction boundary.

This is a shortcut for explicitly calling begin() and commit() and optionally rollback() when exceptions are raised. The given *args and **kwargs will be passed to the function, as well as the Connection used in the transaction.

back to section top

class ExecutionContext(object)

A messenger object for a Dialect that corresponds to a single execution.

ExecutionContext should have these datamembers:

connection
Connection object which can be freely used by default value generators to execute SQL. This Connection should reference the same underlying connection/transactional resources of root_connection.
root_connection
Connection object which is the source of this ExecutionContext. This Connection may have close_with_result=True set, in which case it can only be used once.
dialect
dialect which created this ExecutionContext.
cursor
DBAPI cursor procured from the connection
compiled
if passed to constructor, sql.Compiled object being executed
statement
string version of the statement to be executed. Is either passed to the constructor, or must be created from the sql.Compiled object by the time pre_exec() has completed.
parameters
bind parameters passed to the execute() method. for compiled statements, this is a dictionary or list of dictionaries. for textual statements, it should be in a format suitable for the dialect's paramstyle (i.e. dict or list of dicts for non positional, list or list of lists/tuples for positional).
isinsert
True if the statement is an INSERT
isupdate
True if the statement is an UPDATE

The Dialect should provide an ExecutionContext via the create_execution_context() method. The pre_exec and post_exec methods will be called for compiled statements.

def create_cursor(self)

Return a new cursor generated from this ExecutionContext's connection.

Some dialects may wish to change the behavior of connection.cursor(), such as postgres which may return a PG "server side" cursor.

def get_rowcount(self)

Return the count of rows updated/deleted for an UPDATE/DELETE statement.

def last_inserted_ids(self)

Return the list of the primary key values for the last insert statement executed.

This does not apply to straight textual clauses; only to sql.Insert objects compiled against a schema.Table object. The order of items in the list is the same as that of the Table's 'primary_key' attribute.

def last_inserted_params(self)

Return a dictionary of the full parameter dictionary for the last compiled INSERT statement.

Includes any ColumnDefaults or Sequences that were pre-executed.

def last_updated_params(self)

Return a dictionary of the full parameter dictionary for the last compiled UPDATE statement.

Includes any ColumnDefaults that were pre-executed.

def lastrow_has_defaults(self)

Return True if the last row INSERTED via a compiled insert statement contained PassiveDefaults.

The presence of PassiveDefaults indicates that the database inserted data beyond that which we passed to the query programmatically.

def post_execution(self)

Called after the execution of a compiled statement.

If a compiled statement was passed to this ExecutionContext, the last_insert_ids, last_inserted_params, etc. datamembers should be available after this method completes.

def postfetch_cols(self)

return a list of Column objects for which a 'passive' server-side default value was fired off. applies to inserts and updates.

def pre_execution(self)

Called before an execution of a compiled statement.

If a compiled statement was passed to this ExecutionContext, the statement and parameters datamembers must be initialized after this statement is complete.

def result(self)

return a result object corresponding to this ExecutionContext.

Returns a ResultProxy.

back to section top

class NestedTransaction(Transaction)

def __init__(self, connection, parent)

Construct a new NestedTransaction.

back to section top

class ResultProxy(object)

Wraps a DBAPI cursor object to provide easier access to row columns.

Individual columns may be accessed by their integer position, case-insensitive column name, or by schema.Column object. e.g.:

row = fetchone()

col1 = row[0]    # access via integer position

col2 = row['col2']   # access via name

col3 = row[mytable.c.mycol] # access via Column object.

ResultProxy also contains a map of TypeEngine objects and will invoke the appropriate convert_result_value() method before returning columns, as well as the ExecutionContext corresponding to the statement execution. It provides several methods for which to obtain information from the underlying ExecutionContext.

def __init__(self, context)

ResultProxy objects are constructed via the execute() method on SQLEngine.

def close(self)

Close this ResultProxy, and the underlying DBAPI cursor corresponding to the execution.

If this ResultProxy was generated from an implicit execution, the underlying Connection will also be closed (returns the underlying DBAPI connection to the connection pool.)

This method is also called automatically when all result rows are exhausted.

connection = property()
def fetchall(self)

Fetch all rows, just like DBAPI cursor.fetchall().

def fetchmany(self, size=None)

Fetch many rows, just like DBAPI cursor.fetchmany(size=cursor.arraysize).

def fetchone(self)

Fetch one row, just like DBAPI cursor.fetchone().

keys = property()
def last_inserted_ids(self)

Return last_inserted_ids() from the underlying ExecutionContext.

See ExecutionContext for details.

def last_inserted_params(self)

Return last_inserted_params() from the underlying ExecutionContext.

See ExecutionContext for details.

def last_updated_params(self)

Return last_updated_params() from the underlying ExecutionContext.

See ExecutionContext for details.

def lastrow_has_defaults(self)

Return lastrow_has_defaults() from the underlying ExecutionContext.

See ExecutionContext for details.

lastrowid = property()
out_parameters = property()
rowcount = property()
def scalar(self)

Fetch the first column of the first row, and close the result set.

def supports_sane_rowcount(self)

Return supports_sane_rowcount() from the underlying ExecutionContext.

See ExecutionContext for details.

def __iter__(self)
back to section top

class RootTransaction(Transaction)

def __init__(self, connection)

Construct a new RootTransaction.

back to section top

class RowProxy(object)

Proxy a single cursor row for a parent ResultProxy.

Mostly follows "ordered dictionary" behavior, mapping result values to the string-based column name, the integer position of the result in the row, as well as Column instances which can be mapped to the original Columns that produced this result set (for results that correspond to constructed SQL expressions).

def __init__(self, parent, row)

RowProxy objects are constructed by ResultProxy objects.

def close(self)

Close the parent ResultProxy.

def has_key(self, key)

Return True if this RowProxy contains the given key.

def items(self)

Return a list of tuples, each tuple containing a key/value pair.

def keys(self)

Return the list of keys as strings represented by this RowProxy.

def values(self)

Return the values represented by this RowProxy as a list.

def __contains__(self, key)
def __eq__(self, other)
def __getattr__(self, name)
def __getitem__(self, key)
def __iter__(self)
def __len__(self)
back to section top

class SchemaIterator(SchemaVisitor)

A visitor that can gather text into a buffer and execute the contents of the buffer.

def __init__(self, connection)

Construct a new SchemaIterator.

def append(self, s)

Append content to the SchemaIterator's query buffer.

def execute(self)

Execute the contents of the SchemaIterator's buffer.

back to section top

class Transaction(object)

Represent a Transaction in progress.

The Transaction object is not threadsafe.

def __init__(self, connection, parent)

Construct a new Transaction.

def commit(self)
connection = property()

The Connection object referenced by this Transaction

is_active = property()
def rollback(self)
def __enter__(self)
def __exit__(self, type, value, traceback)
back to section top

class TwoPhaseTransaction(Transaction)

def __init__(self, connection, xid)

Construct a new TwoPhaseTransaction.

def commit(self)
def prepare(self)
back to section top
Up: API Documentation | Previous: module sqlalchemy.types | Next: module sqlalchemy.engine.url