15.2.5. crate_anon.common.sql


Copyright (C) 2015-2018 Rudolf Cardinal (rudolf@pobox.com).

This file is part of CRATE.

CRATE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CRATE is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CRATE. If not, see <http://www.gnu.org/licenses/>.


crate_anon.common.sql.escape_percent_for_python_dbapi(sql: str) → str[source]

Escapes % by converting it to %%. Use this for SQL within Python where % characters are used for argument placeholders.

crate_anon.common.sql.escape_percent_in_literal(sql: str) → str[source]

Escapes % by converting it to %. Use this for LIKE clauses. http://dev.mysql.com/doc/refman/5.7/en/string-literals.html

crate_anon.common.sql.escape_quote_in_literal(s: str) → str[source]

Escape ‘. We could use ‘’ or ‘. Let’s use . for consistency with percent escaping.

crate_anon.common.sql.escape_sql_string_literal(s: str) → str[source]

Escapes SQL string literal fragments against quotes and parameter substitution.

crate_anon.common.sql.get_column_names(engine: sqlalchemy.engine.base.Engine, tablename: str, to_lower: bool = False, sort: bool = False) → List[str][source]

Reads columns names afresh from the database (in case metadata is out of date).

crate_anon.common.sql.get_first_from_table(parsed: pyparsing.ParseResults, match_db: str = '', match_schema: str = '', match_table: str = '') → crate_anon.common.sql.TableId[source]

Given a set of parsed results from a SELECT statement, returns the (db, schema, table) tuple representing the first table in the FROM clause.

Optionally, the match may be constrained with the match* parameters.

crate_anon.common.sql.get_index_names(engine: sqlalchemy.engine.base.Engine, tablename: str, to_lower: bool = False) → List[str][source]

Reads index names from the database.

crate_anon.common.sql.parser_add_from_tables(parsed: pyparsing.ParseResults, join_info_list: List[crate_anon.common.sql.JoinInfo], grammar: cardinal_pythonlib.sql.sql_grammar.SqlGrammar) → pyparsing.ParseResults[source]

Presupposes at least one table already in the FROM clause.

crate_anon.common.sql.sql_fragment_cast_to_int(expr: str, big: bool = True, dialect: sqlalchemy.engine.interfaces.Dialect = None, viewmaker: crate_anon.common.sql.ViewMaker = None) → str[source]

For Microsoft SQL Server. Conversion to INT:

Note that the regex-like expression supported by LIKE is extremely limited.

The only things supported are:

%   any characters
_   any single character
[]  single character in range or set, e.g. [a-f], [abcdef]
[^] single character NOT in range or set, e.g. [^a-f], [abcdef]

SQL Server does not support a REGEXP command directly.

So the best bet is to have the LIKE clause check for a non-integer:

CASE
    WHEN something LIKE '%[^0-9]%' THEN NULL
    ELSE CAST(something AS BIGINT)
END

… which doesn’t deal with spaces properly, but there you go. Could also strip whitespace left/right:

CASE
    WHEN LTRIM(RTRIM(something)) LIKE '%[^0-9]%' THEN NULL
    ELSE CAST(something AS BIGINT)
END

Only works for positive integers. LTRIM/RTRIM are not ANSI SQL. Nor are unusual LIKE clauses; see http://stackoverflow.com/questions/712580/list-of-special-characters-for-sql-like-clause

The other, for SQL Server 2012 or higher, is TRY_CAST:

TRY_CAST(something AS BIGINT)

… which returns NULL upon failure; see https://msdn.microsoft.com/en-us/library/hh974669.aspx

crate_anon.common.sql.translate_sql_qmark_to_percent(sql: str) → str[source]

MySQL likes ‘?’ as a placeholder. - https://dev.mysql.com/doc/refman/5.7/en/sql-syntax-prepared-statements.html

Python DBAPI allows several: ‘%s’, ‘?’, ‘:1’, ‘:name’, ‘%(name)s’. - https://www.python.org/dev/peps/pep-0249/#paramstyle

Django uses ‘%s’. - https://docs.djangoproject.com/en/1.8/topics/db/sql/

Microsoft like ‘?’, @paramname’, and ‘:paramname’. - https://msdn.microsoft.com/en-us/library/yy6y35y8(v=vs.110).aspx

We need to parse SQL with argument placeholders. - See SqlGrammar classes, particularly: bind_parameter

I prefer ?, because % is used in LIKE clauses, and the databases we’re using like it.

So:

  • We use %s when using cursor.execute() directly, via Django.
  • We use ? when talking to users, and SqlGrammar objects, so that the visual appearance matches what they expect from their database.

This function translates SQL using ? placeholders to SQL using %s placeholders, without breaking literal ‘?’ or ‘%’, e.g. inside string literals.