15.1.14. crate_anon.anonymise.make_demo_database¶
Copyright (C) 2015-2018 Rudolf Cardinal (rudolf@pobox.com).
This file is part of CRATE.
CRATE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
CRATE is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with CRATE. If not, see <http://www.gnu.org/licenses/>.
Makes a giant test database for anonymisation testing.
See also:
- http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3751474/
- http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3751474/table/T7/
After anonymisation, check with:
SELECT * FROM anonymous_output.notes WHERE brcid IN (
SELECT brcid
FROM anonymous_mapping.secret_map
WHERE patient_id < 2
);
SELECT * FROM test.patients WHERE patient_id < 2;
-
crate_anon.anonymise.make_demo_database.
compile_blob_mysql
(type_, compiler, **kw)[source]¶ MySQL: http://dev.mysql.com/doc/refman/5.7/en/storage-requirements.html
TINYBLOB: up to 2^8 bytes BLOB: up to 2^16 bytes = 64 KiB MEDIUMBLOB: up to 2^24 bytes = 16 MiB <-- minimum for docs LONGBLOB: up to 2^32 bytes = 4 GiB VARBINARY: up to 65535 = 64 KiB
SQL Server: https://msdn.microsoft.com/en-us/library/ms188362.aspx
BINARY: up to 8000 bytes = 8 KB VARBINARY(MAX): up to 2^31 - 1 bytes = 2 GiB <-- minimum for docs IMAGE: deprecated; up to 2^31 - 1 bytes = 2 GiB https://msdn.microsoft.com/en-us/library/ms187993.aspx
SQL Alchemy:
_Binary: base class LargeBinary: translates to BLOB in MySQL VARBINARY, as an SQL base data type dialects.mysql.base.LONGBLOB dialects.mssql.base.VARBINARY
Therefore, we can take the LargeBinary type and modify it: