Coverage for cc_modules/cc_cache.py: 100%
6 statements
« prev ^ index » next coverage.py v6.5.0, created at 2022-11-08 23:14 +0000
« prev ^ index » next coverage.py v6.5.0, created at 2022-11-08 23:14 +0000
1#!/usr/bin/env python
3# noinspection HttpUrlsUsage
4"""
5camcops_server/cc_modules/cc_cache.py
7===============================================================================
9 Copyright (C) 2012, University of Cambridge, Department of Psychiatry.
10 Created by Rudolf Cardinal (rnc1001@cam.ac.uk).
12 This file is part of CamCOPS.
14 CamCOPS is free software: you can redistribute it and/or modify
15 it under the terms of the GNU General Public License as published by
16 the Free Software Foundation, either version 3 of the License, or
17 (at your option) any later version.
19 CamCOPS is distributed in the hope that it will be useful,
20 but WITHOUT ANY WARRANTY; without even the implied warranty of
21 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
22 GNU General Public License for more details.
24 You should have received a copy of the GNU General Public License
25 along with CamCOPS. If not, see <https://www.gnu.org/licenses/>.
27===============================================================================
29**Cache functions.**
311. The basic cache objects.
332. FIX FOR DOGPILE.CACHE FOR DECORATED FUNCTIONS, 2017-07-28 (PLUS SOME OTHER
34 IMPROVEMENTS). SEE
35 https://bitbucket.org/zzzeek/dogpile.cache/issues/96/error-in-python-35-with-use-of-deprecated
37Crash using type-hinted functions under Python 3.5 with dogpile.cache==0.6.4:
39.. code-block:: none
41 Traceback (most recent call last):
42 File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
43 "__main__", mod_spec)
44 File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
45 exec(code, run_globals)
46 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 64, in <module>
47 unit_test_cache()
48 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 50, in unit_test_cache
49 def testfunc() -> str:
50 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/region.py", line 1215, in decorator
51 key_generator = function_key_generator(namespace, fn)
52 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/util.py", line 31, in function_key_generator
53 args = inspect.getargspec(fn)
54 File "/usr/lib/python3.5/inspect.py", line 1045, in getargspec
55 raise ValueError("Function has keyword-only arguments or annotations"
56 ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them
583. CACHING NOTES
60- We currently use 'dogpile.cache.memory' as the backend.
61 This means that for single-process (single-thread or multithreaded) servers,
62 the cache is unique, but that won't work for multi-process (e.g. Gunicorn)
63 servers.
65- That means that in a multiprocess environment it's fine to continue to use a
66 memory cache for file-based stuff (e.g. configs, XML strings), but not for
67 database-based stuff (e.g. which ID numbers are valid).
69- Correct solutions WITH a cache for those database-based things include:
71 - ignoring Python caching and relying on the MySQL query cache -- but this is
72 being removed because it's not all that great:
74 https://mysqlserverteam.com/mysql-8-0-retiring-support-for-the-query-cache/
76 - using memcached (via dogpile.cache.pylibmc)
78 https://www.ubergizmo.com/how-to/install-memcached-windows/
80 - using redis (via dogpile.cache.redis and
81 https://pypi.python.org/pypi/redis/ )
83 https://stackoverflow.com/questions/10558465/memcached-vs-redis
84 https://redis.io/
85 https://web.archive.org/web/20120118030804/http://simonwillison.net/static/2010/redis-tutorial/
86 http://oldblog.antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html
87 https://redis.io/topics/security
89 redis unsupported under Windows:
90 https://redis.io/download
92- The other obvious alternative: don't cache such stuff! This may all be
93 premature optimization.
95 https://msol.io/blog/tech/youre-probably-wrong-about-caching/
97 The actual price is of the order of 0.6-1 ms per query, for the queries
98 "find me all the ID number definitions" and "fetch the server settings".
100- The answer is probably:
102 - continue to use dogpile.cache.memory for simple "fixed" stuff read from
103 disk;
104 - continue to use Pyramid's per-request caching mechanism (@reify);
105 - forget about database caching for now;
106 - if it becomes a problem later, move to Redis
108- Therefore:
110 - there should be no calls to cache_region_static.delete
112""" # noqa
115# =============================================================================
116# Imports; logging
117# =============================================================================
119from cardinal_pythonlib.dogpile_cache import kw_fkg_allowing_type_hints as fkg
120from dogpile.cache import make_region
123# =============================================================================
124# The main cache: static for the lifetime of this process.
125# =============================================================================
127cache_region_static = make_region()
128cache_region_static.configure(
129 backend="dogpile.cache.memory"
130 # Consider later: memcached via dogpile.cache.pylibmc
131)
133# Can now use:
134# @cache_region_static.cache_on_arguments(function_key_generator=fkg)
136# https://stackoverflow.com/questions/44834/can-someone-explain-all-in-python
137__all__ = ["cache_region_static", "fkg"] # prevents "Unused import statement"