Coverage for cc_modules/cc_cache.py: 100%

6 statements  

« prev     ^ index     » next       coverage.py v6.5.0, created at 2022-11-08 23:14 +0000

1#!/usr/bin/env python 

2 

3# noinspection HttpUrlsUsage 

4""" 

5camcops_server/cc_modules/cc_cache.py 

6 

7=============================================================================== 

8 

9 Copyright (C) 2012, University of Cambridge, Department of Psychiatry. 

10 Created by Rudolf Cardinal (rnc1001@cam.ac.uk). 

11 

12 This file is part of CamCOPS. 

13 

14 CamCOPS is free software: you can redistribute it and/or modify 

15 it under the terms of the GNU General Public License as published by 

16 the Free Software Foundation, either version 3 of the License, or 

17 (at your option) any later version. 

18 

19 CamCOPS is distributed in the hope that it will be useful, 

20 but WITHOUT ANY WARRANTY; without even the implied warranty of 

21 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

22 GNU General Public License for more details. 

23 

24 You should have received a copy of the GNU General Public License 

25 along with CamCOPS. If not, see <https://www.gnu.org/licenses/>. 

26 

27=============================================================================== 

28 

29**Cache functions.** 

30 

311. The basic cache objects. 

32 

332. FIX FOR DOGPILE.CACHE FOR DECORATED FUNCTIONS, 2017-07-28 (PLUS SOME OTHER 

34 IMPROVEMENTS). SEE 

35 https://bitbucket.org/zzzeek/dogpile.cache/issues/96/error-in-python-35-with-use-of-deprecated 

36 

37Crash using type-hinted functions under Python 3.5 with dogpile.cache==0.6.4: 

38 

39.. code-block:: none 

40 

41 Traceback (most recent call last): 

42 File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main 

43 "__main__", mod_spec) 

44 File "/usr/lib/python3.5/runpy.py", line 85, in _run_code 

45 exec(code, run_globals) 

46 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 64, in <module> 

47 unit_test_cache() 

48 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 50, in unit_test_cache 

49 def testfunc() -> str: 

50 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/region.py", line 1215, in decorator 

51 key_generator = function_key_generator(namespace, fn) 

52 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/util.py", line 31, in function_key_generator 

53 args = inspect.getargspec(fn) 

54 File "/usr/lib/python3.5/inspect.py", line 1045, in getargspec 

55 raise ValueError("Function has keyword-only arguments or annotations" 

56 ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them 

57 

583. CACHING NOTES 

59 

60- We currently use 'dogpile.cache.memory' as the backend. 

61 This means that for single-process (single-thread or multithreaded) servers, 

62 the cache is unique, but that won't work for multi-process (e.g. Gunicorn) 

63 servers. 

64 

65- That means that in a multiprocess environment it's fine to continue to use a 

66 memory cache for file-based stuff (e.g. configs, XML strings), but not for 

67 database-based stuff (e.g. which ID numbers are valid). 

68 

69- Correct solutions WITH a cache for those database-based things include: 

70 

71 - ignoring Python caching and relying on the MySQL query cache -- but this is 

72 being removed because it's not all that great: 

73 

74 https://mysqlserverteam.com/mysql-8-0-retiring-support-for-the-query-cache/ 

75 

76 - using memcached (via dogpile.cache.pylibmc) 

77 

78 https://www.ubergizmo.com/how-to/install-memcached-windows/ 

79 

80 - using redis (via dogpile.cache.redis and 

81 https://pypi.python.org/pypi/redis/ ) 

82 

83 https://stackoverflow.com/questions/10558465/memcached-vs-redis 

84 https://redis.io/ 

85 https://web.archive.org/web/20120118030804/http://simonwillison.net/static/2010/redis-tutorial/ 

86 http://oldblog.antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html 

87 https://redis.io/topics/security 

88 

89 redis unsupported under Windows: 

90 https://redis.io/download 

91 

92- The other obvious alternative: don't cache such stuff! This may all be 

93 premature optimization. 

94 

95 https://msol.io/blog/tech/youre-probably-wrong-about-caching/ 

96 

97 The actual price is of the order of 0.6-1 ms per query, for the queries 

98 "find me all the ID number definitions" and "fetch the server settings". 

99 

100- The answer is probably: 

101 

102 - continue to use dogpile.cache.memory for simple "fixed" stuff read from 

103 disk; 

104 - continue to use Pyramid's per-request caching mechanism (@reify); 

105 - forget about database caching for now; 

106 - if it becomes a problem later, move to Redis 

107 

108- Therefore: 

109 

110 - there should be no calls to cache_region_static.delete 

111 

112""" # noqa 

113 

114 

115# ============================================================================= 

116# Imports; logging 

117# ============================================================================= 

118 

119from cardinal_pythonlib.dogpile_cache import kw_fkg_allowing_type_hints as fkg 

120from dogpile.cache import make_region 

121 

122 

123# ============================================================================= 

124# The main cache: static for the lifetime of this process. 

125# ============================================================================= 

126 

127cache_region_static = make_region() 

128cache_region_static.configure( 

129 backend="dogpile.cache.memory" 

130 # Consider later: memcached via dogpile.cache.pylibmc 

131) 

132 

133# Can now use: 

134# @cache_region_static.cache_on_arguments(function_key_generator=fkg) 

135 

136# https://stackoverflow.com/questions/44834/can-someone-explain-all-in-python 

137__all__ = ["cache_region_static", "fkg"] # prevents "Unused import statement"