Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1#!/usr/bin/env python 

2 

3# noinspection HttpUrlsUsage 

4""" 

5camcops_server/cc_modules/cc_cache.py 

6 

7=============================================================================== 

8 

9 Copyright (C) 2012-2020 Rudolf Cardinal (rudolf@pobox.com). 

10 

11 This file is part of CamCOPS. 

12 

13 CamCOPS is free software: you can redistribute it and/or modify 

14 it under the terms of the GNU General Public License as published by 

15 the Free Software Foundation, either version 3 of the License, or 

16 (at your option) any later version. 

17 

18 CamCOPS is distributed in the hope that it will be useful, 

19 but WITHOUT ANY WARRANTY; without even the implied warranty of 

20 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

21 GNU General Public License for more details. 

22 

23 You should have received a copy of the GNU General Public License 

24 along with CamCOPS. If not, see <https://www.gnu.org/licenses/>. 

25 

26=============================================================================== 

27 

28**Cache functions.** 

29 

301. The basic cache objects. 

31 

322. FIX FOR DOGPILE.CACHE FOR DECORATED FUNCTIONS, 2017-07-28 (PLUS SOME OTHER 

33 IMPROVEMENTS). SEE 

34 https://bitbucket.org/zzzeek/dogpile.cache/issues/96/error-in-python-35-with-use-of-deprecated 

35 

36Crash using type-hinted functions under Python 3.5 with dogpile.cache==0.6.4: 

37 

38.. code-block:: none 

39 

40 Traceback (most recent call last): 

41 File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main 

42 "__main__", mod_spec) 

43 File "/usr/lib/python3.5/runpy.py", line 85, in _run_code 

44 exec(code, run_globals) 

45 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 64, in <module> 

46 unit_test_cache() 

47 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 50, in unit_test_cache 

48 def testfunc() -> str: 

49 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/region.py", line 1215, in decorator 

50 key_generator = function_key_generator(namespace, fn) 

51 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/util.py", line 31, in function_key_generator 

52 args = inspect.getargspec(fn) 

53 File "/usr/lib/python3.5/inspect.py", line 1045, in getargspec 

54 raise ValueError("Function has keyword-only arguments or annotations" 

55 ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them 

56 

573. CACHING NOTES 

58 

59- We currently use 'dogpile.cache.memory' as the backend. 

60 This means that for single-process (single-thread or multithreaded) servers, 

61 the cache is unique, but that won't work for multi-process (e.g. Gunicorn) 

62 servers. 

63 

64- That means that in a multiprocess environment it's fine to continue to use a 

65 memory cache for file-based stuff (e.g. configs, XML strings), but not for 

66 database-based stuff (e.g. which ID numbers are valid). 

67 

68- Correct solutions WITH a cache for those database-based things include: 

69 

70 - ignoring Python caching and relying on the MySQL query cache -- but this is 

71 being removed because it's not all that great: 

72 

73 https://mysqlserverteam.com/mysql-8-0-retiring-support-for-the-query-cache/ 

74 

75 - using memcached (via dogpile.cache.pylibmc) 

76 

77 https://www.ubergizmo.com/how-to/install-memcached-windows/ 

78 

79 - using redis (via dogpile.cache.redis and 

80 https://pypi.python.org/pypi/redis/ ) 

81 

82 https://stackoverflow.com/questions/10558465/memcached-vs-redis 

83 https://redis.io/ 

84 https://web.archive.org/web/20120118030804/http://simonwillison.net/static/2010/redis-tutorial/ 

85 http://oldblog.antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html 

86 https://redis.io/topics/security 

87 

88 redis unsupported under Windows: 

89 https://redis.io/download 

90 

91- The other obvious alternative: don't cache such stuff! This may all be 

92 premature optimization. 

93 

94 https://msol.io/blog/tech/youre-probably-wrong-about-caching/ 

95 

96 The actual price is of the order of 0.6-1 ms per query, for the queries 

97 "find me all the ID number definitions" and "fetch the server settings". 

98 

99- The answer is probably: 

100 

101 - continue to use dogpile.cache.memory for simple "fixed" stuff read from 

102 disk; 

103 - continue to use Pyramid's per-request caching mechanism (@reify); 

104 - forget about database caching for now; 

105 - if it becomes a problem later, move to Redis 

106 

107- Therefore: 

108 

109 - there should be no calls to cache_region_static.delete 

110 

111""" # noqa 

112 

113 

114# ============================================================================= 

115# Imports; logging 

116# ============================================================================= 

117 

118from cardinal_pythonlib.dogpile_cache import kw_fkg_allowing_type_hints as fkg 

119from dogpile.cache import make_region 

120 

121# ============================================================================= 

122# The main cache: static for the lifetime of this process. 

123# ============================================================================= 

124 

125cache_region_static = make_region() 

126cache_region_static.configure( 

127 backend='dogpile.cache.memory' 

128 # Consider later: memcached via dogpile.cache.pylibmc 

129) 

130 

131# Can now use: 

132# @cache_region_static.cache_on_arguments(function_key_generator=fkg) 

133 

134# https://stackoverflow.com/questions/44834/can-someone-explain-all-in-python 

135__all__ = ['cache_region_static', 'fkg'] # prevents "Unused import statement"