Coverage for /home/martinb/.local/share/virtualenvs/camcops/lib/python3.6/site-packages/zope/interface/ro.py : 56%

Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1##############################################################################
2#
3# Copyright (c) 2003 Zope Foundation and Contributors.
4# All Rights Reserved.
5#
6# This software is subject to the provisions of the Zope Public License,
7# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
8# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
9# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
10# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
11# FOR A PARTICULAR PURPOSE.
12#
13##############################################################################
14"""
15Compute a resolution order for an object and its bases.
17.. versionchanged:: 5.0
18 The resolution order is now based on the same C3 order that Python
19 uses for classes. In complex instances of multiple inheritance, this
20 may result in a different ordering.
22 In older versions, the ordering wasn't required to be C3 compliant,
23 and for backwards compatibility, it still isn't. If the ordering
24 isn't C3 compliant (if it is *inconsistent*), zope.interface will
25 make a best guess to try to produce a reasonable resolution order.
26 Still (just as before), the results in such cases may be
27 surprising.
29.. rubric:: Environment Variables
31Due to the change in 5.0, certain environment variables can be used to control errors
32and warnings about inconsistent resolution orders. They are listed in priority order, with
33variables at the bottom generally overriding variables above them.
35ZOPE_INTERFACE_WARN_BAD_IRO
36 If this is set to "1", then if there is at least one inconsistent resolution
37 order discovered, a warning (:class:`InconsistentResolutionOrderWarning`) will
38 be issued. Use the usual warning mechanisms to control this behaviour. The warning
39 text will contain additional information on debugging.
40ZOPE_INTERFACE_TRACK_BAD_IRO
41 If this is set to "1", then zope.interface will log information about each
42 inconsistent resolution order discovered, and keep those details in memory in this module
43 for later inspection.
44ZOPE_INTERFACE_STRICT_IRO
45 If this is set to "1", any attempt to use :func:`ro` that would produce a non-C3
46 ordering will fail by raising :class:`InconsistentResolutionOrderError`.
48.. important::
50 ``ZOPE_INTERFACE_STRICT_IRO`` is intended to become the default in the future.
52There are two environment variables that are independent.
54ZOPE_INTERFACE_LOG_CHANGED_IRO
55 If this is set to "1", then if the C3 resolution order is different from
56 the legacy resolution order for any given object, a message explaining the differences
57 will be logged. This is intended to be used for debugging complicated IROs.
58ZOPE_INTERFACE_USE_LEGACY_IRO
59 If this is set to "1", then the C3 resolution order will *not* be used. The
60 legacy IRO will be used instead. This is a temporary measure and will be removed in the
61 future. It is intended to help during the transition.
62 It implies ``ZOPE_INTERFACE_LOG_CHANGED_IRO``.
64.. rubric:: Debugging Behaviour Changes in zope.interface 5
66Most behaviour changes from zope.interface 4 to 5 are related to
67inconsistent resolution orders. ``ZOPE_INTERFACE_STRICT_IRO`` is the
68most effective tool to find such inconsistent resolution orders, and
69we recommend running your code with this variable set if at all
70possible. Doing so will ensure that all interface resolution orders
71are consistent, and if they're not, will immediately point the way to
72where this is violated.
74Occasionally, however, this may not be enough. This is because in some
75cases, a C3 ordering can be found (the resolution order is fully
76consistent) that is substantially different from the ad-hoc legacy
77ordering. In such cases, you may find that you get an unexpected value
78returned when adapting one or more objects to an interface. To debug
79this, *also* enable ``ZOPE_INTERFACE_LOG_CHANGED_IRO`` and examine the
80output. The main thing to look for is changes in the relative
81positions of interfaces for which there are registered adapters.
82"""
83from __future__ import print_function
84__docformat__ = 'restructuredtext'
86__all__ = [
87 'ro',
88 'InconsistentResolutionOrderError',
89 'InconsistentResolutionOrderWarning',
90]
92__logger = None
94def _logger():
95 global __logger # pylint:disable=global-statement
96 if __logger is None:
97 import logging
98 __logger = logging.getLogger(__name__)
99 return __logger
101def _legacy_mergeOrderings(orderings):
102 """Merge multiple orderings so that within-ordering order is preserved
104 Orderings are constrained in such a way that if an object appears
105 in two or more orderings, then the suffix that begins with the
106 object must be in both orderings.
108 For example:
110 >>> _mergeOrderings([
111 ... ['x', 'y', 'z'],
112 ... ['q', 'z'],
113 ... [1, 3, 5],
114 ... ['z']
115 ... ])
116 ['x', 'y', 'q', 1, 3, 5, 'z']
118 """
120 seen = set()
121 result = []
122 for ordering in reversed(orderings):
123 for o in reversed(ordering):
124 if o not in seen:
125 seen.add(o)
126 result.insert(0, o)
128 return result
130def _legacy_flatten(begin):
131 result = [begin]
132 i = 0
133 for ob in iter(result):
134 i += 1
135 # The recursive calls can be avoided by inserting the base classes
136 # into the dynamically growing list directly after the currently
137 # considered object; the iterator makes sure this will keep working
138 # in the future, since it cannot rely on the length of the list
139 # by definition.
140 result[i:i] = ob.__bases__
141 return result
143def _legacy_ro(ob):
144 return _legacy_mergeOrderings([_legacy_flatten(ob)])
146###
147# Compare base objects using identity, not equality. This matches what
148# the CPython MRO algorithm does, and is *much* faster to boot: that,
149# plus some other small tweaks makes the difference between 25s and 6s
150# in loading 446 plone/zope interface.py modules (1925 InterfaceClass,
151# 1200 Implements, 1100 ClassProvides objects)
152###
155class InconsistentResolutionOrderWarning(PendingDeprecationWarning):
156 """
157 The warning issued when an invalid IRO is requested.
158 """
160class InconsistentResolutionOrderError(TypeError):
161 """
162 The error raised when an invalid IRO is requested in strict mode.
163 """
165 def __init__(self, c3, base_tree_remaining):
166 self.C = c3.leaf
167 base_tree = c3.base_tree
168 self.base_ros = {
169 base: base_tree[i + 1]
170 for i, base in enumerate(self.C.__bases__)
171 }
172 # Unfortunately, this doesn't necessarily directly match
173 # up to any transformation on C.__bases__, because
174 # if any were fully used up, they were removed already.
175 self.base_tree_remaining = base_tree_remaining
177 TypeError.__init__(self)
179 def __str__(self):
180 import pprint
181 return "%s: For object %r.\nBase ROs:\n%s\nConflict Location:\n%s" % (
182 self.__class__.__name__,
183 self.C,
184 pprint.pformat(self.base_ros),
185 pprint.pformat(self.base_tree_remaining),
186 )
189class _NamedBool(int): # cannot actually inherit bool
191 def __new__(cls, val, name):
192 inst = super(cls, _NamedBool).__new__(cls, val)
193 inst.__name__ = name
194 return inst
197class _ClassBoolFromEnv(object):
198 """
199 Non-data descriptor that reads a transformed environment variable
200 as a boolean, and caches the result in the class.
201 """
203 def __get__(self, inst, klass):
204 import os
205 for cls in klass.__mro__:
206 my_name = None
207 for k in dir(klass):
208 if k in cls.__dict__ and cls.__dict__[k] is self:
209 my_name = k
210 break
211 if my_name is not None:
212 break
213 else: # pragma: no cover
214 raise RuntimeError("Unable to find self")
216 env_name = 'ZOPE_INTERFACE_' + my_name
217 val = os.environ.get(env_name, '') == '1'
218 val = _NamedBool(val, my_name)
219 setattr(klass, my_name, val)
220 setattr(klass, 'ORIG_' + my_name, self)
221 return val
224class _StaticMRO(object):
225 # A previously resolved MRO, supplied by the caller.
226 # Used in place of calculating it.
228 had_inconsistency = None # We don't know...
230 def __init__(self, C, mro):
231 self.leaf = C
232 self.__mro = tuple(mro)
234 def mro(self):
235 return list(self.__mro)
238class C3(object):
239 # Holds the shared state during computation of an MRO.
241 @staticmethod
242 def resolver(C, strict, base_mros):
243 strict = strict if strict is not None else C3.STRICT_IRO
244 factory = C3
245 if strict:
246 factory = _StrictC3
247 elif C3.TRACK_BAD_IRO:
248 factory = _TrackingC3
250 memo = {}
251 base_mros = base_mros or {}
252 for base, mro in base_mros.items():
253 assert base in C.__bases__
254 memo[base] = _StaticMRO(base, mro)
256 return factory(C, memo)
258 __mro = None
259 __legacy_ro = None
260 direct_inconsistency = False
262 def __init__(self, C, memo):
263 self.leaf = C
264 self.memo = memo
265 kind = self.__class__
267 base_resolvers = []
268 for base in C.__bases__:
269 if base not in memo:
270 resolver = kind(base, memo)
271 memo[base] = resolver
272 base_resolvers.append(memo[base])
274 self.base_tree = [
275 [C]
276 ] + [
277 memo[base].mro() for base in C.__bases__
278 ] + [
279 list(C.__bases__)
280 ]
282 self.bases_had_inconsistency = any(base.had_inconsistency for base in base_resolvers)
284 if len(C.__bases__) == 1:
285 self.__mro = [C] + memo[C.__bases__[0]].mro()
287 @property
288 def had_inconsistency(self):
289 return self.direct_inconsistency or self.bases_had_inconsistency
291 @property
292 def legacy_ro(self):
293 if self.__legacy_ro is None:
294 self.__legacy_ro = tuple(_legacy_ro(self.leaf))
295 return list(self.__legacy_ro)
297 TRACK_BAD_IRO = _ClassBoolFromEnv()
298 STRICT_IRO = _ClassBoolFromEnv()
299 WARN_BAD_IRO = _ClassBoolFromEnv()
300 LOG_CHANGED_IRO = _ClassBoolFromEnv()
301 USE_LEGACY_IRO = _ClassBoolFromEnv()
302 BAD_IROS = ()
304 def _warn_iro(self):
305 if not self.WARN_BAD_IRO:
306 # For the initial release, one must opt-in to see the warning.
307 # In the future (2021?) seeing at least the first warning will
308 # be the default
309 return
310 import warnings
311 warnings.warn(
312 "An inconsistent resolution order is being requested. "
313 "(Interfaces should follow the Python class rules known as C3.) "
314 "For backwards compatibility, zope.interface will allow this, "
315 "making the best guess it can to produce as meaningful an order as possible. "
316 "In the future this might be an error. Set the warning filter to error, or set "
317 "the environment variable 'ZOPE_INTERFACE_TRACK_BAD_IRO' to '1' and examine "
318 "ro.C3.BAD_IROS to debug, or set 'ZOPE_INTERFACE_STRICT_IRO' to raise exceptions.",
319 InconsistentResolutionOrderWarning,
320 )
322 @staticmethod
323 def _can_choose_base(base, base_tree_remaining):
324 # From C3:
325 # nothead = [s for s in nonemptyseqs if cand in s[1:]]
326 for bases in base_tree_remaining:
327 if not bases or bases[0] is base:
328 continue
330 for b in bases:
331 if b is base:
332 return False
333 return True
335 @staticmethod
336 def _nonempty_bases_ignoring(base_tree, ignoring):
337 return list(filter(None, [
338 [b for b in bases if b is not ignoring]
339 for bases
340 in base_tree
341 ]))
343 def _choose_next_base(self, base_tree_remaining):
344 """
345 Return the next base.
347 The return value will either fit the C3 constraints or be our best
348 guess about what to do. If we cannot guess, this may raise an exception.
349 """
350 base = self._find_next_C3_base(base_tree_remaining)
351 if base is not None:
352 return base
353 return self._guess_next_base(base_tree_remaining)
355 def _find_next_C3_base(self, base_tree_remaining):
356 """
357 Return the next base that fits the constraints, or ``None`` if there isn't one.
358 """
359 for bases in base_tree_remaining:
360 base = bases[0]
361 if self._can_choose_base(base, base_tree_remaining):
362 return base
363 return None
365 class _UseLegacyRO(Exception):
366 pass
368 def _guess_next_base(self, base_tree_remaining):
369 # Narf. We may have an inconsistent order (we won't know for
370 # sure until we check all the bases). Python cannot create
371 # classes like this:
372 #
373 # class B1:
374 # pass
375 # class B2(B1):
376 # pass
377 # class C(B1, B2): # -> TypeError; this is like saying C(B1, B2, B1).
378 # pass
379 #
380 # However, older versions of zope.interface were fine with this order.
381 # A good example is ``providedBy(IOError())``. Because of the way
382 # ``classImplements`` works, it winds up with ``__bases__`` ==
383 # ``[IEnvironmentError, IIOError, IOSError, <implementedBy Exception>]``
384 # (on Python 3). But ``IEnvironmentError`` is a base of both ``IIOError``
385 # and ``IOSError``. Previously, we would get a resolution order of
386 # ``[IIOError, IOSError, IEnvironmentError, IStandardError, IException, Interface]``
387 # but the standard Python algorithm would forbid creating that order entirely.
389 # Unlike Python's MRO, we attempt to resolve the issue. A few
390 # heuristics have been tried. One was:
391 #
392 # Strip off the first (highest priority) base of each direct
393 # base one at a time and seeing if we can come to an agreement
394 # with the other bases. (We're trying for a partial ordering
395 # here.) This often resolves cases (such as the IOSError case
396 # above), and frequently produces the same ordering as the
397 # legacy MRO did. If we looked at all the highest priority
398 # bases and couldn't find any partial ordering, then we strip
399 # them *all* out and begin the C3 step again. We take care not
400 # to promote a common root over all others.
401 #
402 # If we only did the first part, stripped off the first
403 # element of the first item, we could resolve simple cases.
404 # But it tended to fail badly. If we did the whole thing, it
405 # could be extremely painful from a performance perspective
406 # for deep/wide things like Zope's OFS.SimpleItem.Item. Plus,
407 # anytime you get ExtensionClass.Base into the mix, you're
408 # likely to wind up in trouble, because it messes with the MRO
409 # of classes. Sigh.
410 #
411 # So now, we fall back to the old linearization (fast to compute).
412 self._warn_iro()
413 self.direct_inconsistency = InconsistentResolutionOrderError(self, base_tree_remaining)
414 raise self._UseLegacyRO
416 def _merge(self):
417 # Returns a merged *list*.
418 result = self.__mro = []
419 base_tree_remaining = self.base_tree
420 base = None
421 while 1:
422 # Take last picked base out of the base tree wherever it is.
423 # This differs slightly from the standard Python MRO and is needed
424 # because we have no other step that prevents duplicates
425 # from coming in (e.g., in the inconsistent fallback path)
426 base_tree_remaining = self._nonempty_bases_ignoring(base_tree_remaining, base)
428 if not base_tree_remaining:
429 return result
430 try:
431 base = self._choose_next_base(base_tree_remaining)
432 except self._UseLegacyRO:
433 self.__mro = self.legacy_ro
434 return self.legacy_ro
436 result.append(base)
438 def mro(self):
439 if self.__mro is None:
440 self.__mro = tuple(self._merge())
441 return list(self.__mro)
444class _StrictC3(C3):
445 __slots__ = ()
446 def _guess_next_base(self, base_tree_remaining):
447 raise InconsistentResolutionOrderError(self, base_tree_remaining)
450class _TrackingC3(C3):
451 __slots__ = ()
452 def _guess_next_base(self, base_tree_remaining):
453 import traceback
454 bad_iros = C3.BAD_IROS
455 if self.leaf not in bad_iros:
456 if bad_iros == ():
457 import weakref
458 # This is a race condition, but it doesn't matter much.
459 bad_iros = C3.BAD_IROS = weakref.WeakKeyDictionary()
460 bad_iros[self.leaf] = t = (
461 InconsistentResolutionOrderError(self, base_tree_remaining),
462 traceback.format_stack()
463 )
464 _logger().warning("Tracking inconsistent IRO: %s", t[0])
465 return C3._guess_next_base(self, base_tree_remaining)
468class _ROComparison(object):
469 # Exists to compute and print a pretty string comparison
470 # for differing ROs.
471 # Since we're used in a logging context, and may actually never be printed,
472 # this is a class so we can defer computing the diff until asked.
474 # Components we use to build up the comparison report
475 class Item(object):
476 prefix = ' '
477 def __init__(self, item):
478 self.item = item
479 def __str__(self):
480 return "%s%s" % (
481 self.prefix,
482 self.item,
483 )
485 class Deleted(Item):
486 prefix = '- '
488 class Inserted(Item):
489 prefix = '+ '
491 Empty = str
493 class ReplacedBy(object): # pragma: no cover
494 prefix = '- '
495 suffix = ''
496 def __init__(self, chunk, total_count):
497 self.chunk = chunk
498 self.total_count = total_count
500 def __iter__(self):
501 lines = [
502 self.prefix + str(item) + self.suffix
503 for item in self.chunk
504 ]
505 while len(lines) < self.total_count:
506 lines.append('')
508 return iter(lines)
510 class Replacing(ReplacedBy):
511 prefix = "+ "
512 suffix = ''
515 _c3_report = None
516 _legacy_report = None
518 def __init__(self, c3, c3_ro, legacy_ro):
519 self.c3 = c3
520 self.c3_ro = c3_ro
521 self.legacy_ro = legacy_ro
523 def __move(self, from_, to_, chunk, operation):
524 for x in chunk:
525 to_.append(operation(x))
526 from_.append(self.Empty())
528 def _generate_report(self):
529 if self._c3_report is None:
530 import difflib
531 # The opcodes we get describe how to turn 'a' into 'b'. So
532 # the old one (legacy) needs to be first ('a')
533 matcher = difflib.SequenceMatcher(None, self.legacy_ro, self.c3_ro)
534 # The reports are equal length sequences. We're going for a
535 # side-by-side diff.
536 self._c3_report = c3_report = []
537 self._legacy_report = legacy_report = []
538 for opcode, leg1, leg2, c31, c32 in matcher.get_opcodes():
539 c3_chunk = self.c3_ro[c31:c32]
540 legacy_chunk = self.legacy_ro[leg1:leg2]
542 if opcode == 'equal':
543 # Guaranteed same length
544 c3_report.extend((self.Item(x) for x in c3_chunk))
545 legacy_report.extend(self.Item(x) for x in legacy_chunk)
546 if opcode == 'delete':
547 # Guaranteed same length
548 assert not c3_chunk
549 self.__move(c3_report, legacy_report, legacy_chunk, self.Deleted)
550 if opcode == 'insert':
551 # Guaranteed same length
552 assert not legacy_chunk
553 self.__move(legacy_report, c3_report, c3_chunk, self.Inserted)
554 if opcode == 'replace': # pragma: no cover (How do you make it output this?)
555 # Either side could be longer.
556 chunk_size = max(len(c3_chunk), len(legacy_chunk))
557 c3_report.extend(self.Replacing(c3_chunk, chunk_size))
558 legacy_report.extend(self.ReplacedBy(legacy_chunk, chunk_size))
560 return self._c3_report, self._legacy_report
562 @property
563 def _inconsistent_label(self):
564 inconsistent = []
565 if self.c3.direct_inconsistency:
566 inconsistent.append('direct')
567 if self.c3.bases_had_inconsistency:
568 inconsistent.append('bases')
569 return '+'.join(inconsistent) if inconsistent else 'no'
571 def __str__(self):
572 c3_report, legacy_report = self._generate_report()
573 assert len(c3_report) == len(legacy_report)
575 left_lines = [str(x) for x in legacy_report]
576 right_lines = [str(x) for x in c3_report]
578 # We have the same number of lines in the report; this is not
579 # necessarily the same as the number of items in either RO.
580 assert len(left_lines) == len(right_lines)
582 padding = ' ' * 2
583 max_left = max(len(x) for x in left_lines)
584 max_right = max(len(x) for x in right_lines)
586 left_title = 'Legacy RO (len=%s)' % (len(self.legacy_ro),)
588 right_title = 'C3 RO (len=%s; inconsistent=%s)' % (
589 len(self.c3_ro),
590 self._inconsistent_label,
591 )
592 lines = [
593 (padding + left_title.ljust(max_left) + padding + right_title.ljust(max_right)),
594 padding + '=' * (max_left + len(padding) + max_right)
595 ]
596 lines += [
597 padding + left.ljust(max_left) + padding + right
598 for left, right in zip(left_lines, right_lines)
599 ]
601 return '\n'.join(lines)
604# Set to `Interface` once it is defined. This is used to
605# avoid logging false positives about changed ROs.
606_ROOT = None
608def ro(C, strict=None, base_mros=None, log_changed_ro=None, use_legacy_ro=None):
609 """
610 ro(C) -> list
612 Compute the precedence list (mro) according to C3.
614 :return: A fresh `list` object.
616 .. versionchanged:: 5.0.0
617 Add the *strict*, *log_changed_ro* and *use_legacy_ro*
618 keyword arguments. These are provisional and likely to be
619 removed in the future. They are most useful for testing.
620 """
621 # The ``base_mros`` argument is for internal optimization and
622 # not documented.
623 resolver = C3.resolver(C, strict, base_mros)
624 mro = resolver.mro()
626 log_changed = log_changed_ro if log_changed_ro is not None else resolver.LOG_CHANGED_IRO
627 use_legacy = use_legacy_ro if use_legacy_ro is not None else resolver.USE_LEGACY_IRO
629 if log_changed or use_legacy:
630 legacy_ro = resolver.legacy_ro
631 assert isinstance(legacy_ro, list)
632 assert isinstance(mro, list)
633 changed = legacy_ro != mro
634 if changed:
635 # Did only Interface move? The fix for issue #8 made that
636 # somewhat common. It's almost certainly not a problem, though,
637 # so allow ignoring it.
638 legacy_without_root = [x for x in legacy_ro if x is not _ROOT]
639 mro_without_root = [x for x in mro if x is not _ROOT]
640 changed = legacy_without_root != mro_without_root
642 if changed:
643 comparison = _ROComparison(resolver, mro, legacy_ro)
644 _logger().warning(
645 "Object %r has different legacy and C3 MROs:\n%s",
646 C, comparison
647 )
648 if resolver.had_inconsistency and legacy_ro == mro:
649 comparison = _ROComparison(resolver, mro, legacy_ro)
650 _logger().warning(
651 "Object %r had inconsistent IRO and used the legacy RO:\n%s"
652 "\nInconsistency entered at:\n%s",
653 C, comparison, resolver.direct_inconsistency
654 )
655 if use_legacy:
656 return legacy_ro
658 return mro
661def is_consistent(C):
662 """
663 Check if the resolution order for *C*, as computed by :func:`ro`, is consistent
664 according to C3.
665 """
666 return not C3.resolver(C, False, None).had_inconsistency