Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# orm/collections.py 

2# Copyright (C) 2005-2020 the SQLAlchemy authors and contributors 

3# <see AUTHORS file> 

4# 

5# This module is part of SQLAlchemy and is released under 

6# the MIT License: http://www.opensource.org/licenses/mit-license.php 

7 

8"""Support for collections of mapped entities. 

9 

10The collections package supplies the machinery used to inform the ORM of 

11collection membership changes. An instrumentation via decoration approach is 

12used, allowing arbitrary types (including built-ins) to be used as entity 

13collections without requiring inheritance from a base class. 

14 

15Instrumentation decoration relays membership change events to the 

16:class:`.CollectionAttributeImpl` that is currently managing the collection. 

17The decorators observe function call arguments and return values, tracking 

18entities entering or leaving the collection. Two decorator approaches are 

19provided. One is a bundle of generic decorators that map function arguments 

20and return values to events:: 

21 

22 from sqlalchemy.orm.collections import collection 

23 class MyClass(object): 

24 # ... 

25 

26 @collection.adds(1) 

27 def store(self, item): 

28 self.data.append(item) 

29 

30 @collection.removes_return() 

31 def pop(self): 

32 return self.data.pop() 

33 

34 

35The second approach is a bundle of targeted decorators that wrap appropriate 

36append and remove notifiers around the mutation methods present in the 

37standard Python ``list``, ``set`` and ``dict`` interfaces. These could be 

38specified in terms of generic decorator recipes, but are instead hand-tooled 

39for increased efficiency. The targeted decorators occasionally implement 

40adapter-like behavior, such as mapping bulk-set methods (``extend``, 

41``update``, ``__setslice__``, etc.) into the series of atomic mutation events 

42that the ORM requires. 

43 

44The targeted decorators are used internally for automatic instrumentation of 

45entity collection classes. Every collection class goes through a 

46transformation process roughly like so: 

47 

481. If the class is a built-in, substitute a trivial sub-class 

492. Is this class already instrumented? 

503. Add in generic decorators 

514. Sniff out the collection interface through duck-typing 

525. Add targeted decoration to any undecorated interface method 

53 

54This process modifies the class at runtime, decorating methods and adding some 

55bookkeeping properties. This isn't possible (or desirable) for built-in 

56classes like ``list``, so trivial sub-classes are substituted to hold 

57decoration:: 

58 

59 class InstrumentedList(list): 

60 pass 

61 

62Collection classes can be specified in ``relationship(collection_class=)`` as 

63types or a function that returns an instance. Collection classes are 

64inspected and instrumented during the mapper compilation phase. The 

65collection_class callable will be executed once to produce a specimen 

66instance, and the type of that specimen will be instrumented. Functions that 

67return built-in types like ``lists`` will be adapted to produce instrumented 

68instances. 

69 

70When extending a known type like ``list``, additional decorations are not 

71generally not needed. Odds are, the extension method will delegate to a 

72method that's already instrumented. For example:: 

73 

74 class QueueIsh(list): 

75 def push(self, item): 

76 self.append(item) 

77 def shift(self): 

78 return self.pop(0) 

79 

80There's no need to decorate these methods. ``append`` and ``pop`` are already 

81instrumented as part of the ``list`` interface. Decorating them would fire 

82duplicate events, which should be avoided. 

83 

84The targeted decoration tries not to rely on other methods in the underlying 

85collection class, but some are unavoidable. Many depend on 'read' methods 

86being present to properly instrument a 'write', for example, ``__setitem__`` 

87needs ``__getitem__``. "Bulk" methods like ``update`` and ``extend`` may also 

88reimplemented in terms of atomic appends and removes, so the ``extend`` 

89decoration will actually perform many ``append`` operations and not call the 

90underlying method at all. 

91 

92Tight control over bulk operation and the firing of events is also possible by 

93implementing the instrumentation internally in your methods. The basic 

94instrumentation package works under the general assumption that collection 

95mutation will not raise unusual exceptions. If you want to closely 

96orchestrate append and remove events with exception management, internal 

97instrumentation may be the answer. Within your method, 

98``collection_adapter(self)`` will retrieve an object that you can use for 

99explicit control over triggering append and remove events. 

100 

101The owning object and :class:`.CollectionAttributeImpl` are also reachable 

102through the adapter, allowing for some very sophisticated behavior. 

103 

104""" 

105 

106import operator 

107import weakref 

108 

109from sqlalchemy.util.compat import inspect_getfullargspec 

110from . import base 

111from .. import exc as sa_exc 

112from .. import util 

113from ..sql import expression 

114 

115 

116__all__ = [ 

117 "collection", 

118 "collection_adapter", 

119 "mapped_collection", 

120 "column_mapped_collection", 

121 "attribute_mapped_collection", 

122] 

123 

124__instrumentation_mutex = util.threading.Lock() 

125 

126 

127class _PlainColumnGetter(object): 

128 """Plain column getter, stores collection of Column objects 

129 directly. 

130 

131 Serializes to a :class:`._SerializableColumnGetterV2` 

132 which has more expensive __call__() performance 

133 and some rare caveats. 

134 

135 """ 

136 

137 def __init__(self, cols): 

138 self.cols = cols 

139 self.composite = len(cols) > 1 

140 

141 def __reduce__(self): 

142 return _SerializableColumnGetterV2._reduce_from_cols(self.cols) 

143 

144 def _cols(self, mapper): 

145 return self.cols 

146 

147 def __call__(self, value): 

148 state = base.instance_state(value) 

149 m = base._state_mapper(state) 

150 

151 key = [ 

152 m._get_state_attr_by_column(state, state.dict, col) 

153 for col in self._cols(m) 

154 ] 

155 

156 if self.composite: 

157 return tuple(key) 

158 else: 

159 return key[0] 

160 

161 

162class _SerializableColumnGetter(object): 

163 """Column-based getter used in version 0.7.6 only. 

164 

165 Remains here for pickle compatibility with 0.7.6. 

166 

167 """ 

168 

169 def __init__(self, colkeys): 

170 self.colkeys = colkeys 

171 self.composite = len(colkeys) > 1 

172 

173 def __reduce__(self): 

174 return _SerializableColumnGetter, (self.colkeys,) 

175 

176 def __call__(self, value): 

177 state = base.instance_state(value) 

178 m = base._state_mapper(state) 

179 key = [ 

180 m._get_state_attr_by_column( 

181 state, state.dict, m.mapped_table.columns[k] 

182 ) 

183 for k in self.colkeys 

184 ] 

185 if self.composite: 

186 return tuple(key) 

187 else: 

188 return key[0] 

189 

190 

191class _SerializableColumnGetterV2(_PlainColumnGetter): 

192 """Updated serializable getter which deals with 

193 multi-table mapped classes. 

194 

195 Two extremely unusual cases are not supported. 

196 Mappings which have tables across multiple metadata 

197 objects, or which are mapped to non-Table selectables 

198 linked across inheriting mappers may fail to function 

199 here. 

200 

201 """ 

202 

203 def __init__(self, colkeys): 

204 self.colkeys = colkeys 

205 self.composite = len(colkeys) > 1 

206 

207 def __reduce__(self): 

208 return self.__class__, (self.colkeys,) 

209 

210 @classmethod 

211 def _reduce_from_cols(cls, cols): 

212 def _table_key(c): 

213 if not isinstance(c.table, expression.TableClause): 

214 return None 

215 else: 

216 return c.table.key 

217 

218 colkeys = [(c.key, _table_key(c)) for c in cols] 

219 return _SerializableColumnGetterV2, (colkeys,) 

220 

221 def _cols(self, mapper): 

222 cols = [] 

223 metadata = getattr(mapper.local_table, "metadata", None) 

224 for (ckey, tkey) in self.colkeys: 

225 if tkey is None or metadata is None or tkey not in metadata: 

226 cols.append(mapper.local_table.c[ckey]) 

227 else: 

228 cols.append(metadata.tables[tkey].c[ckey]) 

229 return cols 

230 

231 

232def column_mapped_collection(mapping_spec): 

233 """A dictionary-based collection type with column-based keying. 

234 

235 Returns a :class:`.MappedCollection` factory with a keying function 

236 generated from mapping_spec, which may be a Column or a sequence 

237 of Columns. 

238 

239 The key value must be immutable for the lifetime of the object. You 

240 can not, for example, map on foreign key values if those key values will 

241 change during the session, i.e. from None to a database-assigned integer 

242 after a session flush. 

243 

244 """ 

245 cols = [ 

246 expression._only_column_elements(q, "mapping_spec") 

247 for q in util.to_list(mapping_spec) 

248 ] 

249 keyfunc = _PlainColumnGetter(cols) 

250 return lambda: MappedCollection(keyfunc) 

251 

252 

253class _SerializableAttrGetter(object): 

254 def __init__(self, name): 

255 self.name = name 

256 self.getter = operator.attrgetter(name) 

257 

258 def __call__(self, target): 

259 return self.getter(target) 

260 

261 def __reduce__(self): 

262 return _SerializableAttrGetter, (self.name,) 

263 

264 

265def attribute_mapped_collection(attr_name): 

266 """A dictionary-based collection type with attribute-based keying. 

267 

268 Returns a :class:`.MappedCollection` factory with a keying based on the 

269 'attr_name' attribute of entities in the collection, where ``attr_name`` 

270 is the string name of the attribute. 

271 

272 The key value must be immutable for the lifetime of the object. You 

273 can not, for example, map on foreign key values if those key values will 

274 change during the session, i.e. from None to a database-assigned integer 

275 after a session flush. 

276 

277 """ 

278 getter = _SerializableAttrGetter(attr_name) 

279 return lambda: MappedCollection(getter) 

280 

281 

282def mapped_collection(keyfunc): 

283 """A dictionary-based collection type with arbitrary keying. 

284 

285 Returns a :class:`.MappedCollection` factory with a keying function 

286 generated from keyfunc, a callable that takes an entity and returns a 

287 key value. 

288 

289 The key value must be immutable for the lifetime of the object. You 

290 can not, for example, map on foreign key values if those key values will 

291 change during the session, i.e. from None to a database-assigned integer 

292 after a session flush. 

293 

294 """ 

295 return lambda: MappedCollection(keyfunc) 

296 

297 

298class collection(object): 

299 """Decorators for entity collection classes. 

300 

301 The decorators fall into two groups: annotations and interception recipes. 

302 

303 The annotating decorators (appender, remover, iterator, linker, converter, 

304 internally_instrumented) indicate the method's purpose and take no 

305 arguments. They are not written with parens:: 

306 

307 @collection.appender 

308 def append(self, append): ... 

309 

310 The recipe decorators all require parens, even those that take no 

311 arguments:: 

312 

313 @collection.adds('entity') 

314 def insert(self, position, entity): ... 

315 

316 @collection.removes_return() 

317 def popitem(self): ... 

318 

319 """ 

320 

321 # Bundled as a class solely for ease of use: packaging, doc strings, 

322 # importability. 

323 

324 @staticmethod 

325 def appender(fn): 

326 """Tag the method as the collection appender. 

327 

328 The appender method is called with one positional argument: the value 

329 to append. The method will be automatically decorated with 'adds(1)' 

330 if not already decorated:: 

331 

332 @collection.appender 

333 def add(self, append): ... 

334 

335 # or, equivalently 

336 @collection.appender 

337 @collection.adds(1) 

338 def add(self, append): ... 

339 

340 # for mapping type, an 'append' may kick out a previous value 

341 # that occupies that slot. consider d['a'] = 'foo'- any previous 

342 # value in d['a'] is discarded. 

343 @collection.appender 

344 @collection.replaces(1) 

345 def add(self, entity): 

346 key = some_key_func(entity) 

347 previous = None 

348 if key in self: 

349 previous = self[key] 

350 self[key] = entity 

351 return previous 

352 

353 If the value to append is not allowed in the collection, you may 

354 raise an exception. Something to remember is that the appender 

355 will be called for each object mapped by a database query. If the 

356 database contains rows that violate your collection semantics, you 

357 will need to get creative to fix the problem, as access via the 

358 collection will not work. 

359 

360 If the appender method is internally instrumented, you must also 

361 receive the keyword argument '_sa_initiator' and ensure its 

362 promulgation to collection events. 

363 

364 """ 

365 fn._sa_instrument_role = "appender" 

366 return fn 

367 

368 @staticmethod 

369 def remover(fn): 

370 """Tag the method as the collection remover. 

371 

372 The remover method is called with one positional argument: the value 

373 to remove. The method will be automatically decorated with 

374 :meth:`removes_return` if not already decorated:: 

375 

376 @collection.remover 

377 def zap(self, entity): ... 

378 

379 # or, equivalently 

380 @collection.remover 

381 @collection.removes_return() 

382 def zap(self, ): ... 

383 

384 If the value to remove is not present in the collection, you may 

385 raise an exception or return None to ignore the error. 

386 

387 If the remove method is internally instrumented, you must also 

388 receive the keyword argument '_sa_initiator' and ensure its 

389 promulgation to collection events. 

390 

391 """ 

392 fn._sa_instrument_role = "remover" 

393 return fn 

394 

395 @staticmethod 

396 def iterator(fn): 

397 """Tag the method as the collection remover. 

398 

399 The iterator method is called with no arguments. It is expected to 

400 return an iterator over all collection members:: 

401 

402 @collection.iterator 

403 def __iter__(self): ... 

404 

405 """ 

406 fn._sa_instrument_role = "iterator" 

407 return fn 

408 

409 @staticmethod 

410 def internally_instrumented(fn): 

411 """Tag the method as instrumented. 

412 

413 This tag will prevent any decoration from being applied to the 

414 method. Use this if you are orchestrating your own calls to 

415 :func:`.collection_adapter` in one of the basic SQLAlchemy 

416 interface methods, or to prevent an automatic ABC method 

417 decoration from wrapping your implementation:: 

418 

419 # normally an 'extend' method on a list-like class would be 

420 # automatically intercepted and re-implemented in terms of 

421 # SQLAlchemy events and append(). your implementation will 

422 # never be called, unless: 

423 @collection.internally_instrumented 

424 def extend(self, items): ... 

425 

426 """ 

427 fn._sa_instrumented = True 

428 return fn 

429 

430 @staticmethod 

431 @util.deprecated( 

432 "1.0", 

433 "The :meth:`.collection.linker` handler is deprecated and will " 

434 "be removed in a future release. Please refer to the " 

435 ":meth:`.AttributeEvents.init_collection` " 

436 "and :meth:`.AttributeEvents.dispose_collection` event handlers. ", 

437 ) 

438 def linker(fn): 

439 """Tag the method as a "linked to attribute" event handler. 

440 

441 This optional event handler will be called when the collection class 

442 is linked to or unlinked from the InstrumentedAttribute. It is 

443 invoked immediately after the '_sa_adapter' property is set on 

444 the instance. A single argument is passed: the collection adapter 

445 that has been linked, or None if unlinking. 

446 

447 

448 """ 

449 fn._sa_instrument_role = "linker" 

450 return fn 

451 

452 link = linker 

453 """Synonym for :meth:`.collection.linker`. 

454 

455 .. deprecated:: 1.0 - :meth:`.collection.link` is deprecated and will be 

456 removed in a future release. 

457 

458 """ 

459 

460 @staticmethod 

461 @util.deprecated( 

462 "1.3", 

463 "The :meth:`.collection.converter` handler is deprecated and will " 

464 "be removed in a future release. Please refer to the " 

465 ":class:`.AttributeEvents.bulk_replace` listener interface in " 

466 "conjunction with the :func:`.event.listen` function.", 

467 ) 

468 def converter(fn): 

469 """Tag the method as the collection converter. 

470 

471 This optional method will be called when a collection is being 

472 replaced entirely, as in:: 

473 

474 myobj.acollection = [newvalue1, newvalue2] 

475 

476 The converter method will receive the object being assigned and should 

477 return an iterable of values suitable for use by the ``appender`` 

478 method. A converter must not assign values or mutate the collection, 

479 its sole job is to adapt the value the user provides into an iterable 

480 of values for the ORM's use. 

481 

482 The default converter implementation will use duck-typing to do the 

483 conversion. A dict-like collection will be convert into an iterable 

484 of dictionary values, and other types will simply be iterated:: 

485 

486 @collection.converter 

487 def convert(self, other): ... 

488 

489 If the duck-typing of the object does not match the type of this 

490 collection, a TypeError is raised. 

491 

492 Supply an implementation of this method if you want to expand the 

493 range of possible types that can be assigned in bulk or perform 

494 validation on the values about to be assigned. 

495 

496 """ 

497 fn._sa_instrument_role = "converter" 

498 return fn 

499 

500 @staticmethod 

501 def adds(arg): 

502 """Mark the method as adding an entity to the collection. 

503 

504 Adds "add to collection" handling to the method. The decorator 

505 argument indicates which method argument holds the SQLAlchemy-relevant 

506 value. Arguments can be specified positionally (i.e. integer) or by 

507 name:: 

508 

509 @collection.adds(1) 

510 def push(self, item): ... 

511 

512 @collection.adds('entity') 

513 def do_stuff(self, thing, entity=None): ... 

514 

515 """ 

516 

517 def decorator(fn): 

518 fn._sa_instrument_before = ("fire_append_event", arg) 

519 return fn 

520 

521 return decorator 

522 

523 @staticmethod 

524 def replaces(arg): 

525 """Mark the method as replacing an entity in the collection. 

526 

527 Adds "add to collection" and "remove from collection" handling to 

528 the method. The decorator argument indicates which method argument 

529 holds the SQLAlchemy-relevant value to be added, and return value, if 

530 any will be considered the value to remove. 

531 

532 Arguments can be specified positionally (i.e. integer) or by name:: 

533 

534 @collection.replaces(2) 

535 def __setitem__(self, index, item): ... 

536 

537 """ 

538 

539 def decorator(fn): 

540 fn._sa_instrument_before = ("fire_append_event", arg) 

541 fn._sa_instrument_after = "fire_remove_event" 

542 return fn 

543 

544 return decorator 

545 

546 @staticmethod 

547 def removes(arg): 

548 """Mark the method as removing an entity in the collection. 

549 

550 Adds "remove from collection" handling to the method. The decorator 

551 argument indicates which method argument holds the SQLAlchemy-relevant 

552 value to be removed. Arguments can be specified positionally (i.e. 

553 integer) or by name:: 

554 

555 @collection.removes(1) 

556 def zap(self, item): ... 

557 

558 For methods where the value to remove is not known at call-time, use 

559 collection.removes_return. 

560 

561 """ 

562 

563 def decorator(fn): 

564 fn._sa_instrument_before = ("fire_remove_event", arg) 

565 return fn 

566 

567 return decorator 

568 

569 @staticmethod 

570 def removes_return(): 

571 """Mark the method as removing an entity in the collection. 

572 

573 Adds "remove from collection" handling to the method. The return 

574 value of the method, if any, is considered the value to remove. The 

575 method arguments are not inspected:: 

576 

577 @collection.removes_return() 

578 def pop(self): ... 

579 

580 For methods where the value to remove is known at call-time, use 

581 collection.remove. 

582 

583 """ 

584 

585 def decorator(fn): 

586 fn._sa_instrument_after = "fire_remove_event" 

587 return fn 

588 

589 return decorator 

590 

591 

592collection_adapter = operator.attrgetter("_sa_adapter") 

593"""Fetch the :class:`.CollectionAdapter` for a collection.""" 

594 

595 

596class CollectionAdapter(object): 

597 """Bridges between the ORM and arbitrary Python collections. 

598 

599 Proxies base-level collection operations (append, remove, iterate) 

600 to the underlying Python collection, and emits add/remove events for 

601 entities entering or leaving the collection. 

602 

603 The ORM uses :class:`.CollectionAdapter` exclusively for interaction with 

604 entity collections. 

605 

606 

607 """ 

608 

609 __slots__ = ( 

610 "attr", 

611 "_key", 

612 "_data", 

613 "owner_state", 

614 "_converter", 

615 "invalidated", 

616 ) 

617 

618 def __init__(self, attr, owner_state, data): 

619 self.attr = attr 

620 self._key = attr.key 

621 self._data = weakref.ref(data) 

622 self.owner_state = owner_state 

623 data._sa_adapter = self 

624 self._converter = data._sa_converter 

625 self.invalidated = False 

626 

627 def _warn_invalidated(self): 

628 util.warn("This collection has been invalidated.") 

629 

630 @property 

631 def data(self): 

632 "The entity collection being adapted." 

633 return self._data() 

634 

635 @property 

636 def _referenced_by_owner(self): 

637 """return True if the owner state still refers to this collection. 

638 

639 This will return False within a bulk replace operation, 

640 where this collection is the one being replaced. 

641 

642 """ 

643 return self.owner_state.dict[self._key] is self._data() 

644 

645 def bulk_appender(self): 

646 return self._data()._sa_appender 

647 

648 def append_with_event(self, item, initiator=None): 

649 """Add an entity to the collection, firing mutation events.""" 

650 

651 self._data()._sa_appender(item, _sa_initiator=initiator) 

652 

653 def append_without_event(self, item): 

654 """Add or restore an entity to the collection, firing no events.""" 

655 self._data()._sa_appender(item, _sa_initiator=False) 

656 

657 def append_multiple_without_event(self, items): 

658 """Add or restore an entity to the collection, firing no events.""" 

659 appender = self._data()._sa_appender 

660 for item in items: 

661 appender(item, _sa_initiator=False) 

662 

663 def bulk_remover(self): 

664 return self._data()._sa_remover 

665 

666 def remove_with_event(self, item, initiator=None): 

667 """Remove an entity from the collection, firing mutation events.""" 

668 self._data()._sa_remover(item, _sa_initiator=initiator) 

669 

670 def remove_without_event(self, item): 

671 """Remove an entity from the collection, firing no events.""" 

672 self._data()._sa_remover(item, _sa_initiator=False) 

673 

674 def clear_with_event(self, initiator=None): 

675 """Empty the collection, firing a mutation event for each entity.""" 

676 

677 remover = self._data()._sa_remover 

678 for item in list(self): 

679 remover(item, _sa_initiator=initiator) 

680 

681 def clear_without_event(self): 

682 """Empty the collection, firing no events.""" 

683 

684 remover = self._data()._sa_remover 

685 for item in list(self): 

686 remover(item, _sa_initiator=False) 

687 

688 def __iter__(self): 

689 """Iterate over entities in the collection.""" 

690 

691 return iter(self._data()._sa_iterator()) 

692 

693 def __len__(self): 

694 """Count entities in the collection.""" 

695 return len(list(self._data()._sa_iterator())) 

696 

697 def __bool__(self): 

698 return True 

699 

700 __nonzero__ = __bool__ 

701 

702 def fire_append_event(self, item, initiator=None): 

703 """Notify that a entity has entered the collection. 

704 

705 Initiator is a token owned by the InstrumentedAttribute that 

706 initiated the membership mutation, and should be left as None 

707 unless you are passing along an initiator value from a chained 

708 operation. 

709 

710 """ 

711 if initiator is not False: 

712 if self.invalidated: 

713 self._warn_invalidated() 

714 return self.attr.fire_append_event( 

715 self.owner_state, self.owner_state.dict, item, initiator 

716 ) 

717 else: 

718 return item 

719 

720 def fire_remove_event(self, item, initiator=None): 

721 """Notify that a entity has been removed from the collection. 

722 

723 Initiator is the InstrumentedAttribute that initiated the membership 

724 mutation, and should be left as None unless you are passing along 

725 an initiator value from a chained operation. 

726 

727 """ 

728 if initiator is not False: 

729 if self.invalidated: 

730 self._warn_invalidated() 

731 self.attr.fire_remove_event( 

732 self.owner_state, self.owner_state.dict, item, initiator 

733 ) 

734 

735 def fire_pre_remove_event(self, initiator=None): 

736 """Notify that an entity is about to be removed from the collection. 

737 

738 Only called if the entity cannot be removed after calling 

739 fire_remove_event(). 

740 

741 """ 

742 if self.invalidated: 

743 self._warn_invalidated() 

744 self.attr.fire_pre_remove_event( 

745 self.owner_state, self.owner_state.dict, initiator=initiator 

746 ) 

747 

748 def __getstate__(self): 

749 return { 

750 "key": self._key, 

751 "owner_state": self.owner_state, 

752 "owner_cls": self.owner_state.class_, 

753 "data": self.data, 

754 "invalidated": self.invalidated, 

755 } 

756 

757 def __setstate__(self, d): 

758 self._key = d["key"] 

759 self.owner_state = d["owner_state"] 

760 self._data = weakref.ref(d["data"]) 

761 self._converter = d["data"]._sa_converter 

762 d["data"]._sa_adapter = self 

763 self.invalidated = d["invalidated"] 

764 self.attr = getattr(d["owner_cls"], self._key).impl 

765 

766 

767def bulk_replace(values, existing_adapter, new_adapter, initiator=None): 

768 """Load a new collection, firing events based on prior like membership. 

769 

770 Appends instances in ``values`` onto the ``new_adapter``. Events will be 

771 fired for any instance not present in the ``existing_adapter``. Any 

772 instances in ``existing_adapter`` not present in ``values`` will have 

773 remove events fired upon them. 

774 

775 :param values: An iterable of collection member instances 

776 

777 :param existing_adapter: A :class:`.CollectionAdapter` of 

778 instances to be replaced 

779 

780 :param new_adapter: An empty :class:`.CollectionAdapter` 

781 to load with ``values`` 

782 

783 

784 """ 

785 

786 assert isinstance(values, list) 

787 

788 idset = util.IdentitySet 

789 existing_idset = idset(existing_adapter or ()) 

790 constants = existing_idset.intersection(values or ()) 

791 additions = idset(values or ()).difference(constants) 

792 removals = existing_idset.difference(constants) 

793 

794 appender = new_adapter.bulk_appender() 

795 

796 for member in values or (): 

797 if member in additions: 

798 appender(member, _sa_initiator=initiator) 

799 elif member in constants: 

800 appender(member, _sa_initiator=False) 

801 

802 if existing_adapter: 

803 for member in removals: 

804 existing_adapter.fire_remove_event(member, initiator=initiator) 

805 

806 

807def prepare_instrumentation(factory): 

808 """Prepare a callable for future use as a collection class factory. 

809 

810 Given a collection class factory (either a type or no-arg callable), 

811 return another factory that will produce compatible instances when 

812 called. 

813 

814 This function is responsible for converting collection_class=list 

815 into the run-time behavior of collection_class=InstrumentedList. 

816 

817 """ 

818 # Convert a builtin to 'Instrumented*' 

819 if factory in __canned_instrumentation: 

820 factory = __canned_instrumentation[factory] 

821 

822 # Create a specimen 

823 cls = type(factory()) 

824 

825 # Did factory callable return a builtin? 

826 if cls in __canned_instrumentation: 

827 # Wrap it so that it returns our 'Instrumented*' 

828 factory = __converting_factory(cls, factory) 

829 cls = factory() 

830 

831 # Instrument the class if needed. 

832 if __instrumentation_mutex.acquire(): 

833 try: 

834 if getattr(cls, "_sa_instrumented", None) != id(cls): 

835 _instrument_class(cls) 

836 finally: 

837 __instrumentation_mutex.release() 

838 

839 return factory 

840 

841 

842def __converting_factory(specimen_cls, original_factory): 

843 """Return a wrapper that converts a "canned" collection like 

844 set, dict, list into the Instrumented* version. 

845 

846 """ 

847 

848 instrumented_cls = __canned_instrumentation[specimen_cls] 

849 

850 def wrapper(): 

851 collection = original_factory() 

852 return instrumented_cls(collection) 

853 

854 # often flawed but better than nothing 

855 wrapper.__name__ = "%sWrapper" % original_factory.__name__ 

856 wrapper.__doc__ = original_factory.__doc__ 

857 

858 return wrapper 

859 

860 

861def _instrument_class(cls): 

862 """Modify methods in a class and install instrumentation.""" 

863 

864 # In the normal call flow, a request for any of the 3 basic collection 

865 # types is transformed into one of our trivial subclasses 

866 # (e.g. InstrumentedList). Catch anything else that sneaks in here... 

867 if cls.__module__ == "__builtin__": 

868 raise sa_exc.ArgumentError( 

869 "Can not instrument a built-in type. Use a " 

870 "subclass, even a trivial one." 

871 ) 

872 

873 roles, methods = _locate_roles_and_methods(cls) 

874 

875 _setup_canned_roles(cls, roles, methods) 

876 

877 _assert_required_roles(cls, roles, methods) 

878 

879 _set_collection_attributes(cls, roles, methods) 

880 

881 

882def _locate_roles_and_methods(cls): 

883 """search for _sa_instrument_role-decorated methods in 

884 method resolution order, assign to roles. 

885 

886 """ 

887 

888 roles = {} 

889 methods = {} 

890 

891 for supercls in cls.__mro__: 

892 for name, method in vars(supercls).items(): 

893 if not util.callable(method): 

894 continue 

895 

896 # note role declarations 

897 if hasattr(method, "_sa_instrument_role"): 

898 role = method._sa_instrument_role 

899 assert role in ( 

900 "appender", 

901 "remover", 

902 "iterator", 

903 "linker", 

904 "converter", 

905 ) 

906 roles.setdefault(role, name) 

907 

908 # transfer instrumentation requests from decorated function 

909 # to the combined queue 

910 before, after = None, None 

911 if hasattr(method, "_sa_instrument_before"): 

912 op, argument = method._sa_instrument_before 

913 assert op in ("fire_append_event", "fire_remove_event") 

914 before = op, argument 

915 if hasattr(method, "_sa_instrument_after"): 

916 op = method._sa_instrument_after 

917 assert op in ("fire_append_event", "fire_remove_event") 

918 after = op 

919 if before: 

920 methods[name] = before + (after,) 

921 elif after: 

922 methods[name] = None, None, after 

923 return roles, methods 

924 

925 

926def _setup_canned_roles(cls, roles, methods): 

927 """see if this class has "canned" roles based on a known 

928 collection type (dict, set, list). Apply those roles 

929 as needed to the "roles" dictionary, and also 

930 prepare "decorator" methods 

931 

932 """ 

933 collection_type = util.duck_type_collection(cls) 

934 if collection_type in __interfaces: 

935 canned_roles, decorators = __interfaces[collection_type] 

936 for role, name in canned_roles.items(): 

937 roles.setdefault(role, name) 

938 

939 # apply ABC auto-decoration to methods that need it 

940 for method, decorator in decorators.items(): 

941 fn = getattr(cls, method, None) 

942 if ( 

943 fn 

944 and method not in methods 

945 and not hasattr(fn, "_sa_instrumented") 

946 ): 

947 setattr(cls, method, decorator(fn)) 

948 

949 

950def _assert_required_roles(cls, roles, methods): 

951 """ensure all roles are present, and apply implicit instrumentation if 

952 needed 

953 

954 """ 

955 if "appender" not in roles or not hasattr(cls, roles["appender"]): 

956 raise sa_exc.ArgumentError( 

957 "Type %s must elect an appender method to be " 

958 "a collection class" % cls.__name__ 

959 ) 

960 elif roles["appender"] not in methods and not hasattr( 

961 getattr(cls, roles["appender"]), "_sa_instrumented" 

962 ): 

963 methods[roles["appender"]] = ("fire_append_event", 1, None) 

964 

965 if "remover" not in roles or not hasattr(cls, roles["remover"]): 

966 raise sa_exc.ArgumentError( 

967 "Type %s must elect a remover method to be " 

968 "a collection class" % cls.__name__ 

969 ) 

970 elif roles["remover"] not in methods and not hasattr( 

971 getattr(cls, roles["remover"]), "_sa_instrumented" 

972 ): 

973 methods[roles["remover"]] = ("fire_remove_event", 1, None) 

974 

975 if "iterator" not in roles or not hasattr(cls, roles["iterator"]): 

976 raise sa_exc.ArgumentError( 

977 "Type %s must elect an iterator method to be " 

978 "a collection class" % cls.__name__ 

979 ) 

980 

981 

982def _set_collection_attributes(cls, roles, methods): 

983 """apply ad-hoc instrumentation from decorators, class-level defaults 

984 and implicit role declarations 

985 

986 """ 

987 for method_name, (before, argument, after) in methods.items(): 

988 setattr( 

989 cls, 

990 method_name, 

991 _instrument_membership_mutator( 

992 getattr(cls, method_name), before, argument, after 

993 ), 

994 ) 

995 # intern the role map 

996 for role, method_name in roles.items(): 

997 setattr(cls, "_sa_%s" % role, getattr(cls, method_name)) 

998 

999 cls._sa_adapter = None 

1000 

1001 if not hasattr(cls, "_sa_converter"): 

1002 cls._sa_converter = None 

1003 cls._sa_instrumented = id(cls) 

1004 

1005 

1006def _instrument_membership_mutator(method, before, argument, after): 

1007 """Route method args and/or return value through the collection 

1008 adapter.""" 

1009 # This isn't smart enough to handle @adds(1) for 'def fn(self, (a, b))' 

1010 if before: 

1011 fn_args = list( 

1012 util.flatten_iterator(inspect_getfullargspec(method)[0]) 

1013 ) 

1014 if isinstance(argument, int): 

1015 pos_arg = argument 

1016 named_arg = len(fn_args) > argument and fn_args[argument] or None 

1017 else: 

1018 if argument in fn_args: 

1019 pos_arg = fn_args.index(argument) 

1020 else: 

1021 pos_arg = None 

1022 named_arg = argument 

1023 del fn_args 

1024 

1025 def wrapper(*args, **kw): 

1026 if before: 

1027 if pos_arg is None: 

1028 if named_arg not in kw: 

1029 raise sa_exc.ArgumentError( 

1030 "Missing argument %s" % argument 

1031 ) 

1032 value = kw[named_arg] 

1033 else: 

1034 if len(args) > pos_arg: 

1035 value = args[pos_arg] 

1036 elif named_arg in kw: 

1037 value = kw[named_arg] 

1038 else: 

1039 raise sa_exc.ArgumentError( 

1040 "Missing argument %s" % argument 

1041 ) 

1042 

1043 initiator = kw.pop("_sa_initiator", None) 

1044 if initiator is False: 

1045 executor = None 

1046 else: 

1047 executor = args[0]._sa_adapter 

1048 

1049 if before and executor: 

1050 getattr(executor, before)(value, initiator) 

1051 

1052 if not after or not executor: 

1053 return method(*args, **kw) 

1054 else: 

1055 res = method(*args, **kw) 

1056 if res is not None: 

1057 getattr(executor, after)(res, initiator) 

1058 return res 

1059 

1060 wrapper._sa_instrumented = True 

1061 if hasattr(method, "_sa_instrument_role"): 

1062 wrapper._sa_instrument_role = method._sa_instrument_role 

1063 wrapper.__name__ = method.__name__ 

1064 wrapper.__doc__ = method.__doc__ 

1065 return wrapper 

1066 

1067 

1068def __set(collection, item, _sa_initiator=None): 

1069 """Run set events. 

1070 

1071 This event always occurs before the collection is actually mutated. 

1072 

1073 """ 

1074 

1075 if _sa_initiator is not False: 

1076 executor = collection._sa_adapter 

1077 if executor: 

1078 item = executor.fire_append_event(item, _sa_initiator) 

1079 return item 

1080 

1081 

1082def __del(collection, item, _sa_initiator=None): 

1083 """Run del events. 

1084 

1085 This event occurs before the collection is actually mutated, *except* 

1086 in the case of a pop operation, in which case it occurs afterwards. 

1087 For pop operations, the __before_pop hook is called before the 

1088 operation occurs. 

1089 

1090 """ 

1091 if _sa_initiator is not False: 

1092 executor = collection._sa_adapter 

1093 if executor: 

1094 executor.fire_remove_event(item, _sa_initiator) 

1095 

1096 

1097def __before_pop(collection, _sa_initiator=None): 

1098 """An event which occurs on a before a pop() operation occurs.""" 

1099 executor = collection._sa_adapter 

1100 if executor: 

1101 executor.fire_pre_remove_event(_sa_initiator) 

1102 

1103 

1104def _list_decorators(): 

1105 """Tailored instrumentation wrappers for any list-like class.""" 

1106 

1107 def _tidy(fn): 

1108 fn._sa_instrumented = True 

1109 fn.__doc__ = getattr(list, fn.__name__).__doc__ 

1110 

1111 def append(fn): 

1112 def append(self, item, _sa_initiator=None): 

1113 item = __set(self, item, _sa_initiator) 

1114 fn(self, item) 

1115 

1116 _tidy(append) 

1117 return append 

1118 

1119 def remove(fn): 

1120 def remove(self, value, _sa_initiator=None): 

1121 __del(self, value, _sa_initiator) 

1122 # testlib.pragma exempt:__eq__ 

1123 fn(self, value) 

1124 

1125 _tidy(remove) 

1126 return remove 

1127 

1128 def insert(fn): 

1129 def insert(self, index, value): 

1130 value = __set(self, value) 

1131 fn(self, index, value) 

1132 

1133 _tidy(insert) 

1134 return insert 

1135 

1136 def __setitem__(fn): 

1137 def __setitem__(self, index, value): 

1138 if not isinstance(index, slice): 

1139 existing = self[index] 

1140 if existing is not None: 

1141 __del(self, existing) 

1142 value = __set(self, value) 

1143 fn(self, index, value) 

1144 else: 

1145 # slice assignment requires __delitem__, insert, __len__ 

1146 step = index.step or 1 

1147 start = index.start or 0 

1148 if start < 0: 

1149 start += len(self) 

1150 if index.stop is not None: 

1151 stop = index.stop 

1152 else: 

1153 stop = len(self) 

1154 if stop < 0: 

1155 stop += len(self) 

1156 

1157 if step == 1: 

1158 if value is self: 

1159 return 

1160 for i in range(start, stop, step): 

1161 if len(self) > start: 

1162 del self[start] 

1163 

1164 for i, item in enumerate(value): 

1165 self.insert(i + start, item) 

1166 else: 

1167 rng = list(range(start, stop, step)) 

1168 if len(value) != len(rng): 

1169 raise ValueError( 

1170 "attempt to assign sequence of size %s to " 

1171 "extended slice of size %s" 

1172 % (len(value), len(rng)) 

1173 ) 

1174 for i, item in zip(rng, value): 

1175 self.__setitem__(i, item) 

1176 

1177 _tidy(__setitem__) 

1178 return __setitem__ 

1179 

1180 def __delitem__(fn): 

1181 def __delitem__(self, index): 

1182 if not isinstance(index, slice): 

1183 item = self[index] 

1184 __del(self, item) 

1185 fn(self, index) 

1186 else: 

1187 # slice deletion requires __getslice__ and a slice-groking 

1188 # __getitem__ for stepped deletion 

1189 # note: not breaking this into atomic dels 

1190 for item in self[index]: 

1191 __del(self, item) 

1192 fn(self, index) 

1193 

1194 _tidy(__delitem__) 

1195 return __delitem__ 

1196 

1197 if util.py2k: 

1198 

1199 def __setslice__(fn): 

1200 def __setslice__(self, start, end, values): 

1201 for value in self[start:end]: 

1202 __del(self, value) 

1203 values = [__set(self, value) for value in values] 

1204 fn(self, start, end, values) 

1205 

1206 _tidy(__setslice__) 

1207 return __setslice__ 

1208 

1209 def __delslice__(fn): 

1210 def __delslice__(self, start, end): 

1211 for value in self[start:end]: 

1212 __del(self, value) 

1213 fn(self, start, end) 

1214 

1215 _tidy(__delslice__) 

1216 return __delslice__ 

1217 

1218 def extend(fn): 

1219 def extend(self, iterable): 

1220 for value in iterable: 

1221 self.append(value) 

1222 

1223 _tidy(extend) 

1224 return extend 

1225 

1226 def __iadd__(fn): 

1227 def __iadd__(self, iterable): 

1228 # list.__iadd__ takes any iterable and seems to let TypeError 

1229 # raise as-is instead of returning NotImplemented 

1230 for value in iterable: 

1231 self.append(value) 

1232 return self 

1233 

1234 _tidy(__iadd__) 

1235 return __iadd__ 

1236 

1237 def pop(fn): 

1238 def pop(self, index=-1): 

1239 __before_pop(self) 

1240 item = fn(self, index) 

1241 __del(self, item) 

1242 return item 

1243 

1244 _tidy(pop) 

1245 return pop 

1246 

1247 if not util.py2k: 

1248 

1249 def clear(fn): 

1250 def clear(self, index=-1): 

1251 for item in self: 

1252 __del(self, item) 

1253 fn(self) 

1254 

1255 _tidy(clear) 

1256 return clear 

1257 

1258 # __imul__ : not wrapping this. all members of the collection are already 

1259 # present, so no need to fire appends... wrapping it with an explicit 

1260 # decorator is still possible, so events on *= can be had if they're 

1261 # desired. hard to imagine a use case for __imul__, though. 

1262 

1263 l = locals().copy() 

1264 l.pop("_tidy") 

1265 return l 

1266 

1267 

1268def _dict_decorators(): 

1269 """Tailored instrumentation wrappers for any dict-like mapping class.""" 

1270 

1271 def _tidy(fn): 

1272 fn._sa_instrumented = True 

1273 fn.__doc__ = getattr(dict, fn.__name__).__doc__ 

1274 

1275 Unspecified = util.symbol("Unspecified") 

1276 

1277 def __setitem__(fn): 

1278 def __setitem__(self, key, value, _sa_initiator=None): 

1279 if key in self: 

1280 __del(self, self[key], _sa_initiator) 

1281 value = __set(self, value, _sa_initiator) 

1282 fn(self, key, value) 

1283 

1284 _tidy(__setitem__) 

1285 return __setitem__ 

1286 

1287 def __delitem__(fn): 

1288 def __delitem__(self, key, _sa_initiator=None): 

1289 if key in self: 

1290 __del(self, self[key], _sa_initiator) 

1291 fn(self, key) 

1292 

1293 _tidy(__delitem__) 

1294 return __delitem__ 

1295 

1296 def clear(fn): 

1297 def clear(self): 

1298 for key in self: 

1299 __del(self, self[key]) 

1300 fn(self) 

1301 

1302 _tidy(clear) 

1303 return clear 

1304 

1305 def pop(fn): 

1306 def pop(self, key, default=Unspecified): 

1307 __before_pop(self) 

1308 _to_del = key in self 

1309 if default is Unspecified: 

1310 item = fn(self, key) 

1311 else: 

1312 item = fn(self, key, default) 

1313 if _to_del: 

1314 __del(self, item) 

1315 return item 

1316 

1317 _tidy(pop) 

1318 return pop 

1319 

1320 def popitem(fn): 

1321 def popitem(self): 

1322 __before_pop(self) 

1323 item = fn(self) 

1324 __del(self, item[1]) 

1325 return item 

1326 

1327 _tidy(popitem) 

1328 return popitem 

1329 

1330 def setdefault(fn): 

1331 def setdefault(self, key, default=None): 

1332 if key not in self: 

1333 self.__setitem__(key, default) 

1334 return default 

1335 else: 

1336 return self.__getitem__(key) 

1337 

1338 _tidy(setdefault) 

1339 return setdefault 

1340 

1341 def update(fn): 

1342 def update(self, __other=Unspecified, **kw): 

1343 if __other is not Unspecified: 

1344 if hasattr(__other, "keys"): 

1345 for key in list(__other): 

1346 if key not in self or self[key] is not __other[key]: 

1347 self[key] = __other[key] 

1348 else: 

1349 for key, value in __other: 

1350 if key not in self or self[key] is not value: 

1351 self[key] = value 

1352 for key in kw: 

1353 if key not in self or self[key] is not kw[key]: 

1354 self[key] = kw[key] 

1355 

1356 _tidy(update) 

1357 return update 

1358 

1359 l = locals().copy() 

1360 l.pop("_tidy") 

1361 l.pop("Unspecified") 

1362 return l 

1363 

1364 

1365_set_binop_bases = (set, frozenset) 

1366 

1367 

1368def _set_binops_check_strict(self, obj): 

1369 """Allow only set, frozenset and self.__class__-derived 

1370 objects in binops.""" 

1371 return isinstance(obj, _set_binop_bases + (self.__class__,)) 

1372 

1373 

1374def _set_binops_check_loose(self, obj): 

1375 """Allow anything set-like to participate in set binops.""" 

1376 return ( 

1377 isinstance(obj, _set_binop_bases + (self.__class__,)) 

1378 or util.duck_type_collection(obj) == set 

1379 ) 

1380 

1381 

1382def _set_decorators(): 

1383 """Tailored instrumentation wrappers for any set-like class.""" 

1384 

1385 def _tidy(fn): 

1386 fn._sa_instrumented = True 

1387 fn.__doc__ = getattr(set, fn.__name__).__doc__ 

1388 

1389 Unspecified = util.symbol("Unspecified") 

1390 

1391 def add(fn): 

1392 def add(self, value, _sa_initiator=None): 

1393 if value not in self: 

1394 value = __set(self, value, _sa_initiator) 

1395 # testlib.pragma exempt:__hash__ 

1396 fn(self, value) 

1397 

1398 _tidy(add) 

1399 return add 

1400 

1401 def discard(fn): 

1402 def discard(self, value, _sa_initiator=None): 

1403 # testlib.pragma exempt:__hash__ 

1404 if value in self: 

1405 __del(self, value, _sa_initiator) 

1406 # testlib.pragma exempt:__hash__ 

1407 fn(self, value) 

1408 

1409 _tidy(discard) 

1410 return discard 

1411 

1412 def remove(fn): 

1413 def remove(self, value, _sa_initiator=None): 

1414 # testlib.pragma exempt:__hash__ 

1415 if value in self: 

1416 __del(self, value, _sa_initiator) 

1417 # testlib.pragma exempt:__hash__ 

1418 fn(self, value) 

1419 

1420 _tidy(remove) 

1421 return remove 

1422 

1423 def pop(fn): 

1424 def pop(self): 

1425 __before_pop(self) 

1426 item = fn(self) 

1427 # for set in particular, we have no way to access the item 

1428 # that will be popped before pop is called. 

1429 __del(self, item) 

1430 return item 

1431 

1432 _tidy(pop) 

1433 return pop 

1434 

1435 def clear(fn): 

1436 def clear(self): 

1437 for item in list(self): 

1438 self.remove(item) 

1439 

1440 _tidy(clear) 

1441 return clear 

1442 

1443 def update(fn): 

1444 def update(self, value): 

1445 for item in value: 

1446 self.add(item) 

1447 

1448 _tidy(update) 

1449 return update 

1450 

1451 def __ior__(fn): 

1452 def __ior__(self, value): 

1453 if not _set_binops_check_strict(self, value): 

1454 return NotImplemented 

1455 for item in value: 

1456 self.add(item) 

1457 return self 

1458 

1459 _tidy(__ior__) 

1460 return __ior__ 

1461 

1462 def difference_update(fn): 

1463 def difference_update(self, value): 

1464 for item in value: 

1465 self.discard(item) 

1466 

1467 _tidy(difference_update) 

1468 return difference_update 

1469 

1470 def __isub__(fn): 

1471 def __isub__(self, value): 

1472 if not _set_binops_check_strict(self, value): 

1473 return NotImplemented 

1474 for item in value: 

1475 self.discard(item) 

1476 return self 

1477 

1478 _tidy(__isub__) 

1479 return __isub__ 

1480 

1481 def intersection_update(fn): 

1482 def intersection_update(self, other): 

1483 want, have = self.intersection(other), set(self) 

1484 remove, add = have - want, want - have 

1485 

1486 for item in remove: 

1487 self.remove(item) 

1488 for item in add: 

1489 self.add(item) 

1490 

1491 _tidy(intersection_update) 

1492 return intersection_update 

1493 

1494 def __iand__(fn): 

1495 def __iand__(self, other): 

1496 if not _set_binops_check_strict(self, other): 

1497 return NotImplemented 

1498 want, have = self.intersection(other), set(self) 

1499 remove, add = have - want, want - have 

1500 

1501 for item in remove: 

1502 self.remove(item) 

1503 for item in add: 

1504 self.add(item) 

1505 return self 

1506 

1507 _tidy(__iand__) 

1508 return __iand__ 

1509 

1510 def symmetric_difference_update(fn): 

1511 def symmetric_difference_update(self, other): 

1512 want, have = self.symmetric_difference(other), set(self) 

1513 remove, add = have - want, want - have 

1514 

1515 for item in remove: 

1516 self.remove(item) 

1517 for item in add: 

1518 self.add(item) 

1519 

1520 _tidy(symmetric_difference_update) 

1521 return symmetric_difference_update 

1522 

1523 def __ixor__(fn): 

1524 def __ixor__(self, other): 

1525 if not _set_binops_check_strict(self, other): 

1526 return NotImplemented 

1527 want, have = self.symmetric_difference(other), set(self) 

1528 remove, add = have - want, want - have 

1529 

1530 for item in remove: 

1531 self.remove(item) 

1532 for item in add: 

1533 self.add(item) 

1534 return self 

1535 

1536 _tidy(__ixor__) 

1537 return __ixor__ 

1538 

1539 l = locals().copy() 

1540 l.pop("_tidy") 

1541 l.pop("Unspecified") 

1542 return l 

1543 

1544 

1545class InstrumentedList(list): 

1546 """An instrumented version of the built-in list.""" 

1547 

1548 

1549class InstrumentedSet(set): 

1550 """An instrumented version of the built-in set.""" 

1551 

1552 

1553class InstrumentedDict(dict): 

1554 """An instrumented version of the built-in dict.""" 

1555 

1556 

1557__canned_instrumentation = { 

1558 list: InstrumentedList, 

1559 set: InstrumentedSet, 

1560 dict: InstrumentedDict, 

1561} 

1562 

1563__interfaces = { 

1564 list: ( 

1565 {"appender": "append", "remover": "remove", "iterator": "__iter__"}, 

1566 _list_decorators(), 

1567 ), 

1568 set: ( 

1569 {"appender": "add", "remover": "remove", "iterator": "__iter__"}, 

1570 _set_decorators(), 

1571 ), 

1572 # decorators are required for dicts and object collections. 

1573 dict: ({"iterator": "values"}, _dict_decorators()) 

1574 if util.py3k 

1575 else ({"iterator": "itervalues"}, _dict_decorators()), 

1576} 

1577 

1578 

1579class MappedCollection(dict): 

1580 """A basic dictionary-based collection class. 

1581 

1582 Extends dict with the minimal bag semantics that collection 

1583 classes require. ``set`` and ``remove`` are implemented in terms 

1584 of a keying function: any callable that takes an object and 

1585 returns an object for use as a dictionary key. 

1586 

1587 """ 

1588 

1589 def __init__(self, keyfunc): 

1590 """Create a new collection with keying provided by keyfunc. 

1591 

1592 keyfunc may be any callable that takes an object and returns an object 

1593 for use as a dictionary key. 

1594 

1595 The keyfunc will be called every time the ORM needs to add a member by 

1596 value-only (such as when loading instances from the database) or 

1597 remove a member. The usual cautions about dictionary keying apply- 

1598 ``keyfunc(object)`` should return the same output for the life of the 

1599 collection. Keying based on mutable properties can result in 

1600 unreachable instances "lost" in the collection. 

1601 

1602 """ 

1603 self.keyfunc = keyfunc 

1604 

1605 @collection.appender 

1606 @collection.internally_instrumented 

1607 def set(self, value, _sa_initiator=None): 

1608 """Add an item by value, consulting the keyfunc for the key.""" 

1609 

1610 key = self.keyfunc(value) 

1611 self.__setitem__(key, value, _sa_initiator) 

1612 

1613 @collection.remover 

1614 @collection.internally_instrumented 

1615 def remove(self, value, _sa_initiator=None): 

1616 """Remove an item by value, consulting the keyfunc for the key.""" 

1617 

1618 key = self.keyfunc(value) 

1619 # Let self[key] raise if key is not in this collection 

1620 # testlib.pragma exempt:__ne__ 

1621 if self[key] != value: 

1622 raise sa_exc.InvalidRequestError( 

1623 "Can not remove '%s': collection holds '%s' for key '%s'. " 

1624 "Possible cause: is the MappedCollection key function " 

1625 "based on mutable properties or properties that only obtain " 

1626 "values after flush?" % (value, self[key], key) 

1627 ) 

1628 self.__delitem__(key, _sa_initiator) 

1629 

1630 

1631# ensure instrumentation is associated with 

1632# these built-in classes; if a user-defined class 

1633# subclasses these and uses @internally_instrumented, 

1634# the superclass is otherwise not instrumented. 

1635# see [ticket:2406]. 

1636_instrument_class(MappedCollection) 

1637_instrument_class(InstrumentedList) 

1638_instrument_class(InstrumentedSet)