Token
This module implements token.
- class Token
The token API.
Implemented by SingleToken and MultiToken.
- __init__()
Initialize the token.
- strategy_data: Any
A data store reserved for use by strategies.
- __len__() int
Return how many tokens are in the tree.
- __iter__() Iterator[Optional[super_collator.token.Token[super_collator.token.TT]]]
Iterate the tokens in the tree.
- __getitem__(n) Optional[super_collator.token.Token[super_collator.token.TT]]
Return the nth token in the iteration.
- __str__() str
Return a string represenation of the token tree.
- class SingleToken(data: super_collator.token.TT, user_data: Optional[Any] = None)
Represents one input token.
- __init__(data: super_collator.token.TT, user_data: Optional[Any] = None)
Initialize the token.
- Parameters
user_data (Any) – user data that will be round-tripped unchanged.
- data: super_collator.token.TT
The data to align.
- user_data: Any
A data store reserved for user data.
- __len__() int
Return how many tokens are in the tree.
- __iter__() Iterator[Optional[super_collator.token.Token[super_collator.token.TT]]]
Iterate the tokens in the tree.
- __getitem__(n: int) Optional[super_collator.token.Token[super_collator.token.TT]]
Return the nth token in the iteration.
- __str__() str
Return a string represenation of the token tree.
- class MultiToken(*args: Optional[super_collator.token.Token[super_collator.token.TT]])
A container for one or two tokens.
A MultiToken is a node in a binary tree that mimics the order in which multiple aligments were done.
An alignment produces a sequence of MultiToken trees. Each tree contains all the tokens that were aligned at this position plus any gaps represented as None.
- __init__(*args: Optional[super_collator.token.Token[super_collator.token.TT]])
Initialize the token.
- __len__() int
Return how many tokens are in the tree.
- __iter__() Iterator[Optional[super_collator.token.Token[super_collator.token.TT]]]
Iterate the tokens in the tree.
- __getitem__(n: int) Optional[super_collator.token.Token[super_collator.token.TT]]
Return the nth token in the iteration.
- __str__() str
Return a string represenation of the token tree.