Class s.c.m.t.Tokenizer:

Part of spamfighter.core.model.thomas View Source View In Hierarchy

A simple regex-based whitespace tokenizer. It expects a string and can return all tokens lower-cased or in their existing case.
Method __init__ Undocumented
Method tokenize Undocumented
def __init__(self, lower=True): (source)
Undocumented
def tokenize(self, obj): (source)
Undocumented
API Documentation for SpamFighter, generated by pydoctor at 2009-02-27 11:58:37.