tfidfmodel

class gensim.models.tfidfmodel.TfidfModel(corpus, id2word=None, normalize=True)

Objects of this class realize the transformation between word-document co-occurence matrix (integers) into a locally/globally weighted matrix (positive floats).

This is done by combining the term frequency counts (the TF part) with inverse document frequency counts (the IDF part), optionally normalizing the resulting documents to unit length.

The main methods are:
  1. constructor, which calculates IDF weights for all terms in the training corpus.
  2. the [] method, which transforms a simple count representation into the TfIdf space.
>>> tfidf = TfidfModel(corpus)
>>> doc_tfidf = tfidf[doc_tf]

Model persistency is achieved via its load/save methods.

id2word is a mapping from word ids (integers) to words (strings). It is used to determine the vocabulary size, as well as for debugging and topic printing. If not set, it will be determined from the corpus.

normalize dictates whether the resulting vectors will be set to unit length.

apply(corpus)
Apply the transformation to a whole corpus (as opposed to a single document) and return the result as another another corpus.
initialize(corpus)
Compute inverse document weights, which will be used to modify term frequencies for documents.
classmethod load(fname)
Load a previously saved object from file (also see save).
save(fname)
Save the object to file via pickling (also see load).

Previous topic

lsimodel

This Page