Pointwise mutual information
information Theory
In statistics, probability theory and information theory, pointwise mutual information (PMI), or point mutual information, is a measure of association. It compares the probability of two events occurring together to what this probability would be if the events were independent.
PMI (especially in its positive pointwise mutual information variant) has been described as "one of the most important concepts in NLP", where it "draws on the intuition that the best way to weigh the association between two words is to ask how much more the two words co-occur in [a] corpus than we would have expected them to appear by chance."
The concept was introduced in 1961 by Robert Fano under the name of "mutual information", but today that term is instead used for a related measure of dependence between random variables: The mutual information (MI) of two discrete random variables refers to the average PMI of all possible events.
Definition
The PMI of a pair of outcomes x and y belonging to discrete random variables X and Y quantifies the discrepancy between the probability of their coincidence given their joint distribution and their individual distributions, assuming independence. Mathematically:
pmi
(
x
;
y
)
≡
log
2
p
(
x
,
y
)
p
(
x
)
p
(
y
)
=
log
2
p
(
x
|
y
)
p
(
x
)
=
log
2
p
(
y
|
x
)
p
(
y
)
{\displaystyle \operatorname {pmi} (x;y)\equiv \log _{2}{\frac {p(x,y)}{p(x)p(y)}}=\log _{2}{\frac {p(x|y)}{p(x)}}=\log _{2}{\frac {p(y|x)}{p(y)}}}
(with the latter two expressions being equal to the first by Bayes' theorem). The mutual information (MI) of the random variables X and Y is the expected value of the PMI (over all possible outcomes).
The measure is symmetric (
pmi
(
x
;
y
)
=
pmi
(
y
;
x
)
{\displaystyle \operatorname {pmi} (x;y)=\operatorname {pmi} (y;x)}
). It can take positive or negative values, but is zero if X and Y are independent. Note that even though PMI may be negative or positive, its expected outcome over all joint events (MI) is non-negative. PMI maximizes when X and Y are perfectly associated (i.e.
This brief starts where responsible research should: with the source description of “Pointwise mutual information” as information Theory. Everything that follows is an evidence route, not borrowed authority.
Why this record matters
The subject matters to the general reference register because the source frames it as information Theory. Its deeper value depends on whether names, dates, institutions and citations support that framing.
Vocabulary and entity names are the principal evidence signals here, because they determine the precision of every later search. The source revision retrieved here is dated Aug 31, 2026. The linked authority identifier is Q3798612. None of the 0 selected statements returned an explicit reference. The first chronological checks are 1961.
Overview language is designed for orientation and should not be treated as a substitute for the evidence cited beneath it. The source lead contains qualifying language; that uncertainty should survive quotation, summary and reuse. Authority statements aid reconciliation but still require their own references, qualifiers and ranks to be checked.
How to read it
Use the entry as an orientation point, then follow its citations and revision history. Names, dates and institutional relationships should be checked against the original record.
- Subject orientation
- Search vocabulary
- Locating named sources
The closest primary source, responsible institution and strongest cited specialist reference.
Three-step research path
- Establish the record: confirm the title “Pointwise mutual information”, its source revision and the description used here.
- Expand the search: follow Pointwise mutual information primary sources, Pointwise mutual information archive and Pointwise research across catalogues and specialist indexes.
- Test the account: compare the strongest cited source with the responsible institution’s current record and note any disagreement.
Questions for further research
- Which source most directly establishes the central claim about “Pointwise mutual information”?
- Which cited source is closest to the event, object or claim?
- What terminology or title could unlock a more precise catalogue search?
Search terms from this dossier
This entry incorporates text from “Pointwise mutual information” on English Wikipedia. Contributors are listed in the page history. Text is available under the Creative Commons Attribution-ShareAlike 4.0 License. Selected authority identifiers and statements are retrieved from Wikidata under CC0; their references and qualifiers remain part of the verification path.