Limiting density of discrete points
notion in information theory
In information theory, the limiting density of discrete points is an adjustment to the formula of Claude Shannon for differential entropy.
It was formulated by Edwin Thompson Jaynes to address defects in the initial definition of differential entropy.
Definition
Shannon originally wrote down the following formula for the entropy of a continuous distribution, known as differential entropy:
h
(
X
)
=
−
∫
p
(
x
)
log
p
(
x
)
d
x
.
{\displaystyle h(X)=-\int p(x)\log p(x)\,dx.}
Unlike Shannon's formula for the discrete entropy, however, this is not the result of any derivation (Shannon simply replaced the summation symbol in the discrete version with an integral), and it lacks many of the properties that make the discrete entropy a useful measure of uncertainty. In particular, it is not invariant under a change of variables and can become negative. In addition, it is not even dimensionally correct. Since
h
(
X
)
{\displaystyle h(X)}
would be dimensionless,
p
(
x
)
{\displaystyle p(x)}
must have units of
1
d
x
{\displaystyle {\frac {1}{dx}}}
, which means that the argument to the logarithm is not dimensionless as required.
Jaynes argued that the formula for the continuous entropy should be derived by taking the limit of increasingly dense discrete distributions. Suppose that we have a set of
N
{\displaystyle N}
discrete points
{
x
i
}
{\displaystyle \{x_{i}\}}
, such that in the limit
N
→
∞
{\displaystyle N\to \infty }
their density approaches a function
m
(
x
)
{\displaystyle m(x)}
called the "invariant measure":
lim
N
→
∞
1
N
(
number of points in
a
<
x
<
b
)
=
∫
a
b
m
(
x
)
d
x
.
{\displaystyle \lim _{N\to \infty }{\frac {1}{N}}\,({\mbox{number of points in }}a<x<b)=\int _{a}^{b}m(x)\,dx.}
From the above invariant measure, Jaynes derived the following formula for continuous entropy, which he argued should be taken as the correct formula:
lim
N
→
∞
H
N
(
X
)
=
log
(
N
)
−
∫
p
(
x
)
log
p
(
x
)
m
(
x
)
d
x
.
Begin with the source’s own compact description: “Limiting density of discrete points” is notion in information theory. The dossier treats that line as a proposition to test through Limiting, density and discrete, not as a finished interpretation.
Why this record matters
The phrase “notion in information theory” supplies a clear boundary for inquiry. It also exposes the unanswered questions: who defined that boundary, when it became stable and which sources sit outside it.
Vocabulary and entity names are the principal evidence signals here, because they determine the precision of every later search. The source revision retrieved here is dated Aug 15, 2026. The linked authority identifier is Q6549556. None of the 0 selected statements returned an explicit reference.
Overview language is designed for orientation and should not be treated as a substitute for the evidence cited beneath it. The source lead contains qualifying language; that uncertainty should survive quotation, summary and reuse. Authority statements aid reconciliation but still require their own references, qualifiers and ranks to be checked.
How to read it
Use the entry as an orientation point, then follow its citations and revision history. Names, dates and institutional relationships should be checked against the original record.
- Subject orientation
- Search vocabulary
- Locating named sources
The closest primary source, responsible institution and strongest cited specialist reference.
Three-step research path
- Establish the record: confirm the title “Limiting density of discrete points”, its source revision and the description used here.
- Expand the search: follow Limiting density of discrete points primary sources, Limiting density of discrete points archive and Limiting research across catalogues and specialist indexes.
- Test the account: compare the strongest cited source with the responsible institution’s current record and note any disagreement.
Questions for further research
- Which source most directly establishes the central claim about “Limiting density of discrete points”?
- Which institution is responsible for the underlying evidence?
- Which cited source is closest to the event, object or claim?
Search terms from this dossier
This entry incorporates text from “Limiting density of discrete points” on English Wikipedia. Contributors are listed in the page history. Text is available under the Creative Commons Attribution-ShareAlike 4.0 License. Selected authority identifiers and statements are retrieved from Wikidata under CC0; their references and qualifiers remain part of the verification path.