AAWSAP DIRD, An Introduction to the Statistical Drake Equation, March 2010

Text

This text came with the file. It may have mistakes. When the text and the image disagree, the image is right.

1
UNCLASSIFIED/ /FOR OFFIEil.t.k W~i Q~lls¥
Appendix A: Proof of Shannon's 1948 Theorem Stating
That the Uniform Distribution is the "Most Uncertain" One
Over a Finite Range of Values
Information Theory was initiated by Claude Shannon (1916-2001) in his well-known
1948 two papers:
Rqrutted mm commi:ms from
B •1-S,'l•st
Ii
•ral Jour:!li,
\ GI. 1 . pp.
9--!13 , 62J..,iS , July, Occober. I
A Mathematical Theory of Communication
By C. E SHANNO
In this Appendix, we wish to draw attention to a couple of theorems that Shannon
proves on pages 36 and 37 of his work, and read, respectively (note that Shannon
omits the upper and lower limits of all integrals in the first theorem: they are minus
infinity and plus infinity, respectively):
5. Letp(x) bea one-dimensionaldis :i
ti
The onno p(x) ginng ammcimumentropy 1:.ubjec-tto the
c
dition tha
e
dan:l devia on ofx be fixed at a i- G
ian. To show this we must maximize
H(x) = - fp x)logp(x)d
<T- =j p
x- dx
and 1 =/ p x)dx
as cons
•. This requi~ . by
calculu~ of variations maxunizing
/ [- p x)logp(x)
>.p(x
1p(x)] dx.
e condition or lus is
- 1- logp(x
•ng the constllllf5 to sat'
the
and
UNCLASSIFIED/ /FOR: QFFIGI0ls Uii 0111 X
25