A Mathematical Theory of Communication
Fig. 1 â€" Schematic diagram of a general communication system.
a decimal digit is about 3 3 bits. A digit wheel on a desk computing machine has ten stable positions and
therefore has a storage capacity of one decimal digit. In analytical work where integration and differentiation
are involved the base e is sometimes useful. The resulting units of information will be called natural units.
Change from the base a to base b merely requires multiplication by logb a.
By a communication system we will mean a system of the type indicated schematically in Fig. 1. It
consists of essentially ï¬ve parts:
1. An information source which produces a message or sequence of messages to be communicated to the
receiving terminal. The message may be of various types: (a) A sequence of letters as in a telegraph
of teletype system; (b) A single function of time f (t) as in radio or telephony; (c) A function of
time and other variables as in black and white television â€" here the message may be thought of as a
function f (x; y; t) of two space coordinates and time, the light intensity at point (x; y) and time t on a
pickup tube plate; (d) Two or more functions of time, say f (t), g(t), h(t) â€" this is the case in â€œthree-
dimensionalâ€ sound transmission or if the system is intended to service several individual channels in
multiplex; (e) Several functions of several variables â€" in color television the message consists of three
functions f (x; y; t), g(x; y; t), h(x; y; t) deï¬ned in a three-dimensional continuum â€" we may also think
of these three functions as components of a vector ï¬eld deï¬ned in the region â€" similarly, several
black and white television sources would produce â€œmessagesâ€ consisting of a number of functions
of three variables; (f) Various combinations also occur, for example in television with an associated
2. A transmitter which operates on the message in some way to produce a signal suitable for trans-
mission over the channel. In telephony this operation consists merely of changing sound pressure
into a proportional electrical current. In telegraphy we have an encoding operation which produces
a sequence of dots, dashes and spaces on the channel corresponding to the message. In a multiplex
PCM system the different speech functions must be sampled, compressed, quantized and encoded,
and ï¬nally interleaved properly to construct the signal. Vocoder systems, television and frequency
modulation are other examples of complex operations applied to the message to obtain the signal.
3. The channel is merely the medium used to transmit the signal from transmitter to receiver. It may be
a pair of wires, a coaxial cable, a band of radio frequencies, a beam of light, etc.
4. The receiver ordinarily performs the inverse operation of that done by the transmitter, reconstructing
the message from the signal.
5. The destination is the person (or thing) for whom the message is intended.
We wish to consider certain general problems involving communication systems. To do this it is ï¬rst
necessary to represent the various elements involved as mathematical entities, suitably idealized from their