Published in The Bell System Technical Journal in July 1956, this paper interprets information transmission through a gambler’s use of received channel symbols to guide bets. Under fair odds, the maximum exponential growth rate of capital equals the channel’s information transmission rate. The analysis extends to nonfair odds and a bookmaker’s or track’s take, deriving betting allocations that maximize expected logarithmic capital growth. The conclusion distinguishes this objective from maximizing expected wealth and discusses potential applications to investment.
Free preview text
The first 5 pages, shown as text so search engines and readers can find this document.
Page 1
A New Interpretation of Information Rate
reproduced with permission of AT&T
By J. L. Kelly, jr.
(Manuscript received March 21, 1956)
If the input symbols to a communication channel represent the outcomes of a
chance event on which bets are available at odds consistent with their probabilities
(i.e., “fair” odds), a gambler can use the knowledge given him by the received
symbols to cause his money to grow exponentially. The maximum exponential
rate of growth of the gambler’s capital is equal to the rate of transmission of
information over the channel. This result is generalized to include the case of
arbitrary odds.
Thus we find a situation in which the transmission rate is significant even
though no coding is contemplated. Previously this quantity was given significance
only by a theorem of Shannon’s which asserted that, with suitable encoding,
binary digits could be transmitted over the channel at this rate with an arbitrarily
small probability of error.
introduction
Shannon defines the rate of transmission over a noisy communication channel
in terms of various probabilities.1 This definition is given significance by a
theorem which asserts that binary digits may be encoded and transmitted over
the channel at this rate with arbitrarily small probability of error. Many workers
in the field of communication theory have felt a desire to attach significance to
the rate of transmission in cases where no coding was contemplated. Some
have even proceeded on the assumption that such a significance did, in fact,
exist. For example, in systems where no coding was desirable or even possible
(such as radar), detectors have been designed by the criterion of maximum
transmission rate or, what is the same thing, minimum equivocation. Without
further analysis such a procedure is unjustified.
The problem then remains of attaching a value measure to a communication
1C.E. Shannon, A Mathematical Theory of Communication, B.S.T.J., 27, pp. 379-423,
623-656, Oct., 1948.
917
Page 2
918 the bell system technical journal, july 1956
system in which errors are being made at a non-negligible rate, i.e., where
optimum coding is not being used. In its most general formulation this problem
seems to have but one solution. A cost function must be defined on pairs of
symbols which tells how bad it is to receive a certain symbol when a specified
signal is transmitted. Furthermore, this cost function must be such that its
expected value has significance, i.e., a system must be preferable to another if
its average cost is less. The utility theory of Von Neumann2 shows us one way
to obtain such a cost function. Generally this cost function would depend on
things external to the system and not on the probabilities which describe the
system, so that its average value could not be identified with the rate as defined
by Shannon.
The cost function approach is, of course, not limited to studies of commu-
nication systems, but can actually be used to analyze nearly any branch of
human endeavor. The author believes that it is too general to shed any light
on the specific problems of communication theory. The distinguishing feature
of a communication system is that the ultimate receiver (thought of here as a
person) is in a position to profit from any knowledge of the input symbols or
even from a better estimate of their probabilities. A cost function, if it is sup-
posed to apply to a communication system, must somehow reflect this feature.
The point here is that an arbitrary combination of a statistical transducer (i.e.,
a channel) and a cost function does not necessarily constitute a communication
system. In fact (not knowing the exact definition of a communication system on
which the above statements are tacitly based) the author would not know how
to test such an arbitrary combination to see if it were a communication system.
What can be done, however, is to take some real-life situation which seems
to possess the essential features of a communication problem, and to analyze
it without the introduction of an arbitrary cost function. The situation which
will be chosen here is one in which a gambler uses knowledge of the received
symbols of a communication channel in order to make profitable bets on the
transmitted symbols.
the gambler with a private wire
Let us consider a communication channel which is used to transmit the
results of a chance situation before those results become common knowledge, so
that a gambler may still place bets at the original odds. Consider first the case
of a noiseless binary channel, which might be
2Von Neumann and Morgenstein, Theory of Games and Economic Behavior, Princeton
Univ. Press, 2nd Edition, 1947.
Page 3
a new interpretation of information rate 919
used, for example, to transmit the results of a series of baseball games between
two equally matched teams. The gambler could obtain even money bets even
though he already knew the result of each game. The amount of money he
could make would depend only on how much he chose to bet. How much would
he bet? Probably all he had since he would win with certainty. In this case
his capital would grow exponentially and after N bets he would have 2N times
his original bankroll. This exponential growth of capital is not uncommon in
economics. In fact, if the binary digits in the above channel were arriving at the
rate of one per week, the sequence of bets would have the value of an investment
paying 100 per cent interest per week compounded weekly. We will make use
of a quantity G called the exponential rate of growth of the gambler’s capital,
where
G = lim
N →∞
1
N log VN
V0
where VN is the gambler’s capital after N bets, V0 is his starting capital, and
the logarithm is to the base two. In the above example G = 1.
Consider the case now of a noisy binary channel, where each transmitted
symbol has probability, p, of error and q of correct transmission. Now the
gambler could still bet his entire capital each time, and, in fact, this would
maximize the expected value of his capital, 〈VN 〉, which in this case would be
given by
〈VN 〉 = (2q)N V0
This would be little comfort, however, since when N was large he would probably
be broke and, in fact, would be broke with probability one if he continued
indefinitely. Let us, instead, assume that he bets a fraction, `, of his capital
each time. Then
VN = (1 + `)W (1 − `)LV0
where W and L are the number of wins and losses in the N bets. Then
G = lim
N →∞
[ W
N log(1 + `) + L
N log(1 − `)
]
= q log(1 + `) + p log(1 − `) with probability one
Let us maximize G with respect to `. The maximum value with respect to
the Yi of a quantity of the form Z = ∑ Xi log Yi, subject to the constraint∑ Yi = Y , is obtained by putting
Yi = Y
X Xi,
Page 4
920 the bell system technical journal, july 1956
where X = ∑ Xi. This may be shown directly from the convexity of the
logarithm.
Thus we put
(1 + `) = 2q
(1 − `) = 2p
and
Gmax = 1 + p log p + q log q
= R
which is the rate of transmission as defined by Shannon.
One might still argue that the gambler should bet all his money (make ` = 1)
in order to maximize his expected win after N times. It is surely true that if the
game were to be stopped after N bets the answer to this question would depend
on the relative values (to the gambler) of being broke or possessing a fortune. If
we compare the fates of two gamblers, however, playing a nonterminating game,
the one which uses the value ` found above will, with probability one, eventually
get ahead and stay ahead of one using any other `. At any rate, we will assume
that the gambler will always bet so as to maximize G.
the general case
Let us now consider the case in which the channel has several input symbols,
not necessarily equally likely, which represent the outcome of chance events. We
will use the following notation:
p(s) the probability that the transmitted symbol is the s’th one.
p(r/s) the conditional probability that the received symbol is the r’th on the
hypothesis that the transmitted symbol is the s’th one.
p(s, r) the joint probability of the s’th transmitted and r’th received symbol.
q(r) received symbol probability.
q(s/r) conditional probability of transmitted symbol on hypothesis of received
symbol.
αs the odds paid on the occurrence of the s’th transmitted symbol, i.e., αs
is the number of dollars returned for a one-dollar bet (including that one
dollar).
a(s/r) the fraction of the gambler’s capital that he decides to bet on the oc-
currence of the s’th transmitted symbol after observing the r’th received
symbol.
Page 5
a new interpretation of information rate 921
Only the case of independent transmitted symbols and noise will be consid-
ered. We will consider first the case of “fair” odds, i.e.,
αs = 1
p(s)
In any sort of parimutuel betting there is a tendency for the odds to be fair
(ignoring the “track take”). To see this first note that if there is no “track take”
∑ 1
αs
= 1
since all the money collected is paid out to the winner. Next note that if
αs > 1
p(s)
for some s a bettor could insure a profit by making repeated bets on the sth
outcome. The extra betting which would result would lower αs. The same feed-
back mechanism probably takes place in more complicated betting situations,
such as stock market speculation.
There is no loss in generality in assuming that
∑
s
a(s/r) = 1
i.e., the gambler bets his total capital regardless of the received symbol. Since
∑ 1
αs
= 1
he can effectively hold back money by placing canceling bets. Now
VN = ∏
r,s
[a(s/r)αs]Wsr V0
where Wsr is the number of times that the transmitted symbol is s and the
received symbol is r.
log Vn
Vo
= ∑
rs
Wsr log αsa(s/r)
G = lim
N →∞
1
N log VN
V0
= ∑
rs
p(s, r) log αsa(s/r)
(1)
with probability one. Since
αs = 1
p(s)
A New Interpretation of Information Rate: summary, contents and background
This guide helps you understand the scope of A New Interpretation of Information Rate before or alongside reading the original document. It is based on the catalogue record, credited creator, source, classification and topics shown on this page. It does not replace the original work and does not invent claims about passages that Doxxly cannot verify.
Summary and overview
A New Interpretation of Information Rate is a 10-page documentcredited to Dara D. Cressley. Published in The Bell System Technical Journal in July 1956, this paper interprets information transmission through a gambler’s use of received channel symbols to guide bets. Under fair odds, the maximum exponential growth rate of capital equals the channel’s information transmission rate. The analysis extends to nonfair odds and a bookmaker’s or track’s take, deriving betting allocations that maximize expected logarithmic capital growth. The conclusion distinguishes this objective from maximizing expected wealth and discusses potential applications to investment. It appears in the Science & Engineering collection, which groups it with other material that readers may use for comparison and further study.
The item can be approached in several ways. A reader looking for a quick orientation can begin with the free five-page preview, inspect the page structure and use the metadata to judge relevance. Someone undertaking deeper research can work through the complete file, search its indexed text, record useful page references and compare its treatment of Information theory, Communication channels, Gambling, Optimal betting, Logarithmic capital growth, Probability, Investment with the related resources linked below.
Rather than reducing the work to a single claim, this page treats it as a source that should be read in context. The title, creator, publication source, licence and format all help explain what kind of evidence it may offer. Readers should distinguish the original author's statements from Doxxly's catalogue description, and should cite the original edition when using the material in academic, professional or published work.
Key topics and reader takeaways
The catalogue connects this document with Information theory, Communication channels, Gambling, Optimal betting, Logarithmic capital growth, Probability, Investment. These terms offer practical entry points rather than a substitute for close reading. Start by identifying the definitions, examples, arguments, tables or visual explanations attached to each topic. Then note where the creator draws connections, makes distinctions or moves from background material to a conclusion, recommendation, calculation or narrative development.
A useful reading method is to separate description from interpretation. First record what the item actually presents: names, headings, figures, formulas, quotations or stated propositions. Next consider why those details matter within the whole work. Finally compare them with another source from the same subject area. This process helps turn a long digital file into evidence that can be evaluated rather than merely skimmed.
Table of contents and reading guide
The following is an editorial content guide based on the verified catalogue topics. It is not represented as the original author's formal table of contents. Use it to organise notes while moving through the 10 pages in the player.
Part 1
Information theory
Use this section of the reading guide to identify how Information theory appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 2
Communication channels
Use this section of the reading guide to identify how Communication channels appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 3
Gambling
Use this section of the reading guide to identify how Gambling appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 4
Optimal betting
Use this section of the reading guide to identify how Optimal betting appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 5
Logarithmic capital growth
Use this section of the reading guide to identify how Logarithmic capital growth appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 6
Probability
Use this section of the reading guide to identify how Probability appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
Part 7
Investment
Use this section of the reading guide to identify how Investment appears in A New Interpretation of Information Rate, note important terms, and connect the subject to the wider purpose of the work.
When a heading in the original differs from this guide, follow the original heading in your citation. Page numbers can also vary between editions. Doxxly's direct page field is useful for navigation, but formal references should include the title, creator, source and edition details available in the file itself.
About Dara D. Cressley
Doxxly credits Dara D. Cressley as the author or creator of A New Interpretation of Information Rate. This page does not add an unverified biography, job title or institutional affiliation. The attribution is preserved so readers can distinguish the creator from the hosting library and follow the original record when an external author link is available.
Author context matters because a work reflects a particular purpose, audience and point of view. Before relying on the material, consider what the creator set out to explain, when the source was produced and whether later research or revised guidance should also be consulted. For literature and philosophy, compare translations and editions; for technical material, check whether a newer version has superseded the one shown here.
About the source organisation
The recorded source for this item is princeton.edu. Doxxly provides access and discovery tools but does not claim to be the original publisher unless explicitly stated. Source links, creator attribution and the recorded Publicly available file collected for editorial review licence are displayed to help readers assess provenance and permitted use.
Provenance is especially important for reports, standards, presentations and working spreadsheets. Check the original source for amendments, supporting files and current policy. A trustworthy research workflow keeps the source URL with any notes or downloaded copy, verifies important figures against the original, and avoids treating an archived document as automatically current.
Begin with the title and opening pages, then use the guide above to decide whether to read sequentially or move to a particular topic. The Doxxly player provides previous and next navigation, a direct page field, zoom steps, fit-to-width and fit-to-page modes, rotation and fullscreen. These controls are useful when a page contains small type, a wide table, an image, a scanned layout or a dense sequence of notes.
The first five pages are open to every visitor. A plan unlocks the complete item and full-document search. Search is best used for distinctive names, technical terms or short phrases; broad words can produce too many possible contexts. When you find a useful passage, read the surrounding pages so that a sentence, chart or formula is not separated from its explanation.
If downloading is enabled, members can obtain the original PDF file. Keep the original format when formulas, slide sequencing, pagination or layout carries meaning. Respect the Publicly available file collected for editorial review terms displayed on this page, and use the rights-report link if you believe the attribution or availability needs review.
Research questions to consider
As you read A New Interpretation of Information Rate, ask which audience the creator addresses, what problem or idea organises the work, what evidence is presented, and which assumptions remain unstated. Note whether important terms are defined consistently and whether examples represent a broader pattern or a narrow case. For quantitative material, record units, time periods, formulas and data sources. For interpretive material, separate the creator's language from your own summary.
A second set of questions concerns relevance today. Which parts remain durable, which depend on their historical or institutional setting, and what would need confirmation from a newer source? Following the tags and the Science & Engineering collection can reveal useful comparisons. Agreement between sources can strengthen confidence, while disagreement can point to changing definitions, methods, evidence or priorities.
Finally, decide what you need from the item: background knowledge, a quotation, a worked example, a model, a visual explanation or a starting bibliography. That purpose should guide the depth of reading and the notes you keep. Doxxly helps locate and navigate the source; careful interpretation remains the reader's responsibility.