The Information: A History, A Theory, A Flood

The year 1948 was undoubtedly one of the most illustrious in the history of the Bell Labs, the famous American R&D center. That was the year when the invention of the transistor was announced. It was also the year when mathematician and electrical engineer Claude Elwood Shannon (1916-2001) popularized the unit of measurement known as a bit in a seminal paper entitled A Mathematical Theory of Communication. The term – a pun exploiting the fact that a bit is a small piece of something – is a portmanteau of “binary digit”.

 

Although almost everyone knows the gram is a unit of mass, the second is a unit of time and the meter is a unit of length, it is much less easy to grasp what a bit is the benchmark for. Although Shannon specified in his previous article that the bit is a unit measuring information, the fact that there is a concept of information that is unambiguously defined and able to be quantified is far from being self-evident. Award-winning author and science historian James Gleick brilliantly strives to breathe life into the intuition favoring the development of a real understanding of this enigmatic concept in this remarkable book which is aimed at the (well-informed or persevering) general public.

 

The author draws a parallel between the shifting of the meaning of the concept of information that Shannon used to establish his theory in the middle of the 20th century and those that were used in laying the groundwork for Newtonian mechanics. According to Gleick, one reason that 17th century physicists struggled to break out of the quicksand of confusion in which they were bogged down while Sir Isaac Newton was able to make a great leap forward was that he gave himself a solid point of support to drive himself forward by putting forward a mathematical formalization of the vague concepts of force, mass and movement. Similarly, when – under the impetus provided by Claude Shannon

– the concept of information was purged of any impurity or inaccuracy, when it finally ceased being artisanal and hazy and became scientific and precise, it became obvious that information was there, all around, under our noses, almost waiting to finally be noticed.

 

James Gleick writes as if he were weaving a Persian rug. Having carefully selected several invigorating common threads, he weaves them together tirelessly.  While this overlapping could easily result in indescribable chaos, instead one notices a strikingly rich motif.

 

In this book, the author skillfully shows us that Claude Shannon’s ingenious idea was to acknowledge that there were enormous benefits, although it may seem paradoxical at first glance, to decoupling information from its semantic content. The result is a concept of information that is closely associated with uncertainty; that implies a certain surprise; that is indissociable from a certain difficulty and, lastly, that is intrinsically linked to entropy. Uncertainty, surprise, difficulty and entropy. These are the four common threads that Gleick uses to weave a fabric that is complex and dense while never becoming cumbersome.

 

Of all the concepts associated with Shannon’s theory of information addressed in this text, entropy is unquestionably the most enigmatic but also the most powerful and profound. Incidentally, a considerable part of this remarkable book is dedicated to revealing unsuspected links between information and uncertainty; between information and entropy, this murky measure of disorder in thermodynamics.

 

Through the fifteen chapters, Gleick brings us to understand how much each technological revolution in the world of communications has provoked a reconfiguring of the social fabric as well as resulting in numerous scientific and technical challenges.

 

The first genuine communications revolution following the development of verbal language in prehistoric times is the invention of writing. With it came the possibility of permanence for a message. This raised concerns regarding confidentiality, and one can certainly say with confidence that writing and cryptography are two nuts that fell from the same tree. Gleick devotes several magnificent pages to discussing the invention of the alphabet; such an unlikely abstraction that it only occurred once in the history of humanity. That is to say that the Greek, Latin, Armenian, Georgian and Cyrillic, Hebrew, Brahmi, and Perso-Arabic alphabets are all descended from the same proto-alphabet.

 

With the rise of telegraphy, it became necessary to agree to submit to the constraints imposed by this channel of communication. The author ensures that readers properly appreciate the scope of the feat that U.S. scientists Samuel Morse and Alfred Lewis Vail accomplished by designing a code to convert information conveyed by spoken language into a form that could be transmitted via a cable, i.e. with the tapping of a telegraph key. 

 

Later, the arrival of the telephone brought along the need to account for the modulation of amplitude and frequency, and develop signal processing techniques (control, filtering, noise reduction, compression, etc.) 

 

This work also contains a radiant reflection on redundancy as an antidote to ambiguity. The more progress is made in the constant pursuit of efficiency – in other words, the more communication is brief and concise – the more we run the risk of the message becoming unintelligible if everything does not go as planned (if there is stuttering, if there is a lapse in speech, if the message is scrambled by ambient noise, if the issuer of the message commits an error during transmission, if an intermediary retransmits the message incorrectly, or if the recipient of the message makes a translation error). The radio alphabet used in radio communications (where one says Bravo and Victor rather than B and V ) is the go-to example of recourse to redundancy to dissipate ambiguity.  

 

Claude Shannon also stood out through his use of Boolean algebra to describe electrical circuits, and the author uses this opportunity to address the development of mathematical logic by Boole, Frege, Russell, and Peano until the publication by Gödel of his famous theorems of incompleteness in 1931 drove the final nail into the coffin of Hilbert’s program to ensure the foundations of mathematics.

 

Gleick then uses the meeting between Claude Shannon and Alan Turing in 1943 to pivot toward the question of decidability of the calculation of first-order egalitarian predicates – the Entscheidungsproblem – then toward Turing’s abstract computer, Vannevar Bush’s differential analyzer and John von Neumann’s digital computer. The author also takes the opportunity to address the unfinished work of visionary inventor Charles Babbage and programming pioneer Ada Lovelace, who was forced to program in her head since Babbage’s analytical machine never saw the light of day.

---

 

Frederic Morneau-Guerin is a professor in the Department of Education at Universite TELUQ. He holds a Ph.D. in abstract harmonic analysis.