This is the first of a series of posts about Claude Shannon, mathematician, engineer, and computer scientist whose work beginning in the late ’30s helped to define the Information Age that emerged over the subsequent decades. Shannon is largely credited for having developed Information Theory. Over the next few posts, we’ll dive into some of his work, starting with the fundamental building blocks of communication.
When Shannon was a child growing up in the heart of Michigan’s snow belt, he built a crude battery-operated electrical system to send Morse-code messages across the barbed-wire fences of his rural community. Morse code. Dots and dashes. These little impulses could be used to communicate long, complex messages. They could be used to communicate the Declaration of Independence or the Lord’s Prayer. Hell, they could be used to communicate the entire King James Bible or the collected anthology of William Shakespeare.
Shannon’s young mind began to wonder, what was the minimum amount of information needed to communicate messages? And this question set in motion a lifetime of groundbreaking work. In Morse code, all information could be communicated using dots, dashes, and spaces. That seemed pretty simple. But could information be simplified even further? Simple information certainly could.
Take, for instance, the flip of a coin. There are only two possible outcomes. Heads means one thing, and tails means another. True or false. Black or white. Zero or one. On or off. Good or evil. Up or down. Forward or backward. To be or not to be. Lots of information could be attached to this simple binomial outcome, so long as the meaning of each outcome was known by the parties observing it.
What should we call the simplest unit of information? Shannon eventually called it a “bit” of information, short for a “binary digit.”
Like many mathematicians, Shannon’s mind pondered the realm of minimums and maximums, of zeros and infinities and negative infinities. It was a mind that loved abstractions, especially in ways abstractions could be reconciled with the physical world. Shannon earned degrees in both mathematics and engineering, appealing to the symbolic, abstract aspect of mathematical proofs, as well as the concrete aspect of building systems and circuits that could facilitate communication. Even in his old age, his childlike mind maintained its curiosity for what was possible.
If a simple bit of information could be used to communicate a simple message, could it also be used to communicate a more complex message? After all, Morse code was only slightly more complicated than a binary system. A little thinking revealed that, yes, bits could do that. A two-bit sequence could be used to recreate Morse code. For instance, if a bit signified either “on” or “off,” he could use “on-on” for a dot, “on-off” for a dash, and “off-off” for a space.
Again, so long as the parties communicating understood how the two-bit sequences signified Morse code, and they understood how Morse code signified letters to spell out a message, then bits could be used for the most complex messages that a language could communicate.
In the same way, a bit could be used to communicate black or white, as in pixels of an image. A many-bit combination could create a complex image, and the more bits included, the more complex and detailed the image could be.
If bits could be used to create images, and images were projected sequentially as in a film, bits could be used for movies. Thinking some more, bits could be used for sounds (“on” and “off” in different frequencies).
Thinking about the bit paradigm over time, it soon dawned on Claude Shannon that all information that could be communicated could be communicated with binary digits. Bits represented the atoms that served as the building blocks for all communication. This was one of the many singular links that Shannon had in his mind when he enrolled as a graduate student at MIT in the 1930s. There, he would go on to write what some have called the most important master’s thesis of all time.
Stay tuned!