A Mathematical Theory of Communication
Claude Shannon · 1948
"Information can be quantified in bits and transmitted reliably over noisy channels up to a hard capacity limit, regardless of meaning."
Before Shannon, 'communication' was treated as an engineering problem specific to each medium — telephone static was a phone problem, radio noise was a radio problem. Shannon showed that any message, whether text, sound, or image, could be broken down into a stream of bits (his paper popularized the term, suggested to him by colleague John Tukey), and that the real question was always the same underlying math problem: how much information can you reliably squeeze through a channel that has some amount of noise, and how do you protect a message from that noise. He proved there's a hard ceiling (channel capacity) on how much information can pass through reliably, but that below that ceiling, clever encoding can make the error rate as close to zero as you want.
The mechanism splits the problem into two separable pieces: source coding (compressing a message down to its actual information content, removing redundancy — this is why a ZIP file shrinks a text document) and channel coding (deliberately adding back a different, mathematically structured kind of redundancy so errors introduced by noise can be detected and corrected on the receiving end — this is why your phone call doesn't turn to static every time a truck drives by). Shannon proved these two problems can be solved independently of each other and independently of what the message actually means, which is why the exact same mathematical framework works for text, audio, video, and satellite telemetry.
What was radical about Shannon's approach to defining 'information'?
Read more about the topic
The explanation above is written with AI assistance. These are the originals — go to them to check it.
- A Mathematical Theory of Communication (original paper)Bell System Technical Journal, 1948
- A Mathematical Theory of CommunicationWikipedia
The Anatomy of a Large-Scale Hypertextual Web Search Engine
"Ranking web pages by the link structure of the web itself (PageRank) produces dramatically better search than keyword matching."