Initial version

# [SCI] Information Theory
⏎
**Information Theory** (Shannon, 1948) is the mathematical theory of communication, defining entropy as a measure of uncertainty, and proving fundamental limits on data compression and error-free communication.
⏎
## Overview
⏎
Claude Shannon's 1948 paper "A Mathematical Theory of Communication" introduced the bit as the unit of information, defined information entropy H = −Σ pᵢ log₂ pᵢ, and proved two theorems: (1) data can be compressed to at most H bits/symbol (source coding theorem); (2) any noisy channel has a capacity C such that reliable communication is possible at all rates below C (channel coding theorem). Both theorems are tight. Shannon's entropy is mathematically identical to Boltzmann's thermodynamic entropy — a deep connection Shannon acknowledged.
⏎
Information theory underpins all of data compression (ZIP, MP3, JPEG), error-correcting codes (CDs, space probes, 5G), cryptography, and statistical machine learning.
⏎
## Key Figures & Recognition
⏎
- **Claude Shannon** (1916–2001): Father of information theory. Turing Award 1966, Kyoto Prize 1985, National Medal of Science. No Nobel (not physics or chemistry).
- **Norbert Wiener** (1894–1964): Cybernetics, 1948 (related but distinct work on feedback).
⏎
## Seminal Papers
⏎
- Shannon, C.E. ["A Mathematical Theory of Communication." *Bell Syst. Tech. J.* 27 (1948)](https://doi.org/10.1002/j.1538-7305.1948.tb01338.x)
- Shannon, C.E. & Weaver, W. *The Mathematical Theory of Communication*. Illinois, 1949.
⏎
# Parents
⏎
* [SCI] Statistical Mechanics⏎