Shannon Information Entropy Calculator
Calculate Shannon entropy for up to four symbols.
Enter frequencies or counts to find information content in bits, maximum entropy, and coding efficiency.
Shannon entropy, introduced by Claude Shannon in 1948, measures the average information content (or uncertainty) in a message source. It is the foundation of information theory, data compression, and cryptography.
The formula:
H = -sum( p_i x log2(p_i) )
where p_i is the probability of symbol i. The result is in bits per symbol.
What entropy means. A coin flip (50/50) has 1 bit of entropy. You need exactly 1 bit to represent the outcome. A fair die (6 equal outcomes) has log2(6) = 2.58 bits of entropy. A completely predictable source (one symbol with probability 1.0) has 0 bits of entropy. There is no information in a message you already know.
Maximum entropy occurs when all symbols are equally likely: H_max = log2(n) for n symbols. Any skew in the distribution reduces entropy below this maximum.
Coding efficiency is H / H_max expressed as a percentage. A source with 80% efficiency means Huffman or arithmetic coding can compress it to about 80% of the fixed-length code size.
Why it matters for compression. Huffman coding and LZ-based algorithms (ZIP, gzip, Deflate) exploit low entropy. If your source has 2.5 bits of entropy per symbol, no lossless compressor can do better than 2.5 bits per symbol on average. This is Shannon’s source coding theorem.
Entropy in cryptography. A password with n equally likely characters has log2(n^length) bits of entropy. The same formula applies wherever you need to measure unpredictability.