Floating-Point Precision Calculator

Computers represent real numbers using the IEEE 754 floating-point standard. Every format has three parts: a sign bit, an exponent, and a mantissa (also called the significand). The mantissa bits determine precision; the exponent bits determine range.

The three common formats:

float16 (half precision): 1 sign + 5 exponent + 10 mantissa bits. Roughly 3 significant decimal digits. Used in machine learning for memory-efficient inference. Maximum value: 65,504.

float32 (single precision): 1 sign + 8 exponent + 23 mantissa bits. About 7 significant decimal digits. The default in most graphics and many scientific applications.

float64 (double precision): 1 sign + 11 exponent + 52 mantissa bits. About 15-16 significant decimal digits. The default in Python, R, MATLAB, and most numerical computing.

Machine epsilon is the gap between 1.0 and the next number the format can represent above it. It equals 2^(-mantissa_bits): about 1.19e-7 for float32 and about 2.22e-16 for float64. This is the value your language reports as DBL_EPSILON, numpy.finfo().eps or Number.EPSILON.

A word of warning on the definition, because it trips people up. Epsilon is often described as “the smallest e where 1.0 + e is not 1.0”, and that is not quite the same number. Under round-to-nearest, anything above half an epsilon rounds up, so 1 + 1.2e-16 already exceeds 1 in float64 even though epsilon is 2.22e-16. Half of epsilon is the unit roundoff, and it is the bound on relative error in a single correctly-rounded operation.

Why it matters. Subtracting two nearly-equal numbers causes catastrophic cancellation, and you can lose most of your significant digits in one operation. Adding a very small number to a very large one can make the small one vanish outright: in float32, an accumulator past about 16.7 million (2^24) stops changing when you add 1 to it, which is a real and frequently-hit failure in loop counters and running totals. These effects scale with machine epsilon, and they are why numerical analysts reach for float64 by default.

Decimal precision is floor((mantissa_bits + 1) × log10(2)). The +1 counts the implicit leading bit that every normalized float carries but never stores, which is why float32 lands at 7 digits rather than 6.