From Number Bases to Floating Point: How Computers Store Numbers

I first studied this material in a computer organization course several years ago. Returning to it today, I realized that I had never quite understood it. That is why I wrote this article.

Let us begin with a question: how are numbers actually represented inside a computer?

Bases and Place Value#

We are all familiar with decimal, because we are used to counting with ten digits. Computers, however, work with binary bits underneath. We therefore need a way to move freely between different representations of the same number.

Start with an example:

Decimal

13

Base 10

Binary

1101

Base 2

Octal

15

Base 8

Hexadecimal

D

Base 16

These four numbers are exactly equal. A number base is simply a different set of rules for writing the same quantity.

Why does hexadecimal use letters? A single digit needs to represent the values 0 through 15, giving 16 possibilities. The letters A through F stand for 10 through 15.

What, then, is the essence of conversion? Each position carries a different weight.

For example, the decimal number 123:

$$ 123 = 1 \times 100 + 2 \times 10 + 3 \times 1 $$

Binary works the same way, except that the weights of neighboring positions differ by a factor of 2 rather than 10.

Take the decimal number 13:

$$ 13 = 8 + 4 + 1 $$

From right to left, binary place values are 1, 2, 4, 8, 16, and so on. Write 1 in a position when you need its weight, and 0 when you do not.

Click a 0 or 1 below to flip that bit and see how the value changes.

1280 640 320 160 81 41 20 11

Top: place value · Bottom: bit value

Decimal

13

Binary

00001101

Octal

15

Hexadecimal

D

Current value = 8 + 4 + 1 = 13

This gives us the general principle of base conversion: expand a number by its place values to recover its decimal value. In the other direction, convert a decimal integer by repeatedly dividing by the target base and reading the remainders in reverse order.

There is an even more direct way to convert between binary and octal or hexadecimal, because:

$$ 8 = 2^3, \qquad 16 = 2^4 $$

Therefore:

• Every 3 binary bits correspond to exactly 1 octal digit.
• Every 4 binary bits correspond to exactly 1 hexadecimal digit.

For example, convert $11010110_2$ to hexadecimal and octal:

1101D

01106

→D6

To octal: group 3 bits at a time, from right to left

0113

0102

1106

→326

Pad the leftmost group with zeros if needed; leading zeros do not change an integer's value.

So:

$$ 11010110_2 = 326_8 = \mathrm{D6}_{16} = 214_{10} $$

This is why hexadecimal is so common in computer engineering: every 4 binary bits become 1 hexadecimal digit, with no complicated calculation required.

In Python, the prefixes 0b, 0o, and 0x introduce binary, octal, and hexadecimal integer literals, respectively.

Bits and Bytes#

Once binary makes sense, units of capacity follow naturally.

A bit can hold one of two states: 0 or 1. A byte, abbreviated B, usually consists of 8 bits:

$$ 1\,\mathrm{B} = 8\,\mathrm{bit} $$

A byte therefore has $2^8 = 256$ distinct bit patterns.

There is a historical wrinkle in capacity units: computing has traditionally used multiples of 1024, but standard notation distinguishes decimal prefixes from binary prefixes.¹²

Decimal units	Binary units
1 kB = 1000 B	1 KiB = 1024 B
1 MB = 1000 kB	1 MiB = 1024 KiB
1 GB = 1000 MB	1 GiB = 1024 MiB
1 TB = 1000 GB	1 TiB = 1024 GiB

When reading specifications for storage devices or network bandwidth, pay attention to which convention is being used.

For example, a connection advertised as 100 Mbps theoretically corresponds to 12.5 MB/s in decimal units. Actual transfer speeds also depend on protocol overhead and other factors.

The easy mistake here is that a lowercase b means bits, while an uppercase B means bytes.

Encoding Negative Numbers#

Now that we can represent numbers in binary, we can turn to arithmetic.

Binary addition is straightforward: an adder just needs to handle carries correctly. Subtraction is less straightforward.

This is precisely the problem von Neumann discussed in sections 8.1–8.2 of his 1945 First Draft of a Report on the EDVAC. He proposed an idea that remains important today: give negative numbers a special binary representation so that addition and subtraction can use the same addition mechanism.

Sign-Magnitude and Ones’ Complement#

The most intuitive idea is to attach a sign to each number.

Suppose we have only 4 bits and want to represent $+3$ and $-3$.

Naturally, we might use the most significant bit for the sign and the remaining three bits for the absolute value.

+3

0011

−3

1011

This representation is called sign-magnitude.

It is very intuitive. But feed $+3$ and $-3$ directly into an ordinary binary adder, and you get:

$$ \mathtt{0011}_2 + \mathtt{1011}_2 = \mathtt{1110}_2 $$

Interpreted as sign-magnitude, 1110 means $-6$, whereas the result we wanted was clearly 0.

In other words, an ordinary adder knows nothing about the signs we have assigned to the bits.

Sign-magnitude has another small problem: 0000 represents $+0$, and 1000 represents $-0$. One bit pattern is spent giving a second representation to the same numerical zero.

The next design goal is now clear: can we rearrange the encodings of negative numbers so that ordinary binary addition still produces the right answer?

Keep the same 4 bits. To represent $-3$, start with 0011, the encoding of $+3$, and flip every bit:

$$ \mathtt{0011} \xrightarrow{\text{flip every bit}} \mathtt{1100} $$

This is ones’ complement.

It looks like an odd trick, but it has an advantage: changing between a positive number and its negative only requires flipping all the bits. Addition is also much simpler than in sign-magnitude, though it requires an end-around carry, and there are still two representations of zero: 0000 and 1111.

Bit Patterns#

What is an end-around carry, then? And can 1100 mean both 12 and −3?

Before discussing end-around carry, we need one premise: the same sequence of bits can be interpreted as different numbers under different rules. The computer cannot infer your intended meaning from the bits alone.

Take 1100, occupying 4 bits:

Interpretation	Value of 1100
Unsigned integer	12
Sign-magnitude	−4
Ones’ complement	−3
Two’s complement	−4

All four answers are correct. 1100 is only a bit pattern; it does not carry a label saying “I am 12” or “I am −3.”

This is much like 01000001 in memory: you can interpret it as the integer 65 or, using ASCII, as the character A. There is no contradiction.

How does the computer know which interpretation to use? Usually, the program’s data types, instructions, and conventions decide. In C, for example, uint8_t denotes an 8-bit unsigned integer, while int8_t denotes an 8-bit signed integer. Even when they store the same 8 bits, the program can interpret those bits differently.

With that premise in place, end-around carry becomes easier to understand.

Let us calculate $5 - 3$ using ones’ complement. First, fix the rule: all the following numbers use 4-bit ones’ complement.

Positive 5 is 0101, and negative $-3$ is 1100.

Add directly

0101

+1100

10001

Add the carry back

0001 + 0001 = 0010

This is end-around carry: instead of discarding the carry out of the most significant bit, wrap it around and add it to the least significant bit. This is the rule for ones’ complement addition.

From Line to Ring#

You might ask why we need to do something so strange.

I think we need to return to a more fundamental question:

If a computer can store only finitely many numbers, how can the infinitely extending integer number line fit into that finite space?

We are still using only 4 bits. From 0000 through 1111, there are just $2^4 = 16$ bit patterns. If each encoding gets its own position, we should have 16 positions. Keep that count in mind.

Of course, finitely many encodings cannot correspond one-to-one with infinitely many integers. But we can try a different idea: allow some distinct integers to share a position.

Consider −3, 12, and 27 on an ordinary number line. They are three different numbers, but consecutive numbers in this list differ by exactly 15.

Why choose 15? Look back at ones’ complement: flipping every bit of 0011 gives 1100. Read them temporarily as unsigned integers, and they are 3 and 12, which add up to exactly 15.

This is no coincidence. Any 4-bit encoding plus its bitwise complement adds up to 1111, or 15. Thus, the ones’ complement encoding of $-3$, read as unsigned, is $15 - 3 = 12$; for $-5$, it is $15 - 5 = 10$.

Now make a rule: positions on the number line that differ by 15, or any integer multiple of 15, belong to the same class.

This puts −3, 12, and 27 together, because:

$$ -3 = -1 \times 15 + 12, \qquad 12 = 0 \times 15 + 12, \qquad 27 = 1 \times 15 + 12 $$

Each can leave the same remainder, 12, upon division by 15. In mathematics, this relation is called congruence, written:

$$ -3 \equiv 12 \equiv 27 \pmod{15} $$

This does not say that the three integers are equal. It says that if we care only about their remainders upon division by 15, we no longer distinguish them. Addition and subtraction preserve this relation. For example, −3 and 12 are congruent; adding 5 to each gives 2 and 17, which are still congruent.

Next, roll up the number line: 0 overlaps 15, 1 overlaps 16, 2 overlaps 17, and so on. Positions exactly 15 apart overlap. The infinite number line can now be represented by a ring of 15 positions, labeled 0 through 14.

Number line Roll into a ring

4 bits: 16 bit patterns

On the number line, −3, 12, and 27 are each 15 apart.

On this ring, going back 3 steps from 0 and going forward 12 steps reach the same position. This gives a clear reason for letting 1100 play the role of $-3$: its unsigned value, 12, belongs to the same congruence class as $-3$ modulo 15.

Wait: we just said that 4 bits give 16 bit patterns. Why does the ring have only 15 positions? Where did the missing one go?

Place all 16 encodings, from 0000 to 1111, onto the ring one by one. The first 15, read as unsigned, are 0 through 14, occupying one position each. That leaves 1111, whose unsigned value is 15. But under our chosen rule, modulo 15:

$$ 0 \equiv 15 \pmod{15} $$

So 1111 lands on the zero position already occupied by 0000. No encoding is missing; two encodings share one position. In ones’ complement, 0000 is positive zero and 1111 is negative zero. The two zeros overlap on the ring.

The ring’s 15 positions therefore come from our choice of modulo 15. All 16 encodings still exist; under this rule, 0000 and 1111 simply do not occupy distinct positions.

End-Around Carry#

Return to $5 - 3$. Start at $-3$ and add 1 at a time, taking 5 steps in total. The ring below labels remainder positions; compare each step’s actual value with its ones’ complement encoding:

+1 step Start over 0 / 5

−3 + 0 = −3; ones' complement 1100

On the third step, we reach zero with the encoding 1111: negative zero. One more step should take us from zero to $+1$, encoded as 0001.

But an ordinary binary adder computes:

$$ \mathtt{1111} + \mathtt{0001} = \mathtt{1\ 0000} $$

If we simply discard the leftmost carry and keep 0000, the result is still zero. We have gone from negative zero to positive zero without reaching the $+1$ we should have reached.

Why are we one step short? We are grouping by 15, but the carry beyond the fourth bit has a value of 16:

$$ 16 = 15 + 1 \equiv 1 \pmod{15} $$

That carry is therefore still equivalent to a 1 in the least significant position; we cannot simply discard it. Adding it back gives 0001. This is where end-around carry comes from.³

Likewise, adding the bit patterns in our earlier $5 - 3$ example gives:

$$ 5 + 12 = 17 = 16 + 1 \equiv 1 + 1 = 2 \pmod{15} $$

In the circuit, we first obtain 1 0001, then add the carry of 1 back into the low four bits to get 0010. End-around carry makes an ordinary binary adder perform this ones’ complement calculation according to the rules of modulo 15.

Two’s Complement#

Could we make the encodings of negative numbers work modulo 16 as well, eliminating end-around carry?

Keep the same 4 bits and the same $-3$. We want adding it to $+3$ to take us back to zero.

Under the modulo-15 rule, 3 plus 12 gives 15, which returns to zero. That is why ones’ complement encodes $-3$ as 1100.

If we switch to modulo 16, what should we add to 3 to return to zero?

The answer is 13:

$$ 3 + 13 = 16 \equiv 0 \pmod{16} $$

So we change the encoding of $-3$ to 1101. On the 16-position ring, it sits exactly three steps backward from zero.

Calculate $5 - 3$ once more:

$$ \mathtt{0101} + \mathtt{1101} = \mathtt{1\ 0010} $$

This time, simply discard the carry beyond the fourth bit. The remaining 0010 is the 2 we wanted, with no need to add the carry back.

This representation is two’s complement.

Looking back, the two encodings differ by exactly 1:

$$ \begin{aligned} \text{Ones' complement:}&\quad 15 - 3 = 12 = \mathtt{1100}_2 \\ \text{Two's complement:}&\quad 16 - 3 = 13 = \mathtt{1101}_2 \end{aligned} $$

Now the textbook instruction “flip the bits and add one” has a reason behind it. For a negative number $-x$, where $x$ is its absolute value, flipping the bits first gives $15-x$. Adding 1 then gives the encoding that adds to $x$ to make 16. Generalizing to $n$ bits, use $2^n-x$ to represent an in-range, nonzero negative number $-x$, retaining only those $n$ bits.

Switch between ones’ complement and two’s complement below. Both calculate the same $5 - 3$:

Ones' · 15 positions Two's · 16 positions

−3: 1101 → result: 0010 (2)

Discard carry: 1 0010 → 0010

Two’s complement gives the encoding the same period as addition that retains only the low four bits. Within this period, 16 is already congruent to 0, so the carry can be discarded directly.

What happened to the two zeros?

Positive zero is still 0000. Add 1 to the old ones’ complement encoding of negative zero, 1111:

$$ \mathtt{1111} + \mathtt{0001} = \mathtt{1\ 0000} \quad\longrightarrow\quad \mathtt{0000} $$

Retaining only four bits brings it back to 0000 too. The two zeros have merged. On the modulo-16 ring, each of the 16 bit patterns can occupy its own position. Two encodings no longer need to share the zero position.

Signed Overflow#

Another question arises: ones’ complement could represent only 15 distinct integers, while two’s complement can represent 16. What is the extra number?

First, see where the old negative encodings went:

Value	Ones’ complement	Two’s complement
−1	1110	1111
−2	1101	1110
−3	1100	1101
…	…	…
−7	1000	1001

Every nonzero negative encoding has increased by 1. The 1000 that represented $-7$ has become 1001, leaving 1000 available.

If encodings whose most significant bit is 0 still represent nonnegative numbers, then 0000 through 0111 already cover 0 through 7. Encodings whose most significant bit is 1 represent negative numbers; 1001 through 1111 already cover $-7$ through $-1$. The remaining 1000 fits just before $-7$, representing $-8$.

On the ring, this position has a special property: starting from zero, half a turn in either direction reaches it.

Eight clockwise steps and eight counterclockwise steps reach the same remainder position, so:

$$ +8 \equiv -8 \pmod{16} $$

Under the two’s complement rule that a most significant bit of 1 denotes a negative number, 1000 is interpreted as $-8$.

The range of 4-bit two’s complement is now complete: 0000 through 0111 represent 0 through 7, and 1000 through 1111 represent $-8$ through $-1$. There are 8 nonnegative numbers and 8 negative numbers. Because zero occupies one of the nonnegative positions, the magnitudes at the two ends are asymmetric.

For $n$ bits, the range of a two’s complement integer is:

$$ -2^{n-1} \le x \le 2^{n-1}-1 $$

What happens if we insist on representing $+8$ with 4 bits?

Integer to represent

+8

−8 +8

Keep low four bits 1000 → read as −8 in two's complement

+8 is outside the range −8 to 7 and cannot be represented in 4-bit two's complement.

For an input of $+8$, retaining the low four bits does leave 1000, but two’s complement still reads it as $-8$. The bit pattern cannot distinguish the “$+8$ we wanted” from the “$-8$ we can represent.” The former is already out of range.

The position opposite zero also explains a puzzling phenomenon: what happens if we flip the bits of 1000, the encoding of $-8$, and add one?

$$ \mathtt{1000} \xrightarrow{\text{flip bits}} \mathtt{0111} \xrightarrow{\text{add one}} \mathtt{1000} $$

We get the same encoding back, because modulo 16:

$$ 8 + 8 = 16 \equiv 0 \pmod{16} $$

On the ring, 8 is its own additive inverse. Among ordinary integers, however, the negative of $-8$ should be $+8$, and $+8$ cannot be represented in 4-bit two’s complement. This is signed overflow.

Here, 0111 + 0001 = 1000 produces no carry into a fifth bit. A carry out and signed overflow are different things: the former asks whether binary addition exceeds the bit width; the latter asks whether the mathematical result exceeds the range of the chosen encoding.

We can now place the three representations side by side:

4-bit representation	Sign-magnitude	Ones’ complement	Two’s complement
Encoding of −3	1011	1100	1101
Encodings of zero	0000, 1000	0000, 1111	0000
Integer range	−7 to 7	−7 to 7	−8 to 7
Addition and subtraction	Handle sign and magnitude separately	Add the carry back	Retain a fixed bit width for the modular result

What changes between the three is the mapping from encodings to integers. Two’s complement aligns that mapping with the adder’s rule of retaining the low bits.

Fixed and Floating Point#

We can now represent and calculate with positive and negative integers. But another familiar kind of number still has no place: fractions. Where should 2.5 go, for example?

Fixed Point#

Set signs aside for a moment and consider 4-bit unsigned integers. We have 16 encodings, originally interpreted as 0 through 15. Now make a new rule: divide the integer represented by each encoding by 4.

Then 0000 represents 0, 0001 represents 0.25, 0010 represents 0.5, and so on. 1010 can now represent 2.5.

Fractions have appeared, at a cost: the maximum was 15, but is now only 3.75. The number of encodings has not increased; each unit has simply become smaller.

Unit size

0.25

Spacing 0.25 · Maximum 3.75

This is the idea of fixed point: agree on each bit’s weight in advance, keeping the binary point in a fixed position. A larger unit covers a wider range but moves neighboring positions farther apart. A smaller unit gives finer spacing but a narrower range.

Significands and Exponents#

Could numbers at different scales use different spacing?

To see the layout of the scale first, let us keep just three significant binary digits per number and leave exponent storage for later. These three digits include the one before the binary point, giving 1.00, 1.01, 1.10, and 1.11.

The weights to the right of the binary point are $1/2$, $1/4$, and so on. For example:

$$ 1.10_2 = 1 + \frac{1}{2} + 0\times\frac{1}{4} = 1.5 $$

Multiply each of these four significands by $2^2$ and by $2^3$:

Scale	Representable numbers	Adjacent spacing
$1.xx_2 \times 2^2$	4, 5, 6, 7	1
$1.xx_2 \times 2^3$	8, 10, 12, 14	2

The number of significant digits has not increased, but the entire set of marks has doubled in scale. Both range and spacing have grown.

Absolute and Relative Spacing#

Before calling this “precision,” let us measure the distance between two neighboring positions.

Select 6. The next representable number to its right is 7, a distance of $7-6=1$. This is the absolute spacing, denoted $\Delta$. Select 12, and the next representable number is 14, giving an absolute spacing of $14-12=2$.

The distance has indeed increased. But how large is 1 relative to 6, compared with 2 relative to 12?

If the length from 0 to 6 is treated as one unit, the gap of length 1 occupies just $1/6$ of that unit. Do the same for 12, and the gap of length 2 occupies $2/12=1/6$ of the unit.

This is relative spacing: divide the spacing by the magnitude of the current value. For now, we are looking only at positive numbers. When including negatives, the reference length is $|x|$, so the ratio is $\Delta/|x|$.

The two diagrams below show the same pair of neighboring numbers. The left retains the original values. The right divides every distance by the current $x$, fixing the current position at 1 and placing the next at $\text{next}/x$. The blue brackets measure $\Delta$ and $\Delta/x$, respectively. Compare 6 and 12: watch the distance change on the left, and see whether its proportion of the current value changes on the right.

612

Selected value x

6

Original values

Δ = 7 − 6 = 1

Treat x as one unit

1 / 6 ≈ 16.7%

Drag a little farther right. Within the group 4, 5, 6, 7, the absolute spacing stays at 1, while relative spacing is $1/4$, $1/5$, $1/6$, and $1/7$. Select 8, and the next position becomes 10: absolute spacing jumps to 2, and relative spacing returns to $2/8=1/4$.

Relative spacing is therefore not the same everywhere. It decreases within one scale interval, jumps back at the next, and repeats. This is what it means to retain roughly the same relative resolution at different scales.

This is the core idea of floating point: the significand determines the position within a scale, and the exponent determines the size of that scale.

Normalization#

This resembles decimal scientific notation. For example, 12300 can be written as $1.23\times10^4$. In binary, 6 can be written as:

$$ 110_2 = 1.10_2 \times 2^2 $$

But the same number could also be written as $11.0_2\times2^1$ or $0.110_2\times2^3$. Allowing all these forms would give multiple encodings for the same number.

So we first choose a standard form: keep the significand in $[1,2)$ and write $1.xxx\ldots_2\times2^e$. This is normalization. Within the exponent range permitted by a format, numbers represented in this form are called normal numbers. Their values have not changed. We have simply standardized the position of the binary point to make full use of the available significant bits.⁴

We can also read the spacing directly from the weight of the last bit. With three significant digits, moving from 1.00 to 1.01 adds just $1/4$. After multiplying the whole scale by $2^e$, the smallest increment becomes $2^e/4$. Thus an exponent of 2 gives spacing 1, and an exponent of 3 gives spacing 2.

With $p$ significant digits in total, there are $p-1$ digits after the binary point, and the last has weight $2^{-(p-1)}$. Multiplying by the scale $2^e$ gives, within $[2^e,2^{e+1})$:

$$ \Delta = 2^e \times 2^{-(p-1)} = 2^{e-(p-1)} $$

One ULP can be understood through the weight of that last bit at the current scale. It measures the step between representable positions; we will need to distinguish it from actual computational error later.

Rounding#

Now a problem appears. Keep three significant digits, and provisionally limit the exponents to $-2$ through 3. As we saw, in the interval $[8,16)$ we can represent only 8, 10, 12, and 14.

Where does 9 go?

In $8+1$, both operands are exactly representable. The mathematical result, 9, is also within the format’s range, yet it falls between the marks. To store it, we must choose a nearby representable number.

This is rounding.

IEEE 754 defaults to rounding to the nearest value. If a result is exactly halfway between two candidates, choose the one whose least significant retained binary digit is 0: “ties to even.”⁵

For 9, the two candidates are:

$$ 8 = 1.00_2\times2^3, \qquad 10 = 1.01_2\times2^3 $$

The value 9 is exactly halfway, and the significand of 8 ends in 0, so $8+1$ rounds to 8 at this three-bit precision. For $10+1$, the result 11 is also halfway, but the selected value is $1.10_2\times2^3=12$.

8 + 110 + 1

Mathematical result

9 → 8

Exactly halfway · Ties to even

This differs from the integer arithmetic above: integers remain exact when the result stays in range, whereas a floating-point result can remain in range and still fall between two marks.

For basic operations specified by IEEE 754, the result must be equivalent to computing the exact mathematical result first and then rounding to the target format under the chosen rule. This is correct rounding. It specifies the final result; it does not require hardware to store an infinitely precise intermediate number. How a circuit produces that result efficiently is a separate implementation question.⁵

We can now distinguish “spacing” from “error.” The absolute spacing between 8 and 10 is 2. Rounding 9 to 8, however, gives an absolute error of $|8-9|=1$. Relative to the exact mathematical result 9, the relative error is $1/9$. If the input is the representable number 8 itself, rounding error is zero, while the spacing to its right remains 2.

Spacing describes how finely the scale can distinguish values; error describes how far a particular rounding has moved a value.

Subnormal Numbers#

Now look near zero. Normalization requires a leading 1 in the significand. If the smallest exponent is already $-2$, the smallest normal number is:

$$ 1.00_2\times2^{-2}=0.25 $$

What about the interval between 0 and 0.25? If we keep requiring a leading 1, no nonzero number can fit there.

We can hold the exponent at $-2$ and allow the significand to begin with 0:

Representation	Value
$0.00_2\times2^{-2}$	0
$0.01_2\times2^{-2}$	0.0625
$0.10_2\times2^{-2}$	0.125
$0.11_2\times2^{-2}$	0.1875
$1.00_2\times2^{-2}$	0.25

This fills a few positions between zero and the smallest normal number, all spaced 0.0625 apart. The three added nonzero values between 0 and 0.25—0.0625, 0.125, and 0.1875—are subnormal numbers. The value 0.25 itself remains the smallest normal number.

We already know how to measure both kinds of spacing. Select the smallest normal number, 0.25. Its right neighbor is 0.3125, giving an absolute spacing of 0.0625, or $1/4$ of the current value. Select the smallest nonzero number, 0.0625, instead. Its right neighbor is 0.125: absolute spacing is still 0.0625, but now it is as large as the current value itself.

Below, the region near zero is expanded on its own. Drag left: the blue gap on the left stays the same length, while on the right, where the current value is treated as one unit, the same gap grows longer.

0.250.0625

Selected value x

0.25

Original values

Δ = 0.3125 − 0.25 = 0.0625

Treat x as one unit

0.0625 / 0.25 = 25%

The exponent cannot get any smaller, so absolute spacing stays at 0.0625 instead of shrinking with the values. Moving from the smallest normal number toward zero, the selected value passes through $4/16$, $3/16$, $2/16$, and $1/16$, while relative spacing becomes $1/4$, $1/3$, $1/2$, and 1.

This is the concrete meaning of “gradually losing relative precision”: the marks have not become farther apart, but their spacing has become larger relative to the current value. Continuing toward zero with subnormal numbers in this way is called gradual underflow. At zero, we can still measure the gap to the smallest positive number, but relative spacing is undefined because its denominator is zero.⁶

There is a boundary at the other end too. Under our current rules, the largest normal number is $1.11_2\times2^3=14$. In the default rounding mode, sufficiently large results overflow to infinity. Invalid operations such as $0/0$ need NaN as a marker.

IEEE 754#

We have now decided how to arrange the numerical scale, round values that fall between its marks, and handle its boundaries. Only now do we put those rules into bits.

Let us build a 6-bit toy floating-point format: 1 sign bit S, 3 exponent bits E, and 2 fraction bits F.

A normalized significand always begins with 1. For normal numbers, we can therefore omit this 1 from storage and restore it when reading. The fractional part that is actually stored is the fraction field. For example, if the full significand is 1.10, F only needs to store 10. Two fraction bits plus the implicit leading 1 give three bits of precision.

The exponent can be negative, so we add a fixed bias before storing it. Our toy format uses a bias of 3: for ordinary exponent codes, the actual exponent is $e=E-3$. Here, E and F are first read as unsigned integers.

Reserve the two endpoint exponent codes: E=000 for zero and subnormal numbers, and E=111 for infinity and NaN. The remaining codes, E=1 through 6, correspond exactly to the exponents $-2$ through 3 used above.

For subnormal numbers, the leading bit becomes 0 and the exponent stays fixed at the smallest normal exponent, $-2$.

For example, $6=1.10_2\times2^2$, so S=0, E=$2+3=5$, or 101, and F=10. The final encoding is 0 | 101 | 10.

Sign S

0

Exponent E

101

Fraction F

10

6Normal number

1.10₂ × 2²

ResetMin normalMax subnormalInfinityNaN

The sign bit S independently determines whether the value is positive or negative.

Applying the same idea to the real binary32 format gives:

Field	Bits	Role
S	1	Sign
E	8	Biased exponent, with a bias of 127
F	23	Fraction field; normal numbers have an additional implicit leading 1

Normal numbers therefore have 24 bits of binary precision, with the value:

$$ x=(-1)^S\left(1+\frac{F}{2^{23}}\right)2^{E-127},\qquad 1\le E\le254 $$

When E=0, the leading bit becomes 0 and the actual exponent is fixed at $-126$. E=255 is reserved for infinity and NaN.⁷

Exponent field E	Fraction field F	Meaning
0	0	+0 or −0, determined by S
0	Nonzero	Subnormal number
1–254	Any	Normal number
255	0	+∞ or −∞, determined by S
255	Nonzero	NaN

For example, binary32’s smallest normal number is $2^{-126}$, and its smallest positive subnormal number is $2^{-149}$. The minimum spacing near zero is therefore $2^{-149}$ too.

Different formats change this allocation of bits: more exponent bits generally cover more orders of magnitude; more significant bits give finer spacing within the same scale. The common binary16, binary32, and binary64 formats provide 11, 24, and 53 bits of binary precision for normal numbers, respectively.

Precision and Error#

We have measured two distances locally: the absolute spacing $\Delta$ between neighboring marks, and its size relative to the current value, $\Delta/|x|$. Extending the same measurements across the entire finite numerical space gives the two curves below.

The Big Picture#

The overview uses logarithmic coordinates to compress a vast range and reveal the overall trend. Start with the smooth trend, then switch to exact spacing to revisit the plateaus, steps, and changing ratios measured earlier. The two sides are shown symmetrically using the interval step size at each magnitude.

An overview of absolute and relative floating-point spacing

The floating-point scale

FP16 FP32 FP64

Smooth trend Exact spacing

Expand near zero

Absolute spacing Δ

2⁻¹⁴⁹

Absolute spacing increases with magnitude and forms a constant plateau near zero.

Relative spacing Δ / |x|

Undefined at zero

Relative spacing is roughly stable in the normal region and increases in the subnormal region; zero is left blank.

Current position x

0

Back to zero

Piecewise log axis · Expanded near zeroCool: small spacing · Warm: large spacing

How to read the charts

The vertical axis shows powers of 2. Smooth mode displays a continuous upper envelope; exact mode preserves jumps at scale boundaries. Cursor readings always use the interval step of an actual representable value. Both sides are calculated symmetrically by magnitude, with inward spacing used at the maximum finite value.

In the absolute-spacing chart, marks generally become sparser as the magnitude of the value increases. In the subnormal region, spacing stops shrinking and forms a plateau.

Relative spacing behaves differently. In the normal region, both values and spacing grow or shrink with the scale, keeping their ratio within a small range. For binary32, the ratio of interval step size to value lies roughly between $2^{-24}$ and $2^{-23}$. The exact curve slopes downward within each scale interval and jumps back at its boundary; it is not perfectly horizontal.

In the subnormal region, absolute spacing stays fixed while values keep shrinking, so relative spacing grows. At the smallest positive subnormal number, the value itself equals one step, giving a ratio of 1. This ratio is undefined at zero, so the chart leaves a separate gap there.

This also explains why “floating point lacks precision” is often too vague. We need to distinguish between absolute spacing that is too large and differences that are too coarse relative to the current value.

Input and Arithmetic Errors#

With this scale in mind, we can distinguish the sources of common errors in real programs.

The first kind arises when the input itself falls between the marks. Decimal 0.1 is an example: a finite binary fraction can only have a power of 2 as its reduced denominator, whereas $0.1=1/10$ has a factor of 5 in its denominator. It cannot be represented exactly with finitely many binary digits and must be rounded when read.

The second arises when the inputs lie on the marks but the result does not, as in our $8+1$ example. A similar case occurs in real binary32: $2^{24}=16777216$ is exactly representable, but the spacing toward larger numbers is already 2.

Rounding the sum back to binary32 under the default rule therefore gives:

$$ \operatorname{fl}_{32}(16777216+1)=16777216 $$

If each step is stored at this precision, the order of calculation can also affect the result. Add 1 to 16777216 first, then subtract 16777216, and the answer is 0. Calculate $1-16777216=-16777215$ first instead, and that intermediate result is exactly representable; adding 16777216 then gives 1.

The same three operands produce different results when only the parentheses change. Rounding can discard information, and later operations must continue from the value that was actually stored.⁵

Back to Bits#

Return to the question at the start: how does a computer store numbers?

It stores finitely many bit patterns. We assign numerical values to these states and specify how they participate in arithmetic. Two’s complement connects fixed-width addition and subtraction to modular arithmetic. Floating point arranges marks of different densities at different scales and uses rounding when results fall between them.

Many seemingly strange phenomena now have a source: negating the most negative integer returns the same bit pattern, adding one to a large number changes nothing, and moving parentheses changes the answer.

For me, the most valuable part of returning to this material was that formulas I had once memorized separately finally joined into one thread: how can finite states preserve as much of the mathematical information we need as possible?