Machine Numbers

Or, how reasonable-sounding engineering decisions leave you with a really weird subset of the rationals.

Definitions.

Definition: The set of numbers in the form

		 (((-1))^s)×(2^exponent) ×mantissa 

where the mantissa and exponent are binary numbers with a fixed number of digits (Subscript[m, b]) and (Subscript[e, b]) 
are called the floating point numbers with (Subscript[m, b]) mantissa bits and (Subscript[e, b]) exponent bits. 

Each of these numbers requires

		(Subscript[m, b]) + (Subscript[e, b])+ (Overscript[1, (Overscript[⏞, sign bit])])
		
bits of storage.

Binary numbering systems

The binary digits (Subscript[b, l])(Subscript[b, l-1])...(Subscript[b, 0]).(Subscript[b, -1])(Subscript[b, -2])...(Subscript[b, -r]) of an unsigned binary number 
with l digits to the left of the binary point and r digits to the right of the binary point
represent the number

	(Subscript[b, l])× (2^l)+ (Subscript[b, l-1])× (2^(l-1))+ ... + (Subscript[b, 0])×(2^0)+(Subscript[b, -1])×(2^(-1))+...+((Subscript[b, -r])^(-r)) =(∑_(i=-r)^(l-1))(Subscript[b, i])×(2^i) 
	
Example:

	1.01 = 1 (2^0)+0 (2^(-1))+ 0 (2^(-2))
	        = 1+(1/4)

Mantissa bits consist of digits (Subscript[b, 0]).(Subscript[b, -1])...(Subscript[b, mb-1])

BinaryMantissa[m_] := Sum[m[[i]] (2^(-(i-1))),{i,1,Length[m]}];

BinaryMantissa[{1,0,1}]

Offsets and exponents

The exponent stores digits (Subscript[b, eb-1])(Subscript[b, eb-2])...(Subscript[b, 0]). To allow for negative exponents, 
we subtract the biased offset (2^(eb-1))-1. So these digits represent:

	(Subscript[b, eb-1])× (2^(eb-1))+ (Subscript[b, eb-2])× (2^(eb-2))+ ... + (Subscript[b, 0])×(2^0) - ((2^(eb-1))-1) =(∑_(i=0)^(eb-1))(Subscript[b, i])×(2^i)-((2^(eb-1))-1) 
	
Example: The largest exponent is all 1’s.

	(∑_(i=0)^(eb-1))(2^i)-((2^(eb-1))-1) = (2^eb)-1-(2^(eb-1))+1 = (2^(eb-1)).

Example: The smallest exponent is all 0’s.	

	-((2^(eb-1))-1) 
	
Example (of example): For eb=8, that’s a range from

{-((2^(eb-1))-1),(2^(eb-1))} /. eb->8

Generating exponents from bit strings.

We can implement that rule in Mathematica as follows:

BinaryExponent[b_] := Sum[b[[i]]*(2^(Length[b]-i)),{i,1,Length[b]}]-((2^(Length[b]-1))-1);

BinaryExponent[{1,1,1,1,1,1,1,1}]

BinaryExponent[{0,0,0,0,0,0,0,0}]

Generating all the floating point numbers.

We now learn a little bit of Mathematica programming.

Tuples[{0,1},2]

Outer[Times,{a,b},{c,d}]

f /@ {a,b,c,d}

Flatten[{{a,b},{c,d}}]

2^#& [7]

FloatingPointNumbers[mb_,eb_] := Flatten[Outer[Times,
BinaryMantissa /@ Tuples[{0,1},mb],
2^#& /@ BinaryExponent /@ Tuples[{0,1},eb]]];

Looking at all the floating point numbers:

Suppose we have 3 mantissa bits and 2 exponent bits. Then we have the (positive) numbers:

Sort[FloatingPointNumbers[3,2]]

Graphics[{PointSize[Medium],Blue,Point[{#,0}]}& /@ FloatingPointNumbers[3,2],Axes->True]

Note: Weird spacing, redundancy in the list above. Why?

Normalized machine numbers.

In the convention above, we had several ways to represent the number 1:

1.00 x (2^0), 0.10 x (2^1), 0.01 x (2^2)

It seems more efficient to eliminate the ambiguity by only considering 
mantissas where the leading digit is 1.  But if the 1 is always present, why store it?

Definition. The normalized mantissa represented by (Subscript[b, -1])(Subscript[b, -2])...(Subscript[b, -mb]) is the number

	1.(Subscript[b, -1])(Subscript[b, -2])...(Subscript[b, -mb])=1+(∑_(i=1)^mb)(Subscript[b, -i])(2^(-i))

NormalizedBinaryMantissa[m_] := 1+Sum[m[[i]] (2^(-i)),{i,1,Length[m]}];

NormalizedBinaryMantissa[{1,0,0}]

Comparing normalized and standard floating point numbers

We now compare the two:

NormalizedFloatingPointNumbers[mb_,eb_] := Flatten[Outer[Times,
NormalizedBinaryMantissa /@ Tuples[{0,1},mb],
2^#& /@ BinaryExponent /@ Tuples[{0,1},eb]]];

Sort[NormalizedFloatingPointNumbers[3,2]]

Graphics[{{PointSize[Medium],Red,Point[{#,-1}]}& /@ NormalizedFloatingPointNumbers[3,2],
{PointSize[Medium],Blue,Point[{#,1}]}& /@ FloatingPointNumbers[3,2]},Axes->True]