Floating-point is effectively stored as a fixed-precision integer (the
mantissa) multiplied or divided by a power of 2 (the exponent). There
are minor details, but this captures their essence. So 1.25 could be
stored as 101(binary) / 4. For better familiarity, the examples below
will use decimal (base 10) with a three-digit mantissa. For example,
1.25 would be stored as 125/100, but the same applies to binary.
*Absolute* precision decreases the farther a value is from zero, even
though it stays the same *relative* to the value. For example, with a
three-digit mantissa, 1.25 is representable, 1250 is, but 1025 isn't
(1020 and 1030 are the closest that are).
#----#----#----#---...---#------#------#---...---#------#------#
0 1.24 1.25 1.26 1020 1030 1040 1240 1250 1260
Since *relative* precision stays the same no matter how far from zero,
multiplying and diving doesn't alter the precision of a value much (it
can slightly, due to rounding). For example, multiplying 1.25 by 1000
results in 1250; no loss in precision, as dividing by 10 yields the
original value. Below, multiplication shifts the value to a new part of
the number line *and* changes the spacings accordingly, so that two
nearby small values become farther apart and thus are still represented
accurately:
...----#-----#-----#----...
1.24 1.25 1.26
/ | \
/ | \
...----#-------------#------------#----...
1240 1250 1260
Addition or subtraction of large and small values will eliminate some or
all of the smaller value. For example, adding 10 to 1.25 results in 11.3
(11.25 rounded to three digts), similar for subtracting. So repeated
addition/subtraction of small values with a large one can result in
significant error. Error can be reduced by adding the small values
together first, then adding/subtracting the sum with the larger.
Graphically, addition and subtraction merely shift the value along the
number line, possibly to a region where representable values aren't as
closely spaced:
...----#----#----#----#----#----#----#----#----#----#----#----...
1.20 1.21 1.22 1.23 1.24 1.25 1.26 1.27 1.28 1.29 1.30
...----#-------------------------------------------------#----...
11.2 11.3
Subtraction of values that are nearly the same will amplify any
imprecision. This is a common source of large errors, where a
calculation involves two sub-expressions that each have their own small
error, which are then subtracted from each other. For example,
1.25*1000-1.25*1001 = 1250-1250 (1251.25 rounded down) = 0. If this
difference is then the basis for the result, it will have a much higher
error, because most of the precision of the floating-point values was
canceled out by the subtraction (all in this example).
So the most important thing is to try to avoid having intermediate
values consist of a large value that needs to have high absolute
precision; attempt to eliminate or handle separately the large
component, so that values needing high absolute precision are close to
zero.