While integers are excellent for exact counting, they are inadequate for representing a vast range of numbers encountered in scientific, engineering, and graphical applications: numbers that are very large, very small, or contain fractional components. Floating-point numbers address this limitation by adopting a system analogous to scientific notation.

Representation of Fractional Values

• Unlike integers, floating-point numbers can accurately represent values with decimal (or binary) fractions, such as 3.14159, 0.001, or 2.718. This is indispensable for calculations that involve measurements, percentages, or non-whole quantities. Fixed-point numbers can represent fractions but have a limited range and fixed decimal point.

Representation of Very Large Numbers

• Floating-point numbers use an exponent to scale a base number, much like scientific notation (M times 10^E). This allows them to represent extremely large magnitudes, such as the number of atoms in a mole (6.022 times 10^23) or astronomical distances, which would overflow even a 64-bit integer.

Representation of Very Small Numbers

• Conversely, they can represent numbers incredibly close to zero, such as the mass of an electron (9.109 times 10^{-31} kg) or a tiny electrical current. These small values would underflow to zero in fixed-point or integer systems.

Dynamic Range

• The exponential scaling inherent in floating-point representation provides an enormous 'dynamic range' – the ratio between the largest and smallest non-zero numbers that can be represented. This allows calculations to span many orders of magnitude while maintaining a relatively consistent level of relative precision across that range.

A binary floating-point number in a computer is typically composed of three distinct parts, inspired by the scientific notation S times M times B^E (Sign times Mantissa times Base^Exponent):

1. Sign (S): This is a single bit that indicates the polarity of the number.
2. 0 typically represents a positive number.
3. 1 typically represents a negative number.
4. Exponent (E): This field represents the power to which the base (almost always 2 for binary floating-point numbers) is raised. This exponent determines the 'magnitude' of the number by 'floating' the binary (or decimal) point.
5. Mantissa (M) or Significand: This field represents the significant digits or the 'precision' of the number. It's the fractional part of the number, typically normalized to have a leading 1 (or 0, for special cases) before the binary point.

Normalization is a crucial step in floating-point representation that ensures a unique binary representation for most numbers and maximizes the precision within the available bits.

Principle

• For a non-zero binary floating-point number, it is always possible (and desirable) to shift the mantissa bits and adjust the exponent such that the binary point is immediately to the right of the first non-zero bit. In binary, this means the mantissa will always have a leading '1' before the binary point (e.g., 1.xxxx_2).

The 'Implied Leading 1'

• Since the first bit of a normalized binary mantissa is always 1, there's no need to store it explicitly in memory. This 'implied leading 1' (or 'hidden bit') effectively gives an extra bit of precision for the mantissa without consuming any storage space.

Example

• The binary number 101.11_2 is equivalent to 1.0111_2 times 22. The binary number 0.00101_2 is equivalent to 1.01_2 times 2^{-3}. In both cases, the mantissa is shifted until it is in the form 1.xxxx..._2.

The exponent field in floating-point numbers typically uses a biased representation (also called 'excess-K' or 'excess-N' representation) rather than two's complement for handling both positive and negative exponents.

Motivation

• Standard binary number systems (like unsigned or two's complement) have specific ranges and complexities for signed comparisons. By adding a fixed 'bias' value to the true exponent, the entire range of exponents (positive and negative) is mapped to a range of positive unsigned integers. This simplifies hardware design, particularly for comparing floating-point numbers.

Principle

• A constant 'bias' value is chosen. The actual numerical exponent (the 'true exponent') has this bias added to it before being stored in the exponent field.
• To retrieve the true exponent:
• True_Exponent = Stored_Exponent - Bias

Example

• If the bias is 127 (as in IEEE 754 single-precision):
• A true exponent of 0 would be stored as 0 + 127 = 127.
• A true exponent of +1 would be stored as 1 + 127 = 128.

While indispensable, floating-point arithmetic introduces inherent limitations that must be understood to avoid common pitfalls in numerical computation:

1. Finite Precision: Floating-point numbers represent a continuous range of real numbers using a finite number of bits. This means that only a discrete subset of real numbers can be represented exactly.
2. Rounding Errors: Due to this finite precision, almost every arithmetic operation on floating-point numbers involves some degree of rounding.
3. Loss of Significance (Catastrophic Cancellation): A particularly problematic form of rounding error occurs when two floating-point numbers of nearly equal magnitude are subtracted.
4. Non-Associativity of Addition/Multiplication: Floating-point arithmetic is not always strictly associative.
5. Limited Exact Integer Representation: While floating-point numbers can represent integers, they can only do so exactly up to a certain magnitude.
6. Special Values and Their Behavior: The existence of +∞ and NaN means that mathematical operations can produce non-numerical results.