Programming with Python › Python Setup and First Programs › Day 46

Day 46: Numbers, Math, and Precision

Day 46 of 365 — Numbers, Math, and Precision

After this lesson you will be able to choose the right Python number type for any task, explain why 0.1 + 0.2 is not exactly 0.3, compare floats safely, and use the decimal module for exact money math.

Course
Programming with Python
Category
Python Setup and First Programs
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/programming-with-python/day-046-numbers-math-and-precision

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/programming-with-python/day-046-numbers-math-and-precision
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters


The idea in plain language

Diagram: exact integer arithmetic compared with approximate float arithmetic


Historical background


What it is — and what it is not

A float is:

It is not:


Why it was created and what problems it solves


How it works

How a float is stored

Diagram: why the decimal fraction one tenth repeats in binary

Diagram: a 64-bit float split into sign, exponent, and mantissa, and why one tenth cannot be stored exactly

Comparing floats

Diagram: exact float equality compared with a tolerance check

0.1 + 0.2 == 0.3            # False
math.isclose(0.1 + 0.2, 0.3)  # True

Integer division, modulo, and rounding

Diagram: true division, floor division and modulo in Python

When exactness matters

Diagram: constructing a Decimal from a float compared with from a string

Diagram: int, float, Decimal, and Fraction compared with the trade-off each makes

TypeExact?SpeedUse it for
intYes, unboundedFastCounting, indexing, identifiers
floatNoFastestMeasurement, science, anything with a tolerance
DecimalYes, decimalSlowMoney, and anything audited
FractionYes, rationalSlowestExact ratios, symbolic work

Special values

Diagram: how a nan value spreads through subsequent computations


An everyday analogy

A ruler with finite markings.

MeasuringFloats
The finest marking on the rulerThe mantissa’s precision
Reading to the nearest markingRounding to the nearest representable value
A length that falls between markingsA fraction with no exact representation
Adding many measured lengthsAccumulated error
”Within a millimetre”math.isclose
A tape measure that reaches further, less finelyThe exponent trading precision for range

Examples in practice

Diagram: how rounding error accumulates across a long summation


Implications: security, privacy, performance, scalability, and cost


Alternatives: free, open source, and commercial

Diagram: 32-bit floats compared with 16-bit floats

ToolTypeWhere it fits
math.iscloseFree, standard libraryComparing floats, always
decimal.DecimalFree, standard libraryMoney, invoices, anything audited
fractions.FractionFree, standard libraryExact ratios
math.fsumFree, standard libraryAn accurate sum of many floats
NumPyFree, open sourceArrays, and explicit control of precision
float32 / float16Free, in NumPy and frameworksHalf the memory, half the precision

TermWhat it actually is
MantissaThe significant digits
ExponentThe scale they are multiplied by
PrecisionHow many digits are kept
RangeHow large and small a value can be
infOverflow, as a value rather than an error
nanNot a number; unequal to everything, including itself
Banker’s roundingHalves go to the nearest even

When to use it — and when not to

Diagram: representing currency with floats compared with Decimal


The AI connection

Knowledge check

quiz:
  question: "A test asserts that 0.1 + 0.2 equals 0.3 and fails. What is the correct fix?"
  options:
    - "Compare with a tolerance, using math.isclose, because binary floats cannot represent either operand exactly"
    - "Round both sides to one decimal place before comparing them"
    - "Use Decimal(0.1) + Decimal(0.2) so the arithmetic is exact"
    - "Increase the float precision by using a 128-bit type"
  answer_index: 0
  explanation: "Neither 0.1 nor 0.2 has an exact binary representation, so their sum is the nearest representable value to 0.3 rather than 0.3 itself. Rounding hides the issue rather than addressing it, and Decimal(0.1) inherits the float's error before Decimal sees it — the string form would be needed. A tolerance asks the question that actually makes sense about approximations."

Hands-on exercise

See the error, measure it, then eliminate it where it matters. Worked through fully in the Day 46 lab directory.

What you are doingCommand
See the classic result0.1 + 0.2
See what is really storedformat(0.1, '.20f')
Compare properlymath.isclose(0.1 + 0.2, 0.3)
Find the integer limit2**53 == 2**53 + 1
Use exact decimalsDecimal("0.1") + Decimal("0.2")
See the float trapDecimal(0.1)
Sum accuratelysum([0.1]*10) against math.fsum([0.1]*10)

Expected output

0.30000000000000004
0.10000000000000000555
True
True
0.3
0.1000000000000000055511151231257827021181583404541015625
0.9999999999999999   1.0

Validate your work

Troubleshooting

Common mistakes


Practice assignment


Extension challenge


Quiz

Q1. In Python, what does the expression 0.1 + 0.2 == 0.3 evaluate to, and why?

  1. True, because Python rounds all decimals to two places automatically
  2. False — 0.1, 0.2 and 0.3 are not exact in binary, so it is off
  3. True, because floating-point arithmetic is always exact
  4. It raises an error, because you cannot compare floats
Show answer

Answer: B. False — 0.1, 0.2 and 0.3 are not exact in binary, so it is off

Each of 0.1, 0.2, and 0.3 is stored as the nearest available binary fraction, and the rounding errors do not cancel: 0.1 + 0.2 is stored as 0.30000000000000004, which is not the stored value of 0.3, so the comparison is False.

Q2. Which numeric type should you use to total a shopping cart so the amount reconciles exactly to the cent?

  1. float, because it is the fastest
  2. int, because money is always whole numbers
  3. Decimal, because it does exact base-10 arithmetic
  4. complex, because it has two parts
Show answer

Answer: C. Decimal, because it does exact base-10 arithmetic

Decimal performs exact base-10 arithmetic, so decimal fractions like 0.10 and 0.20 add up exactly. float uses base-2 and introduces rounding error; Decimal (or integer cents) is the correct choice for money.

Q3. What is the difference between 7 / 2 and 7 // 2 in Python 3?

  1. They are identical; both give 3.5
  2. 7 / 2 gives 3 (an int) and 7 // 2 gives 3.5 (a float)
  3. 7 // 2 raises an error because you cannot floor-divide odd numbers
  4. 7 / 2 gives 3.5 (float); 7 // 2 gives 3 (floor)
Show answer

Answer: D. 7 / 2 gives 3.5 (float); 7 // 2 gives 3 (floor)

In Python 3, / is true division and always returns a float (7 / 2 is 3.5), while // is floor division that divides and rounds down to a whole number (7 // 2 is 3).

Q4. You need to check whether two computed floats are effectively equal. What is the recommended approach?

  1. Use math.isclose(a, b), which tests equality within a small tolerance
  2. Use == directly; float comparison is always exact
  3. Convert both to strings and compare the strings
  4. Multiply both by 1000 and hope they match
Show answer

Answer: A. Use math.isclose(a, b), which tests equality within a small tolerance

Because floats carry tiny rounding errors, == often reports False for values that are mathematically equal. math.isclose(a, b) returns True when the values are within a small tolerance, which is the correct way to compare computed floats.

Q5. Why is Decimal("0.1") exact but Decimal(0.1) is not?

  1. There is no difference; both are exact
  2. Decimal(0.1) copies the float's error; the string is exact
  3. Decimal("0.1") is actually the inexact one
  4. Decimal(0.1) rounds to the nearest whole number
Show answer

Answer: B. Decimal(0.1) copies the float's error; the string is exact

Decimal(0.1) receives the float 0.1, which already holds 0.1000000000000000055..., and stores that error. Decimal("0.1") parses the string "0.1" and stores exactly one tenth. Always build money Decimals from strings.

Q6. What is special about Python's int type compared with integers in many other languages?

  1. Python ints are limited to 64 bits and overflow silently
  2. Python ints are stored as floats internally
  3. Python ints have arbitrary precision and never overflow
  4. Python ints can only hold values up to one million
Show answer

Answer: C. Python ints have arbitrary precision and never overflow

Python integers have arbitrary precision: they are exact and unbounded, growing to use as much memory as needed, so an expression like 2 ** 100 produces the full exact 31-digit result with no overflow.

Q7. A colleague uses Python's random module to generate password-reset tokens. What is the problem?

  1. random is too slow for generating tokens
  2. There is no problem; random is cryptographically secure
  3. random only produces whole numbers
  4. random is predictable; use the secrets module
Show answer

Answer: D. random is predictable; use the secrets module

The random module uses a predictable Mersenne Twister generator: given enough outputs its future values can be reconstructed. For anything that must be unguessable — tokens, passwords, one-time codes — use the secrets module, which draws from a cryptographically secure source.

Q8. What do the 52 mantissa bits and 11 exponent bits of a 64-bit float control, respectively?

  1. The mantissa controls precision; the exponent controls range
  2. The mantissa controls the sign and the exponent controls the color
  3. Both control only the range
  4. The mantissa controls range and the exponent controls precision
Show answer

Answer: A. The mantissa controls precision; the exponent controls range

In IEEE 754 double precision, the 52 mantissa (fraction) bits set the precision — about 15 to 17 significant decimal digits — while the 11 exponent bits set the scale, giving the enormous range up to about 1.8 x 10^308.

Glossary

int
Python's integer type: an exact whole number with arbitrary precision, meaning it grows as large as memory allows and never overflows.
float
Python's floating-point type: a number with a decimal point stored in 64 bits as an IEEE 754 double, fast and wide-ranging but limited to about 15 to 17 significant digits of precision.
IEEE 754
The international standard, published in 1985, that defines how binary floating-point numbers are laid out in bits and how arithmetic on them rounds, so the same program gives the same result on any conforming hardware.
floating-point error
The small, deterministic rounding difference that arises because most decimal fractions (like 0.1) have no exact representation in binary and must be stored as the nearest available value.
mantissa
The field of a floating-point number that holds its significant binary digits (also called the fraction); a 64-bit double has 52 mantissa bits, which set its precision.
exponent
The field of a floating-point number that sets the scale — which power of two the mantissa is multiplied by; a 64-bit double has 11 exponent bits, which set its range.
floor division
The // operator, which divides two numbers and rounds the result down to a whole number (toward negative infinity); 7 // 2 is 3.
modulo
The % operator, which returns the remainder left over after floor division; 7 % 2 is 1, and (a // b) * b + (a % b) always reconstructs a.
Decimal
A standard-library type (from the decimal module) that performs exact base-10 arithmetic to a chosen precision, correct for money; build it from a string, as Decimal("0.1"), never from a float.
Fraction
A standard-library type (from the fractions module) that stores a number as an exact ratio of two integers, so values like one third stay exact through arithmetic.
math.isclose
A function in the math module that reports whether two floats are equal within a small tolerance; the correct way to compare computed floats instead of using ==.
rounding
Reducing a number to a chosen number of digits; Python's built-in round uses banker's rounding (round half to even), so round(2.5) is 2 and round(3.5) is 4.
quantization
Storing numbers with fewer bits (for example a model's parameters in 8 bits instead of 32) to save memory and bandwidth, at the cost of coarser precision.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.