Computing Foundations › Inside the Machine › Day 2

Day 2: The CPU: Fetch, Decode, Execute

Day 2 of 365 — The CPU: Fetch, Decode, Execute

After this lesson you will be able to open up the CPU in your mind — control unit, ALU, registers, program counter — trace any simple program through the fetch-decode-execute cycle by hand, and use clock speed, instruction sets, and pipelining to reason about real hardware choices, including why GPUs beat CPUs at bulk math.

Course
Computing Foundations
Category
Inside the Machine
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-002-the-cpu-fetch-decode-execute

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/computing-foundations/day-002-the-cpu-fetch-decode-execute
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

Every calculation your machine has ever performed passed through one small loop, repeated billions of times a second.

Diagram: raising clock speed compared with adding cores


The idea in plain language

A processor is a machine that reads one instruction, does it, and moves on — faster than you can imagine, and with no idea what any of it means.


Historical background

Diagram: the power wall and the shift to multiple cores

WhenWhat arrivedWhy it mattered
1945Von Neumann’s stored-program designThe cycle as we still run it
1971Intel 4004, the first commercial microprocessorA whole CPU on one chip
1978Intel 8086The ancestor of every x86 chip in use
Early 1980sThe RISC argumentFewer, simpler instructions, decoded cheaply
1985ARM’s first designThe philosophy that now runs every phone
~2005Clock speeds stallPower and heat end the megahertz race
2020Apple M1A RISC laptop chip outperforming high-end x86

What it is — and what it is not

It is a loop with four stages, driven by a clock, over instructions held in memory as data.

It is not any of these, and each misconception costs you later:


Why it was created and what problems it solves

Diagram: how the program counter creates sequence, loops and decisions

The cycle exists to solve one problem: making a machine general.


How it works

Diagram: the four stages of the fetch-decode-execute cycle

The parts inside a core

Diagram: inside the CPU — the control unit with program counter and instruction register, the ALU, and the registers, connected over a bus to memory

PartRole in the cycle
Program counterHolds the address of the next instruction, then advances
Instruction registerHolds the instruction while it is decoded
Control unitReads the pattern and steers every other part
RegistersThe fastest storage; hold the values in play right now
ALUDoes the arithmetic, and sets flags about the result
FlagsSingle bits recording facts like “the result was zero”
BusThe wiring to memory — far slower to answer than a register

The cycle, in slow motion

A toy CPU with four registers and four instructions — LOAD, ADD, PRINT, HALT — makes each phase visible. Real simulator output, adding 5 and 3:

PC=0  FETCH    LOAD R1,5
      DECODE   opcode=LOAD  dest=R1  value=5
      EXECUTE  R1 <- 5
      REGS     R1=5 R2=0 R3=0 R4=0

PC=2  FETCH    ADD R1,R2->R3
      DECODE   opcode=ADD  src1=R1  src2=R2  dest=R3
      EXECUTE  R3 <- 5 + 3 = 8

Overlapping the stages

Diagram: four instructions overlapping in the fetch, decode, execute, and write-back pipeline stages


An everyday analogy

Diagram: an assembly line as an analogy for pipeline stages

An assembly line, and the reason it is not a single craftsman.

The lineThe processor
A station that fetches the next partFetch
A station that reads the work orderDecode
A station that does the workExecute
A station that puts the finished item awayWrite back
One item leaving the line per beatOne instruction completing per cycle
Each item still visiting four stationsFour cycles of latency per instruction

Examples in practice

Diagram: the cost of a cycle compared with the cost of a memory access

Three numbers worth carrying out of this lesson:

That last line is the bridge to tomorrow: a chip’s speed is rarely the reason a program is slow.


Implications: security, privacy, performance, scalability, and cost

Diagram: sequential instruction execution compared with a pipelined processor

Diagram: how speculative execution leaks data across a security boundary


Alternatives: free, open source, and commercial

As with Day 1, “alternatives” here means other excellent ways to learn and explore this material.

ResourceTypeWhat it offersCost
The Day 2 lab in this courseFreeA toy CPU you run, trace by hand, and program yourselfFree
Crash Course Computer Science (PBS Digital Studios)Free video seriesShort visual episodes on the ALU, registers, the instruction cycle, and clock speedFree
Wikipedia: “Instruction cycle” and “Central processing unit”Free referenceWell-cited walkthroughs of the cycle and of CPU organization, with historyFree
From Nand to Tetris (Schocken & Nisan)Open course materialsYou build a working CPU from gates, then write programs for itFree materials; optional companion book
”Code” by Charles Petzold (2nd ed.)Book (commercial)Chapters that assemble a CPU conceptually, wire by wire, at beginner paceBook purchase
A CPU visualizer or simulator app of your choiceFree/open source toolsMany educational simulators animate registers and the cycle graphicallyTypically free

If today’s lesson left you wanting to see the machinery drawn and animated, the Crash Course episodes on the CPU are the gentlest next step; if it left you wanting to build the machinery, Nand to Tetris is the definitive project and pairs perfectly with this section of the course.

Diagram: CISC compared with RISC instruction set design

Concept AConcept BKey difference
Control unitALUThe control unit decides and steers each step; the ALU performs the arithmetic and logic the step requires
Program counterInstruction registerThe PC holds the address of the next instruction; the IR holds the instruction itself while it is decoded
Clock speed (GHz)IPCCycles per second versus instructions completed per cycle — performance is roughly their product
Machine codeAssembly languageMachine code is the raw bit patterns; assembly is a human-readable notation for the same instructions (our ADD R1,R2->R3 is assembly-style)
x86ARMTwo incompatible instruction-set contracts: complex variable-length instructions versus simple fixed-size ones, with different power and licensing histories
PipeliningMulticorePipelining overlaps stages of one instruction stream; multicore runs separate streams on duplicated hardware
CPU coreGPUA core maximizes speed on a single sequential stream; a GPU maximizes throughput on thousands of identical parallel operations

When to use it — and when not to

Diagram: parallelisable work compared with sequential work

Reason at this level when a job is CPU-bound and you need to know why, you are choosing between architectures, or you are explaining why more cores did not help.

Ignore it when writing ordinary code. The abstraction is sound, and thinking about pipeline stalls while writing a data loader is how you write it slowly and badly.

The tell that you need it: a measurement that says the processor is busy but the work is not progressing — or that it is idle while the job is still slow.


The AI connection

Diagram: a GPU-bound job compared with a CPU-bound one


Knowledge check

Try these from memory before looking back:

  1. Name the five main parts inside a CPU core covered today and give each one’s one-sentence job description.
  2. The program counter advances immediately after fetch, before execution. Explain why that ordering is what makes jumps and loops possible.
  3. A 4.0 GHz processor benchmarks slower than a 3.2 GHz processor on the same task. Give two distinct explanations using today’s vocabulary.
  4. In the four-stage pipeline diagram, why does the machine complete one instruction per cycle from cycle 4 onward, even though every instruction takes four cycles? What everyday system works the same way?
  5. Your training job shows 25% GPU utilization and one CPU core pinned at 100%. Diagnose the situation in two sentences, using the term CPU-bound correctly.

Hands-on exercise

Today you run a CPU you can see inside. The Day 2 lab directory contains toy_cpu.sh, a simulator for a four-instruction machine — LOAD Rn,value, ADD Ra,Rb->Rc, PRINT Rn, HALT, over registers R1–R4 — that prints every fetch, decode, and execute step, plus the register file after each instruction. From the lab directory, run the demonstration program:

bash examples/toy_cpu.sh examples/programs/add-two-numbers.txt

Read the trace against the lesson: find the PC advancing, find the decode line naming opcode and operands, find the write-back landing in R3. Then open starter/trace-worksheet.md and — before running anything else — hand-trace the three worksheet programs, predicting every register value, every OUTPUT: line, and the final instruction count. Only then check yourself:

bash examples/toy_cpu.sh examples/programs/trace-01.txt
bash examples/toy_cpu.sh examples/programs/trace-02.txt
bash examples/toy_cpu.sh examples/programs/trace-03.txt

Finally, run and then extend your own program:

bash examples/toy_cpu.sh starter/my_program.txt
bash tests/run_tests.sh

Expected output

The demonstration program produces exactly this (a real captured run):

=== Toy CPU ===
Program: examples/programs/add-two-numbers.txt (5 instructions in memory)
Registers start at: R1=0 R2=0 R3=0 R4=0

PC=0  FETCH    LOAD R1,5
      DECODE   opcode=LOAD  dest=R1  value=5
      EXECUTE  R1 <- 5
      REGS     R1=5 R2=0 R3=0 R4=0

PC=1  FETCH    LOAD R2,3
      DECODE   opcode=LOAD  dest=R2  value=3
      EXECUTE  R2 <- 3
      REGS     R1=5 R2=3 R3=0 R4=0

PC=2  FETCH    ADD R1,R2->R3
      DECODE   opcode=ADD  src1=R1  src2=R2  dest=R3
      EXECUTE  R3 <- R1 + R2 = 5 + 3 = 8
      REGS     R1=5 R2=3 R3=8 R4=0

PC=3  FETCH    PRINT R3
      DECODE   opcode=PRINT  reg=R3
      EXECUTE  send R3 to the output
OUTPUT: 8
      REGS     R1=5 R2=3 R3=8 R4=0

PC=4  FETCH    HALT
      DECODE   opcode=HALT
      EXECUTE  stop the clock

HALT reached after 5 instructions.
Final registers: R1=5 R2=3 R3=8 R4=0

Line by line: each PC= line is a fetch, and the PC values count 0 through 4 in order because nothing ever overwrites the counter; each DECODE line shows the control unit’s reading of the instruction; each EXECUTE line is the ALU or data movement doing the work; OUTPUT: 8 is the one visible result; and the final line is the register file at halt.

Validate your work

You are done when you can check every box:

Troubleshooting

Common mistakes

Practice assignment

  1. Run the toy CPU on a program you write. Use only LOAD, ADD, PRINT and HALT, and make it produce a number you predicted in advance.

  2. Count the cycles. For a five-instruction program, how many cycles without pipelining, and how many with?

  3. Break it deliberately. Remove HALT and record exactly what the simulator says. Explain the message in terms of the program counter.


Extension challenge

The toy machine has no multiply instruction. Build multiplication out of what it has.

  1. Write it. Multiply two small numbers using only LOAD, ADD, PRINT and HALT, plus repetition you unroll by hand.

  2. Count the cost. How many instructions for 5 x 3? For 5 x 30?

  3. Draw the conclusion. Write two sentences on why real CPUs have a multiply instruction in hardware, in terms of the cycle count you just measured.


Quiz

Q1. During the fetch phase of the instruction cycle, what does the program counter (PC) hold?

  1. The instruction currently being executed
  2. The memory address of the next instruction to fetch
  3. The result of the last ALU operation
  4. The number of instructions executed so far
Show answer

Answer: B. The memory address of the next instruction to fetch

The PC is a special register holding the address of the next instruction. The control unit places that address on the bus to fetch the instruction, then advances the PC — it is an address pointer, not a tally.

Q2. Which component examines a fetched instruction's bit pattern and steers the other parts of the CPU accordingly?

  1. The instruction register
  2. The ALU
  3. The control unit
  4. The bus
Show answer

Answer: C. The control unit

Decoding is the control unit's job: it reads the instruction held in the instruction register and routes operands, selects the operation, and aims the result — like railway points being switched.

Q3. A CPU runs at 3 GHz. What exactly happens three billion times per second?

  1. Three billion instructions complete
  2. The program counter resets to zero
  3. Three billion values move from RAM into registers
  4. The chip's shared timing signal cycles
Show answer

Answer: D. The chip's shared timing signal cycles

Gigahertz counts clock cycles, not instructions. An instruction may take many cycles, and a modern core can also complete several instructions in one cycle — which is why performance is roughly clock speed times instructions per cycle (IPC).

Q4. Why does the program counter advancing at fetch time (before execution) matter so much?

  1. Jumps and loops work by overwriting the already-advanced PC
  2. It makes the fetch phase run at a higher clock speed
  3. It keeps the instruction register from being overwritten
  4. It lets the CPU skip instructions that contain errors
Show answer

Answer: A. Jumps and loops work by overwriting the already-advanced PC

Because the PC is just a register holding the next address, an instruction that writes a different address into it redirects the machine — jumping backward creates loops, and conditional jumps create decisions.

Q5. In a four-stage pipeline (fetch, decode, execute, write-back) that is full, how often does an instruction finish?

  1. Every four cycles, because each instruction takes four stages
  2. Every cycle, though each instruction still takes four
  3. Twice per cycle, because stages overlap
  4. It depends on the clock speed
Show answer

Answer: B. Every cycle, though each instruction still takes four

Pipelining overlaps the stages of consecutive instructions: while one instruction executes, the next decodes and a third is fetched. Once all stages are busy, one instruction completes per cycle — throughput rises fourfold with no stage running faster.

Q6. Which statement correctly contrasts the x86 and ARM instruction sets?

  1. x86 was designed for phones and ARM for desktop computers
  2. ARM programs run natively on x86 chips because both use the same machine code
  3. x86 uses many variable-length instructions; ARM uses fewer fixed-size ones
  4. ARM chips are made only by Intel and AMD, while x86 is licensed to many companies
Show answer

Answer: C. x86 uses many variable-length instructions; ARM uses fewer fixed-size ones

x86 is the CISC-lineage contract descending from the 8086, with instructions from 1 to 15 bytes; ARM began in 1985 as the Acorn RISC Machine with simple uniform instructions, is licensed broadly, and dominates phones — the licensing and origin claims in the other options are reversed or false.

Q7. Why did Apple's 2020 move of Macs from Intel x86 to its own ARM-based M1 chips matter beyond Apple?

  1. It proved x86 processors could no longer run desktop software
  2. It was the first time any computer had used a RISC processor
  3. It doubled the clock speed of laptop processors industry-wide
  4. It showed a phone architecture could outperform high-end x86 laptops
Show answer

Answer: D. It showed a phone architecture could outperform high-end x86 laptops

Apple Silicon demonstrated ARM's performance-per-watt advantage at the high end, not just in phones — and cloud providers drew the same conclusion, expanding ARM server offerings where energy is a dominant cost.

Q8. A training job shows the GPU 30% utilized while one CPU core is pinned at 100%. What is the most likely situation?

  1. The job is CPU-bound: data loading cannot keep the GPU fed
  2. The GPU is defective and should be replaced
  3. The clock speed of the GPU is set too low
  4. The job is GPU-bound, which is why the GPU shows less than full utilization
Show answer

Answer: A. The job is CPU-bound: data loading cannot keep the GPU fed

CPU-bound means the CPU's instruction stream is the bottleneck. The branchy, sequential work of feeding data cannot match the GPU's appetite, so the expensive parallel hardware idles — fixed by more loader workers, compiled preprocessing, or cached data, not by touching the GPU.

Glossary

ALU
The arithmetic logic unit — the block of gate circuitry inside a CPU core that performs arithmetic and logic operations on register values and sets the status flags.
control unit
The part of the CPU that runs the instruction cycle: it fetches each instruction, decodes its bit pattern, and steers the ALU, registers, and bus to carry it out.
register
One of a small, fixed set of ultra-fast storage slots inside the CPU that hold the values being worked on at this instant — the fastest memory in the machine.
program counter
The special register holding the memory address of the next instruction; it advances at fetch time, and instructions that overwrite it create jumps and loops.
instruction register
The register that holds the instruction currently being decoded, parked where the control unit's decoding circuitry can examine its bits.
flags
Single bits set by the ALU recording facts about the last result — such as "it was zero" or "it was negative" — that later instructions can test to make decisions.
instruction set
The published catalogue of operations, visible registers, and bit encodings a processor family promises to execute — the contract between software and silicon.
machine code
The raw binary instructions, encoded per an instruction set, that a CPU executes directly.
assembly language
A human-readable notation for machine instructions, one line per instruction, like the toy machine's ADD R1,R2->R3.
clock cycle
One tick of the CPU's shared timing signal — the chip's indivisible unit of time, within which each pipeline stage must finish its step.
gigahertz
A billion clock cycles per second; a 3 GHz CPU's clock ticks three billion times each second, making one cycle about a third of a nanosecond.
IPC
Instructions per cycle — how many instructions a core completes per clock tick on a given workload; performance is roughly clock speed times IPC.
pipeline
The assembly-line organization of the instruction cycle in which the fetch, decode, execute, and write-back stages of consecutive instructions overlap, so one instruction can finish every cycle.
branch prediction
The hardware's educated bet on which way a conditional jump will go, made so the pipeline can keep fetching; a wrong bet forces the speculative work to be discarded.
x86
The instruction-set family descending from Intel's 1978 8086 — complex, variable-length instructions — that still dominates desktop and much server computing.
ARM
The RISC instruction-set family begun as the Acorn RISC Machine in 1985 — simple, uniform, power-frugal instructions — that powers essentially all smartphones and, since Apple Silicon, a growing share of laptops and servers.
CPU-bound
Describes a workload whose speed is limited by the CPU's instruction stream rather than by memory, disk, network, or an accelerator — as when a starving GPU idles because sequential preprocessing cannot keep up.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.