Computing Foundations › Inside the Machine › Day 2
Day 2: The CPU: Fetch, Decode, Execute
After this lesson you will be able to open up the CPU in your mind — control unit, ALU, registers, program counter — trace any simple program through the fetch-decode-execute cycle by hand, and use clock speed, instruction sets, and pipelining to reason about real hardware choices, including why GPUs beat CPUs at bulk math.
Hands-on lab for this lesson
Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-002-the-cpu-fetch-decode-execute
- Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git cd ai-roadmap-365.github.io - Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
cd labs/sections/computing-foundations/day-002-the-cpu-fetch-decode-execute - Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
- Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
bash tests/run_tests.sh # or the test command named in the lab README
You can also open the lab as a local page (works offline, shows the file tree and expected output).
Learning objectives
By the end of this lesson you will be able to:
- Name the main parts of a CPU core — control unit, ALU, registers, program counter, instruction register, flags — and state each one's role in the instruction cycle
- Trace the fetch-decode-execute-write-back cycle by hand for a short assembly-style program, predicting every register value and output before running it
- Explain why the program counter advances at fetch time and why that mechanism is what makes jumps and loops possible
- Distinguish clock speed (GHz) from instructions per cycle, and explain why gigahertz alone stopped predicting performance after the mid-2000s
- Compare the x86 and ARM instruction sets — design philosophy, strongholds, and licensing — and explain why Apple Silicon's 2020 transition mattered for the industry
- Describe how pipelining overlaps instruction stages to raise throughput, and what branch prediction and speculative execution add and risk
- Diagnose a "CPU-bound" AI workload: explain why a GPU starves when sequential CPU work cannot keep pace, and name the standard remedies
Prerequisites
- Day 1 (How a Computer Works): transistors, logic gates, the von Neumann architecture, and the first look at the instruction cycle
- A terminal with bash for the lab (macOS, Linux, or WSL on Windows)
Why this matters
Every calculation your machine has ever performed passed through one small loop, repeated billions of times a second.
-
Fetch, decode, execute, write back. That is the whole repertoire.
-
Clock speed stopped rising around 2005, so chips got wider instead of faster — which is the reason GPUs exist in the form they do.
-
When your GPU sits at 40% utilisation, this lesson tells you why: something sequential could not feed it fast enough.
The idea in plain language
A processor is a machine that reads one instruction, does it, and moves on — faster than you can imagine, and with no idea what any of it means.
-
It has no plan. There is no strategy, no lookahead in the human sense. Just the next address, over and over.
-
The clock is a drumbeat. A shared signal that says “now” a few billion times a second. Nothing happens between beats.
-
The instruction is bits.
ADD R1, R2 -> R3is a pattern like0001 0001 0010 0011; the human-readable form is for you, not for the chip.
Historical background
| When | What arrived | Why it mattered |
|---|---|---|
| 1945 | Von Neumann’s stored-program design | The cycle as we still run it |
| 1971 | Intel 4004, the first commercial microprocessor | A whole CPU on one chip |
| 1978 | Intel 8086 | The ancestor of every x86 chip in use |
| Early 1980s | The RISC argument | Fewer, simpler instructions, decoded cheaply |
| 1985 | ARM’s first design | The philosophy that now runs every phone |
| ~2005 | Clock speeds stall | Power and heat end the megahertz race |
| 2020 | Apple M1 | A RISC laptop chip outperforming high-end x86 |
-
The 1945 loop has not changed. Everything since is about doing more of it at once, not doing it differently.
-
The RISC-CISC argument was settled by physics, not by taste: simpler decoding costs less power, and power became the binding constraint.
What it is — and what it is not
It is a loop with four stages, driven by a clock, over instructions held in memory as data.
It is not any of these, and each misconception costs you later:
-
Not thinking. No stage involves understanding. Decode means routing bits to circuitry, nothing more.
-
Not one instruction at a time, really. Modern chips overlap several and may finish them out of order — but the logical story stays this loop.
-
Not the same as a core count. “8 CPUs” often means 4 physical cores running two instruction streams each.
-
Not where your program is slow. Almost always it is waiting for memory, which is Day 3.
Why it was created and what problems it solves
The cycle exists to solve one problem: making a machine general.
-
Before it, machines were rewired for each job. Changing the task meant changing the cabling, over hours or days.
-
The stored-program idea put instructions in memory, so changing the task means loading a different file.
-
That only works if something reads them in order, decides what each means, and moves on. That something is this cycle.
How it works
The parts inside a core
| Part | Role in the cycle |
|---|---|
| Program counter | Holds the address of the next instruction, then advances |
| Instruction register | Holds the instruction while it is decoded |
| Control unit | Reads the pattern and steers every other part |
| Registers | The fastest storage; hold the values in play right now |
| ALU | Does the arithmetic, and sets flags about the result |
| Flags | Single bits recording facts like “the result was zero” |
| Bus | The wiring to memory — far slower to answer than a register |
The cycle, in slow motion
A toy CPU with four registers and four instructions — LOAD, ADD, PRINT,
HALT — makes each phase visible. Real simulator output, adding 5 and 3:
PC=0 FETCH LOAD R1,5
DECODE opcode=LOAD dest=R1 value=5
EXECUTE R1 <- 5
REGS R1=5 R2=0 R3=0 R4=0
PC=2 FETCH ADD R1,R2->R3
DECODE opcode=ADD src1=R1 src2=R2 dest=R3
EXECUTE R3 <- 5 + 3 = 8
- Fetch — the PC names an address; memory answers with the instruction.
- Decode — the opcode selects circuitry; the operand fields route registers.
- Execute — the ALU combines the values and sets flags.
- Write back — the result lands in a register or goes out to memory.
Overlapping the stages
-
The stages use different circuitry, so they need not sit idle waiting.
-
While one instruction executes, the next can decode and a third can fetch.
-
A result is finished every cycle, even though each instruction still takes four cycles end to end.
-
A jump breaks it. The chip guessed which way the branch would go and, when wrong, throws away the partly-done work.
An everyday analogy
An assembly line, and the reason it is not a single craftsman.
| The line | The processor |
|---|---|
| A station that fetches the next part | Fetch |
| A station that reads the work order | Decode |
| A station that does the work | Execute |
| A station that puts the finished item away | Write back |
| One item leaving the line per beat | One instruction completing per cycle |
| Each item still visiting four stations | Four cycles of latency per instruction |
-
Throughput and latency are different things. The line finishes one item per beat while every item still takes four beats to traverse.
-
A wrong work order empties the line. That is a mispredicted branch.
Examples in practice
Three numbers worth carrying out of this lesson:
-
A 3 GHz chip completes about three billion cycles a second. The clock is a timing signal, not a count of work done.
-
A cycle is about 0.3 nanoseconds. Light travels roughly 10 cm in that time, which is why physical distance on the chip matters.
-
A memory access costs about 100 nanoseconds — some 300 cycles. The processor is often waiting, not computing.
That last line is the bridge to tomorrow: a chip’s speed is rarely the reason a program is slow.
Implications: security, privacy, performance, scalability, and cost
-
Performance. Clock speed alone tells you almost nothing. Instructions per cycle, cache behaviour and memory latency matter more.
-
Scalability. Sequential work does not get faster with more cores. Only work that can be split does.
-
Security. Speculative execution — guessing past a branch — leaked data across boundaries in Spectre and Meltdown. Performance and isolation are not independent.
-
Cost. Cloud pricing is per core-hour, so a job that cannot use its cores is paying for idle silicon.
-
Energy. Simpler instructions cost less power to decode, which is why phones went RISC first and laptops followed.
Alternatives: free, open source, and commercial
As with Day 1, “alternatives” here means other excellent ways to learn and explore this material.
| Resource | Type | What it offers | Cost |
|---|---|---|---|
| The Day 2 lab in this course | Free | A toy CPU you run, trace by hand, and program yourself | Free |
| Crash Course Computer Science (PBS Digital Studios) | Free video series | Short visual episodes on the ALU, registers, the instruction cycle, and clock speed | Free |
| Wikipedia: “Instruction cycle” and “Central processing unit” | Free reference | Well-cited walkthroughs of the cycle and of CPU organization, with history | Free |
| From Nand to Tetris (Schocken & Nisan) | Open course materials | You build a working CPU from gates, then write programs for it | Free materials; optional companion book |
| ”Code” by Charles Petzold (2nd ed.) | Book (commercial) | Chapters that assemble a CPU conceptually, wire by wire, at beginner pace | Book purchase |
| A CPU visualizer or simulator app of your choice | Free/open source tools | Many educational simulators animate registers and the cycle graphically | Typically free |
If today’s lesson left you wanting to see the machinery drawn and animated, the Crash Course episodes on the CPU are the gentlest next step; if it left you wanting to build the machinery, Nand to Tetris is the definitive project and pairs perfectly with this section of the course.
Comparison with related concepts
| Concept A | Concept B | Key difference |
|---|---|---|
| Control unit | ALU | The control unit decides and steers each step; the ALU performs the arithmetic and logic the step requires |
| Program counter | Instruction register | The PC holds the address of the next instruction; the IR holds the instruction itself while it is decoded |
| Clock speed (GHz) | IPC | Cycles per second versus instructions completed per cycle — performance is roughly their product |
| Machine code | Assembly language | Machine code is the raw bit patterns; assembly is a human-readable notation for the same instructions (our ADD R1,R2->R3 is assembly-style) |
| x86 | ARM | Two incompatible instruction-set contracts: complex variable-length instructions versus simple fixed-size ones, with different power and licensing histories |
| Pipelining | Multicore | Pipelining overlaps stages of one instruction stream; multicore runs separate streams on duplicated hardware |
| CPU core | GPU | A core maximizes speed on a single sequential stream; a GPU maximizes throughput on thousands of identical parallel operations |
When to use it — and when not to
Reason at this level when a job is CPU-bound and you need to know why, you are choosing between architectures, or you are explaining why more cores did not help.
Ignore it when writing ordinary code. The abstraction is sound, and thinking about pipeline stalls while writing a data loader is how you write it slowly and badly.
The tell that you need it: a measurement that says the processor is busy but the work is not progressing — or that it is idle while the job is still slow.
The AI connection
-
The GIL makes data loaders spawn processes, not threads. Only one thread executes Python at a time, so parallel preprocessing needs separate processes.
-
A sawtooth utilisation graph is the signature of a starved accelerator.
-
Pipelining is the fix, one level up: overlap loading the next batch with computing on the current one, and the waiting disappears.
Knowledge check
Try these from memory before looking back:
- Name the five main parts inside a CPU core covered today and give each one’s one-sentence job description.
- The program counter advances immediately after fetch, before execution. Explain why that ordering is what makes jumps and loops possible.
- A 4.0 GHz processor benchmarks slower than a 3.2 GHz processor on the same task. Give two distinct explanations using today’s vocabulary.
- In the four-stage pipeline diagram, why does the machine complete one instruction per cycle from cycle 4 onward, even though every instruction takes four cycles? What everyday system works the same way?
- Your training job shows 25% GPU utilization and one CPU core pinned at 100%. Diagnose the situation in two sentences, using the term CPU-bound correctly.
Hands-on exercise
Today you run a CPU you can see inside. The Day 2 lab directory contains toy_cpu.sh, a simulator for a four-instruction machine — LOAD Rn,value, ADD Ra,Rb->Rc, PRINT Rn, HALT, over registers R1–R4 — that prints every fetch, decode, and execute step, plus the register file after each instruction. From the lab directory, run the demonstration program:
bash examples/toy_cpu.sh examples/programs/add-two-numbers.txt
Read the trace against the lesson: find the PC advancing, find the decode line naming opcode and operands, find the write-back landing in R3. Then open starter/trace-worksheet.md and — before running anything else — hand-trace the three worksheet programs, predicting every register value, every OUTPUT: line, and the final instruction count. Only then check yourself:
bash examples/toy_cpu.sh examples/programs/trace-01.txt
bash examples/toy_cpu.sh examples/programs/trace-02.txt
bash examples/toy_cpu.sh examples/programs/trace-03.txt
Finally, run and then extend your own program:
bash examples/toy_cpu.sh starter/my_program.txt
bash tests/run_tests.sh
Expected output
The demonstration program produces exactly this (a real captured run):
=== Toy CPU ===
Program: examples/programs/add-two-numbers.txt (5 instructions in memory)
Registers start at: R1=0 R2=0 R3=0 R4=0
PC=0 FETCH LOAD R1,5
DECODE opcode=LOAD dest=R1 value=5
EXECUTE R1 <- 5
REGS R1=5 R2=0 R3=0 R4=0
PC=1 FETCH LOAD R2,3
DECODE opcode=LOAD dest=R2 value=3
EXECUTE R2 <- 3
REGS R1=5 R2=3 R3=0 R4=0
PC=2 FETCH ADD R1,R2->R3
DECODE opcode=ADD src1=R1 src2=R2 dest=R3
EXECUTE R3 <- R1 + R2 = 5 + 3 = 8
REGS R1=5 R2=3 R3=8 R4=0
PC=3 FETCH PRINT R3
DECODE opcode=PRINT reg=R3
EXECUTE send R3 to the output
OUTPUT: 8
REGS R1=5 R2=3 R3=8 R4=0
PC=4 FETCH HALT
DECODE opcode=HALT
EXECUTE stop the clock
HALT reached after 5 instructions.
Final registers: R1=5 R2=3 R3=8 R4=0
Line by line: each PC= line is a fetch, and the PC values count 0 through 4 in order because nothing ever overwrites the counter; each DECODE line shows the control unit’s reading of the instruction; each EXECUTE line is the ALU or data movement doing the work; OUTPUT: 8 is the one visible result; and the final line is the register file at halt.
Validate your work
You are done when you can check every box:
- You ran the demonstration program and can point at one fetch, one decode, one execute, and one write-back in its output.
- You completed all three worksheet traces on paper before running the programs.
- Your predicted
OUTPUT:lines, final registers, and instruction counts matched the simulator for all three programs — or you wrote a one-sentence “why” for each miss. - Your extended
starter/my_program.txtuses at least oneADD, prints at least two values, and ends atHALT reached after N instructions.with no errors. -
bash tests/run_tests.shreports27 checks, 0 failure(s).
Troubleshooting
unknown opcode 'load'— opcodes are uppercase:LOAD,ADD,PRINT,HALT. The machine is strict because real decoders are stricter: one wrong bit is a different instruction.unknown register 'R5'or'R0'— the machine has exactly R1 through R4, a fixed register set baked into its design, just like real silicon.ADD needs the form: ADD Ra,Rb->Rc— check for the comma between sources and the two-character arrow->before the destination; an arrow pasted from a word processor (→) will not decode.the program ended without HALT— every program must end withHALT; a real CPU never decides it is finished, it just keeps fetching whatever bits come next. Add the instruction and rerun.program file not found— run commands from the lab directory so relative paths likeexamples/programs/trace-01.txtresolve.
Common mistakes
- Using a register’s new value one instruction too early. In
ADD R2,R2->R2, the ALU reads the old R2 twice, adds, and only then writes back. Worksheet program 2 exists to catch exactly this. - Treating
PRINTas showing the final value.PRINTemits the register’s value at that instant; program 3 prints the same register twice and gets two different numbers. The trace is a timeline, not a summary. - Forgetting that the PC has already advanced. After fetching the instruction at address 2, the PC reads 3 during that instruction’s execution. It feels like a bookkeeping triviality; it is actually the mechanism that will make jumps, loops, and every control structure possible.
- Reading the trace without predicting first. Watching output confirms nothing about your mental model — predictions do. The worksheet’s order (predict, then run) is the whole exercise.
Practice assignment
-
Run the toy CPU on a program you write. Use only
LOAD,ADD,PRINTandHALT, and make it produce a number you predicted in advance. -
Count the cycles. For a five-instruction program, how many cycles without pipelining, and how many with?
-
Break it deliberately. Remove
HALTand record exactly what the simulator says. Explain the message in terms of the program counter.
Extension challenge
The toy machine has no multiply instruction. Build multiplication out of what it has.
-
Write it. Multiply two small numbers using only
LOAD,ADD,PRINTandHALT, plus repetition you unroll by hand. -
Count the cost. How many instructions for 5 x 3? For 5 x 30?
-
Draw the conclusion. Write two sentences on why real CPUs have a multiply instruction in hardware, in terms of the cycle count you just measured.
Quiz
Q1. During the fetch phase of the instruction cycle, what does the program counter (PC) hold?
- The instruction currently being executed
- The memory address of the next instruction to fetch
- The result of the last ALU operation
- The number of instructions executed so far
Show answer
Answer: B. The memory address of the next instruction to fetch
The PC is a special register holding the address of the next instruction. The control unit places that address on the bus to fetch the instruction, then advances the PC — it is an address pointer, not a tally.
Q2. Which component examines a fetched instruction's bit pattern and steers the other parts of the CPU accordingly?
- The instruction register
- The ALU
- The control unit
- The bus
Show answer
Answer: C. The control unit
Decoding is the control unit's job: it reads the instruction held in the instruction register and routes operands, selects the operation, and aims the result — like railway points being switched.
Q3. A CPU runs at 3 GHz. What exactly happens three billion times per second?
- Three billion instructions complete
- The program counter resets to zero
- Three billion values move from RAM into registers
- The chip's shared timing signal cycles
Show answer
Answer: D. The chip's shared timing signal cycles
Gigahertz counts clock cycles, not instructions. An instruction may take many cycles, and a modern core can also complete several instructions in one cycle — which is why performance is roughly clock speed times instructions per cycle (IPC).
Q4. Why does the program counter advancing at fetch time (before execution) matter so much?
- Jumps and loops work by overwriting the already-advanced PC
- It makes the fetch phase run at a higher clock speed
- It keeps the instruction register from being overwritten
- It lets the CPU skip instructions that contain errors
Show answer
Answer: A. Jumps and loops work by overwriting the already-advanced PC
Because the PC is just a register holding the next address, an instruction that writes a different address into it redirects the machine — jumping backward creates loops, and conditional jumps create decisions.
Q5. In a four-stage pipeline (fetch, decode, execute, write-back) that is full, how often does an instruction finish?
- Every four cycles, because each instruction takes four stages
- Every cycle, though each instruction still takes four
- Twice per cycle, because stages overlap
- It depends on the clock speed
Show answer
Answer: B. Every cycle, though each instruction still takes four
Pipelining overlaps the stages of consecutive instructions: while one instruction executes, the next decodes and a third is fetched. Once all stages are busy, one instruction completes per cycle — throughput rises fourfold with no stage running faster.
Q6. Which statement correctly contrasts the x86 and ARM instruction sets?
- x86 was designed for phones and ARM for desktop computers
- ARM programs run natively on x86 chips because both use the same machine code
- x86 uses many variable-length instructions; ARM uses fewer fixed-size ones
- ARM chips are made only by Intel and AMD, while x86 is licensed to many companies
Show answer
Answer: C. x86 uses many variable-length instructions; ARM uses fewer fixed-size ones
x86 is the CISC-lineage contract descending from the 8086, with instructions from 1 to 15 bytes; ARM began in 1985 as the Acorn RISC Machine with simple uniform instructions, is licensed broadly, and dominates phones — the licensing and origin claims in the other options are reversed or false.
Q7. Why did Apple's 2020 move of Macs from Intel x86 to its own ARM-based M1 chips matter beyond Apple?
- It proved x86 processors could no longer run desktop software
- It was the first time any computer had used a RISC processor
- It doubled the clock speed of laptop processors industry-wide
- It showed a phone architecture could outperform high-end x86 laptops
Show answer
Answer: D. It showed a phone architecture could outperform high-end x86 laptops
Apple Silicon demonstrated ARM's performance-per-watt advantage at the high end, not just in phones — and cloud providers drew the same conclusion, expanding ARM server offerings where energy is a dominant cost.
Q8. A training job shows the GPU 30% utilized while one CPU core is pinned at 100%. What is the most likely situation?
- The job is CPU-bound: data loading cannot keep the GPU fed
- The GPU is defective and should be replaced
- The clock speed of the GPU is set too low
- The job is GPU-bound, which is why the GPU shows less than full utilization
Show answer
Answer: A. The job is CPU-bound: data loading cannot keep the GPU fed
CPU-bound means the CPU's instruction stream is the bottleneck. The branchy, sequential work of feeding data cannot match the GPU's appetite, so the expensive parallel hardware idles — fixed by more loader workers, compiled preprocessing, or cached data, not by touching the GPU.
Glossary
- ALU
- The arithmetic logic unit — the block of gate circuitry inside a CPU core that performs arithmetic and logic operations on register values and sets the status flags.
- control unit
- The part of the CPU that runs the instruction cycle: it fetches each instruction, decodes its bit pattern, and steers the ALU, registers, and bus to carry it out.
- register
- One of a small, fixed set of ultra-fast storage slots inside the CPU that hold the values being worked on at this instant — the fastest memory in the machine.
- program counter
- The special register holding the memory address of the next instruction; it advances at fetch time, and instructions that overwrite it create jumps and loops.
- instruction register
- The register that holds the instruction currently being decoded, parked where the control unit's decoding circuitry can examine its bits.
- flags
- Single bits set by the ALU recording facts about the last result — such as "it was zero" or "it was negative" — that later instructions can test to make decisions.
- instruction set
- The published catalogue of operations, visible registers, and bit encodings a processor family promises to execute — the contract between software and silicon.
- machine code
- The raw binary instructions, encoded per an instruction set, that a CPU executes directly.
- assembly language
- A human-readable notation for machine instructions, one line per instruction, like the toy machine's ADD R1,R2->R3.
- clock cycle
- One tick of the CPU's shared timing signal — the chip's indivisible unit of time, within which each pipeline stage must finish its step.
- gigahertz
- A billion clock cycles per second; a 3 GHz CPU's clock ticks three billion times each second, making one cycle about a third of a nanosecond.
- IPC
- Instructions per cycle — how many instructions a core completes per clock tick on a given workload; performance is roughly clock speed times IPC.
- pipeline
- The assembly-line organization of the instruction cycle in which the fetch, decode, execute, and write-back stages of consecutive instructions overlap, so one instruction can finish every cycle.
- branch prediction
- The hardware's educated bet on which way a conditional jump will go, made so the pipeline can keep fetching; a wrong bet forces the speculative work to be discarded.
- x86
- The instruction-set family descending from Intel's 1978 8086 — complex, variable-length instructions — that still dominates desktop and much server computing.
- ARM
- The RISC instruction-set family begun as the Acorn RISC Machine in 1985 — simple, uniform, power-frugal instructions — that powers essentially all smartphones and, since Apple Silicon, a growing share of laptops and servers.
- CPU-bound
- Describes a workload whose speed is limited by the CPU's instruction stream rather than by memory, disk, network, or an accelerator — as when a starving GPU idles because sequential preprocessing cannot keep up.
Sources and further reading
- Crash Course Computer Science — PBS Digital Studios (accessed 2026-07-12) — Free video series; the episodes on the ALU, registers and RAM, the CPU, and instructions cover this lesson visually.
- Instruction cycle — Wikipedia (accessed 2026-07-12) — Reference walkthrough of the fetch-decode-execute cycle and the registers involved.
- Central processing unit — Wikipedia (accessed 2026-07-12) — Well-cited overview of CPU organization, history, clock rates, and parallelism.
- From Nand to Tetris — Shimon Schocken and Noam Nisan (accessed 2026-07-12) — Free course in which you build a working CPU from logic gates and then program it.
- Code: The Hidden Language of Computer Hardware and Software (2nd ed.) — Charles Petzold / Microsoft Press (accessed 2026-07-12) — Assembles a CPU conceptually, wire by wire, at beginner pace.
Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.