Computing Foundations › Inside the Machine › Day 1

Day 1: How a Computer Works: From Transistors to Programs

Day 1 of 365 — How a Computer Works: From Transistors to Programs

After this lesson you will be able to explain, layer by layer, how billions of on/off switches become running programs — and use that mental model to reason about AI hardware, memory limits, and cost.

Course
Computing Foundations
Category
Inside the Machine
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-001-how-a-computer-works-from-transistors

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/computing-foundations/day-001-how-a-computer-works-from-transistors
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

Diagram: the three hardware resources every program competes for

Everything you build in this course runs on the machine described here, and the parts you meet today decide what is possible later.

The lesson has one destination: understanding why AI runs on GPUs, and why that is a fact about hardware rather than a fashion in software.


The idea in plain language

A computer is a machine that does one tiny thing, extremely fast, and is built in layers so that tiny thing adds up to everything else.

Diagram: the ladder of abstraction from transistor to running program


Historical background

The ideas arrived long before the hardware could carry them.

WhenWhat changedWhy it mattered
1936Turing’s universal machineOne machine, reprogrammed, computes anything computable
1945Von Neumann’s stored-program designInstructions live in the same memory as data
Dec 1947The transistorThe switch becomes small, cheap and reliable
1958The integrated circuitMany switches on one piece of silicon
1965Moore’s observationTransistor counts double roughly every two years
2007–GPUs turned to general computingThousands of simple cores — the shape of a network

What it is — and what it is not

Diagram: the layers of computing, from transistors to applications

It is a stack of layers, each built only from the one below.

It is not magic at any layer, and three misconceptions are worth naming:


Why it was created and what problems it solves

Diagram: rewiring a machine compared with loading a stored program

Every layer exists because the one below it was too hard to use directly.

The pattern repeats all the way up: each layer hides the one beneath it well enough that you can forget it — until performance makes you remember.


How it works

From switch to arithmetic

Diagram: how switches become gates, gates become adders, and loops become memory

Arithmetic and memory are not abilities the machine has. They are shapes that gates are arranged into.

The parts, and what each is for

ComponentWhat it doesEveryday analogue
CPUExecutes instructions one after another, very fastThe chef cooking
RAMFast temporary workspace; empties at power-offThe counter and pantry
StorageSlow but permanent home for files and the OSThe warehouse across town
GPUThousands of simple cores doing the same arithmetic at onceA hall of cooks chopping
BusThe wiring connecting themThe corridors

One instruction, four stages

Flowchart: the fetch-decode-execute cycle

Registers R1 and R2 hold 5 and 3. The instruction is ADD R1, R2 -> R3.

About three billion times a second. That is all a CPU ever does.

Because the program counter is just a register, an instruction can change it. Jumping backward is a loop; jumping conditionally is a decision. Loops plus decisions plus arithmetic is everything software is made of.

Why memory is a pyramid

Diagram: the memory hierarchy trade-off between speed, size, and cost

Fast memory needs more transistors per bit and must sit close to the ALU, so machines layer a little of it over larger, slower, cheaper tiers.

It works because programs reuse recent data and march through it in order — locality — so hardware keeps hot data in the fast tiers by itself.


An everyday analogy

A restaurant kitchen, where every part maps onto something real.

KitchenMachine
The chef’s handsThe ALU — where work actually happens
The cutting boardRegisters — a few items, instantly reachable
The counter and pantryRAM — today’s ingredients
The warehouse across townStorage — everything, but a trip away
The recipe cardThe program, held in memory as data
A hall of prep cooksThe GPU — many hands, one repeated task

Examples in practice

Diagram: the stages of launching an application from storage to running

Where this becomes AI:

Day 3 measures this properly. Today it is enough to see that it is the pyramid.


Implications: security, privacy, performance, scalability, and cost


Alternatives: free, open source, and commercial

ResourceTypeWhat it gives you
The Day 1 labFreeYour own machine’s real numbers
Crash Course Computer ScienceFree video seriesShort visual episodes from transistors upward
Nand2TetrisFree courseBuild a working computer from NAND gates
Ben Eater’s breadboard CPUFree video seriesA processor built by hand, one wire at a time
Petzold, CodeBookThe clearest prose account of the whole ladder
Patterson & HennessyTextbookThe standard university treatment

Diagram: RAM compared with storage

Often confusedThe actual difference
CPU vs GPUFew clever cores vs thousands of simple ones doing the same thing
Core vs threadA physical unit vs a stream of instructions it can interleave
Memory vs storageVolatile and fast vs permanent and slow
Machine code vs sourceWhat the CPU executes vs what a person writes

When to use it — and when not to

Diagram: when to reason about hardware and when to ignore it

Reach for this model when something is slower than it should be, a program will not fit in memory, you are choosing hardware, or you need to explain why a model runs on one machine and not another.

Do not reach for it when you are writing ordinary application code. The layers exist so you can ignore them, and reasoning about cache lines while writing a data-loading script is a way to write it badly and slowly.

The rule: work at the highest layer that solves your problem, and go down one level only when a measurement tells you to.


The AI connection

Diagram: how a CPU and a GPU spend the same transistor budget differently


Knowledge check

Try these from memory before looking back:

  1. Name the layers of the computing stack in order from physics to applications, with one sentence on what each provides to the layer above.
  2. A friend says their computer “understands” them because it autocorrects typos. Explain what is really happening and why “understanding” is the wrong word for the hardware.
  3. Without looking at the diagram, describe the four steps of executing ADD R1, R2 -> R3, including the program counter’s role.
  4. Your laptop crawls whenever you open a very large dataset, yet the CPU meter shows low usage. Using the memory hierarchy, give the most likely explanation.
  5. In three sentences or fewer, explain to a non-technical person why AI companies buy GPUs instead of simply faster CPUs.

Hands-on exercise

Ask your own machine what hardware it has, then place each answer in the layers diagram. Full instructions are in the Day 1 lab directory.

What you wantmacOSLinux
CPU modelsysctl -n machdep.cpu.brand_stringlscpu
RAM in bytessysctl hw.memsizefree -h
Logical coressysctl -n hw.ncpunproc
Disk size and freedf -h /df -h /
OS versionsw_verscat /etc/os-release

On Windows, use Get-ComputerInfo in PowerShell — or install WSL, since later lessons assume a Unix-style terminal.

Expected output

$ sysctl -n machdep.cpu.brand_string
Apple M2
$ sysctl hw.memsize
hw.memsize: 17179869184
$ sysctl -n hw.ncpu
8
$ df -h /
/dev/disk3s1s1  460Gi  9.8Gi  180Gi  6%  /

Validate your work

You are done when you can state, from your own machine:

Troubleshooting

Common mistakes


Practice assignment

Find the real numbers for the machine in front of you.

  1. Record the specification. Processor model, core count, RAM, storage type and size, and whether there is a discrete GPU.

  2. Place them on the pyramid. Attach an approximate latency to each tier from the lesson, and note which ones your machine actually has.

  3. Answer one question in two sentences. What is the largest model, at 2 bytes per weight, that could theoretically sit in your RAM — and what would you expect to happen if you tried to run one twice that size?


Extension challenge

Go one level deeper into the hierarchy you have just placed your machine on.

  1. Find your cache sizes. sysctl -a | grep cachesize on macOS, lscpu on Linux. Record L1, L2 and L3.

  2. Compare the tiers. Write the ratio of L1 to L3, and of L3 to your RAM.

  3. Predict, then check. Which is the bigger jump in size, and which in latency? They are not the same tier, and the reason is the whole point of the pyramid.


Quiz

Q1. What is the fundamental role of a transistor in a computer?

  1. It stores a complete file permanently
  2. It cools the processor while it runs
  3. It converts binary numbers into decimal numbers
  4. It acts as a switch that is either on or off
Show answer

Answer: D. It acts as a switch that is either on or off

A transistor is a microscopic switch whose on/off state represents one bit of possibility; billions of them wired into logic gates form all of a computer's processing and memory circuitry.

Q2. Which sequence correctly orders the steps a CPU repeats for every instruction?

  1. Fetch, decode, execute, write back
  2. Execute, decode, fetch, write back
  3. Decode, fetch, execute, write back
  4. Fetch, execute, decode, write back
Show answer

Answer: A. Fetch, decode, execute, write back

The CPU first fetches the instruction from memory, then the control unit decodes its bit pattern, the ALU executes the operation, and the result is written back to a register or memory before the cycle repeats.

Q3. What is the key difference between RAM and storage (SSD/disk)?

  1. RAM holds programs while storage holds only pictures and documents
  2. RAM is fast but empties at power-off; storage is slower but permanent
  3. RAM is measured in terabytes while storage is measured in gigabytes
  4. Storage is faster than RAM but more expensive per byte
Show answer

Answer: B. RAM is fast but empties at power-off; storage is slower but permanent

RAM is the machine's fast, volatile workspace for running programs, while storage is the slower, persistent layer that survives shutdown — roughly a thousand times slower than RAM for an SSD.

Q4. The capital letter A is stored in a computer using ASCII as which of the following?

  1. The binary pattern 00000001, which is the number 1
  2. A miniature photograph of the letter shape
  3. The binary pattern 01000001, which is the number 65
  4. A special letter-transistor distinct from number-transistors
Show answer

Answer: C. The binary pattern 01000001, which is the number 65

ASCII assigns the number 65 to capital A, and 65 in binary is 01000001; text, like everything else in a computer, is bit patterns given meaning by convention.

Q5. What was the central insight of the von Neumann architecture described in the 1945 EDVAC report?

  1. Computers should use decimal arithmetic instead of binary
  2. Every computer needs a separate machine for each task it performs
  3. Processors should be built from vacuum tubes rather than relays
  4. Programs live in the same memory as data, making one machine general-purpose
Show answer

Answer: D. Programs live in the same memory as data, making one machine general-purpose

Storing instructions in memory alongside data means a computer can switch tasks just by loading different instructions, instead of being physically rewired like ENIAC.

Q6. Why do computers use a memory hierarchy instead of building all memory as fast as registers?

  1. Fast memory costs more per byte and must sit close to the CPU
  2. Slow memory produces more accurate results than fast memory
  3. Operating systems cannot address more than a few kilobytes of fast memory
  4. Registers wear out quickly, so they are kept scarce as spare parts
Show answer

Answer: A. Fast memory costs more per byte and must sit close to the CPU

Speed, size, and cost trade off against each other: register-speed memory is expensive and must sit near the ALU, so machines pair tiny fast layers with progressively larger, slower, cheaper ones and rely on locality to make it work.

Q7. Why are GPUs better suited than CPUs for running neural networks?

  1. GPUs have higher clock speeds than any CPU
  2. Thousands of simple cores do the same math on many values at once
  3. GPUs store model weights permanently even when powered off
  4. GPUs understand the meaning of the data they process
Show answer

Answer: B. Thousands of simple cores do the same math on many values at once

Neural networks consist mostly of huge uniform grids of multiplications and additions, which map naturally onto a GPU's thousands of parallel arithmetic cores rather than a CPU's few powerful sequential cores.

Q8. What does it mean to say a computer is a "stored-program" machine?

  1. Its programs are permanently burned into the processor at the factory
  2. It keeps a backup of every program it has ever run
  3. Its instructions live in memory as data, so loading new ones changes its job
  4. Programs can only be loaded from external punched cards
Show answer

Answer: C. Its instructions live in memory as data, so loading new ones changes its job

Because the program lives in memory just like data, the same hardware can become a calculator, a text editor, or a neural network simply by loading different instructions — the foundation of general-purpose computing.

Glossary

transistor
A microscopic electrical switch with no moving parts whose on/off state is the physical basis of all digital computing.
logic gate
A small circuit built from transistors that computes a simple rule, such as AND (output on only if both inputs are on), OR, or NOT.
binary
A way of writing numbers and encoding all data using only two digits, 0 and 1, which map directly onto a switch's off and on states.
bit
The smallest unit of information, a single 0 or 1.
byte
A group of 8 bits, enough to store one of 256 possible values, such as a single ASCII character.
CPU
The central processing unit, the chip that executes program instructions one after another by repeating the fetch-decode-execute cycle.
register
One of a small set of ultra-fast storage slots inside the CPU that hold the values being worked on at this instant.
RAM
Random-access memory, the computer's fast temporary workspace that holds running programs and their data and is emptied when power is cut.
cache
Small, very fast memory on the CPU chip that keeps recently used data close to the processor so it rarely has to wait on slower RAM.
machine code
The raw binary instructions a CPU executes directly, the lowest-level form of any program.
operating system
The master program that manages the hardware, shares the CPU and memory among running programs, and handles all input and output.
memory hierarchy
The layered arrangement of storage from registers through cache and RAM to disk, trading speed against size and cost per byte at each level.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.