Computing Foundations › Inside the Machine › Day 7
Day 7: Processes, Threads, and Scheduling
After this lesson you will be able to explain what a process really is, how the scheduler makes hundreds of programs share a few cores, and spawn, observe, and signal processes yourself from the shell.
Hands-on lab for this lesson
Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-007-processes-threads-and-scheduling
- Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git cd ai-roadmap-365.github.io - Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
cd labs/sections/computing-foundations/day-007-processes-threads-and-scheduling - Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
- Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
bash tests/run_tests.sh # or the test command named in the lab README
You can also open the lab as a local page (works offline, shows the file tree and expected output).
Learning objectives
By the end of this lesson you will be able to:
- Distinguish a program from a process, and name what every process owns: a PID, private memory, file handles, an owner, a parent, and an exit code
- Trace the process lifecycle — spawn, ready, running, waiting, exit — including where zombies come from and what exit codes mean
- Explain why threads are cheaper than processes and why shared memory makes them dangerous, including what a race condition is
- Describe how the scheduler uses time slices, preemption, and context switches to run hundreds of processes on a handful of cores
- Control processes from the shell: start background jobs with &, capture PIDs from $!, inspect with jobs and ps, and send signals with kill
- Distinguish concurrency from parallelism precisely and choose between threads and processes for a given kind of work
- Connect scheduling to AI practice: data-loader worker processes, GPU kernel queues, the GIL preview, and the batch throughput-versus-latency trade-off
Prerequisites
- Days 1–6 of this course, especially Day 6 (operating systems) — this lesson builds directly on the OS mental model
- A terminal on macOS or Linux (or WSL on Windows) for the hands-on exercise
Why this matters
-
Every AI job you run is a process, and every one that dies badly dies in a way this lesson explains.
-
Exit code 137 means the kernel killed it for exceeding a memory limit. You will see this number in container logs more than any other.
-
Data loaders are processes, not threads, for a reason you will understand by the end of today.
-
Closing a terminal kills your background jobs unless you know why, and what to do about it.
-
Yesterday gave you the map. Today walks the busiest part of it.
The idea in plain language
-
A program is a file. A process is that file running — with memory, state and an identity.
-
Your shell is itself a process. When you run a command, it asks the kernel to create a child.
-
A thread is an instruction stream inside a process, sharing its memory with every other thread there.
-
The scheduler decides who runs. Your machine has hundreds of processes and a handful of cores, and the gap between those numbers is today’s subject.
-
Most processes are asleep. That is the whole trick.
Historical background
-
Early machines ran one program at a time, from loading to completion, with the CPU idle whenever the program waited for anything.
-
Multiprogramming kept several programs in memory at once so the CPU could switch to another whenever one blocked.
-
Time-sharing added preemption: a hardware timer stops a program whether or not it is ready to stop. This is what made interactive terminals feel responsive.
-
Threads arrived later, in the 1990s, as processes proved too heavy for programs that wanted many concurrent activities sharing data.
-
The trade was made then and stands now: threads bought speed by sharing memory, and the sharing is what still bites.
What it is — and what it is not
A process is:
- A running instance of a program, with a PID, its own address space, and a parent.
- The unit the kernel isolates, schedules and accounts for.
A process is not:
-
Not a program. One program can be running as fifty processes.
-
Not an application. A browser is many processes; that is deliberate, so one tab crashing does not take the rest.
-
Not always consuming CPU. Most are in waiting, and waiting is free.
Why it was created and what problems it solves
-
Utilisation. A CPU spends most of its life waiting on memory, disk or network. Processes let another program use those gaps.
-
Isolation. Separate address spaces mean one program’s bug cannot corrupt another’s data.
-
Responsiveness. Preemption means a runaway loop cannot lock you out of the machine; the timer will stop it regardless.
-
Accounting. The kernel knows what each process used, which is how limits, quotas and billing exist at all.
-
Control. A process has a name, a number and a parent, which is what makes it something you can inspect, signal and stop.
How it works
The anatomy and lifecycle of a process
-
Only running consumes a core. That is the state that costs.
-
A process in ready wants a core and is queued behind others.
-
A process in waiting asked for something slow — a disk read, a packet, a timer — and is charged nothing until the event arrives.
-
That is how one core serves hundreds of processes: most of them are asleep.
-
The ready-running loop in the middle is the scheduler’s territory: dispatch and preempt, over and over.
-
The excursion to waiting is where processes spend most of their lives on an interactive machine.
-
The exit records a code — the single number by which scripts and pipelines judge success.
echo $?shows the last one. -
A zombie has exited but not been collected. Its parent has not read its exit code, so it lingers as a dead entry in the process table. A few are normal; accumulating zombies mean a parent that never reaps.
Threads: cheap and dangerous
-
Locks are the cure, letting one thread into a critical section at a time.
-
Locks bring their own disease: deadlock, where two threads each hold the lock the other needs, forever.
-
The practical rule: threads shine when work is mostly waiting; for heavy computation in parallel, separate processes are safer — and in Python, historically the only effective choice.
The scheduler: time slices, context switches, preemption
-
Your machine has perhaps 500 processes and a couple of thousand threads, of which almost all are waiting. The scheduler only cares about the ready ones.
-
Each running thread gets a time slice on the order of milliseconds.
-
When it expires, a hardware timer interrupts the core. The thread is not asked. It is stopped.
-
The diagram shows one core and three processes: A runs and is preempted, B runs and blocks early on a disk read, C runs. The thin dark slivers between slices are the context switches.
-
Blocking is free. A thread waiting on disk or network is charged nothing.
-
So how do 100 programs run on 14 cores? They do not. They take very fast turns, and most of them are asleep.
Jobs and signals: taking the controls
-
End a command with
&and the shell starts it as a background job. You get your prompt back;$!holds the new child’s PID. -
jobslists your shell’s background children;psshows processes system-wide;waitpauses until a child finishes and hands you its code. -
You do not converse with a process. You signal it — a tiny numbered notification delivered by the kernel.
| Signal | No. | Sent by | Meaning | Refusable? |
|---|---|---|---|---|
| SIGTERM | 15 | kill PID | ”Please shut down.” | Yes |
| SIGINT | 2 | Ctrl+C | ”Stop what you’re doing.” | Yes |
| SIGKILL | 9 | kill -9 PID | ”Cease to exist, now.” | No |
| SIGSTOP | — | kill -STOP | ”Freeze, resumably.” | No |
| SIGCONT | — | bg, fg | ”Resume.” | — |
| SIGHUP | 1 | Terminal closes | ”Your terminal went away.” | Yes |
-
A signal death reports 128 plus the signal number. SIGTERM gives 143; SIGKILL gives 137.
-
SIGHUP explains the remote-job problem: close the terminal and your background children are told their terminal died, which by default ends them. Hence
nohup, and the terminal multiplexers later in this course.
Concurrency versus parallelism, precisely
-
Concurrency is dealing with many tasks in overlapping time periods. It is structure, not simultaneity.
-
Parallelism is executing more than one task at the same physical instant, which requires more than one execution unit.
-
One core running three processes in slices is concurrent, never parallel.
-
Python threads under the GIL give concurrency, not parallelism of Python computation — which is why frameworks reach for processes when they need real simultaneous work.
An everyday analogy
A single barista with four half-made drinks:
-
Starts a drink, hits a step that waits — milk steaming — and moves to the next.
-
That is concurrency. Four drinks in progress, one pair of hands.
-
Each switch costs something: remembering where the last drink was, finding the right cup, reaching for different tools.
-
Four baristas at four machines is parallelism, and needs four of everything.
-
A barista who switched drinks every three seconds would finish nothing. That is thrashing.
Examples in practice
-
Killedwith exit code 137 in a container log: the kernel sent SIGKILL, almost always for exceeding a memory limit. -
A training job that vanishes when you close your laptop: SIGHUP, delivered to everything attached to that terminal.
-
A data loader that gets slower as you add workers: more busy processes than cores, so the machine spends its time context switching.
-
A counter that is occasionally wrong and never reproducibly so: a race, and no amount of re-running will reliably show it.
-
Accumulating zombie processes: a parent that spawns children and never reaps their exit codes.
Implications: security, privacy, performance, scalability, and cost
-
Security: the process is the isolation unit. Separate address spaces are why a compromised process is not automatically a compromised machine, and why browsers put each tab in its own.
-
Privacy:
psis world-readable by default. Anything you pass as a command-line argument — an API key, a password — is visible to every user on the machine. Use environment variables or files instead. -
Performance: more threads than cores makes things slower, not faster, once the work is genuinely CPU-bound.
-
Scalability: blocking is free, so waiting workloads scale far beyond core count. Computing workloads do not.
-
Cost: SIGKILL loses work. A job killed without cleanup discards whatever was not yet flushed, and you pay again for the hours it takes to redo.
Alternatives: free, open source, and commercial
| Tool | Type | What it gives you |
|---|---|---|
ps, top, kill | Free, built in | Everything in this lesson, on any Unix |
htop | Free, open source | The same, readable, with a tree view |
nohup, setsid | Free, built in | Detach a job so SIGHUP cannot reach it |
tmux, screen | Free, open source | A terminal that survives disconnection |
systemd | Free, open source | Supervising long-running services properly |
| Activity Monitor / Task Manager | Bundled | The same data, with a mouse |
- Learn
htopearly. It shows the tree, the per-core load and the state column, and it makes this lesson visible in one screen.
Comparison with related concepts
| Term | What it actually means |
|---|---|
| Process | A running program with its own address space |
| Thread | An instruction stream sharing a process’s memory |
| Job | A shell’s name for a process it started |
| Daemon | A process deliberately detached from any terminal |
| Coroutine | Concurrency scheduled by your program, not the kernel |
When to use it — and when not to
-
Always try SIGTERM first. It gives the process a chance to flush files and release resources.
-
Reach for SIGKILL only when a process ignores polite requests, and accept that nothing will be cleaned up.
-
Use background jobs for anything longer than your patience, and
nohupor a multiplexer for anything longer than your session. -
Do not reach for threads to make computation faster in Python. Reach for processes.
-
Do not tune worker counts by guessing. Measure — the right number is usually near the core count, not far above it.
The AI connection
-
Exit code 137 is the single most common AI-infrastructure error message, and it means the kernel killed your job for memory.
-
Data loader workers are processes, which is why they cost memory each and why more is not always better.
-
A GPU sits idle while its feeder waits on disk. That waiting is exactly what threads overlap well.
-
Distributed training is many processes coordinating explicitly, on one machine or many — the same model, scaled up.
Knowledge check
quiz:
question: "A containerised training job disappears from the logs with exit code 137 and no traceback. What happened?"
options:
- "The kernel sent SIGKILL, most likely because the job exceeded its memory limit"
- "The Python process raised an unhandled exception that was not logged"
- "The container image was corrupted and the process never fully started"
- "The job completed successfully but the logging buffer was not flushed"
answer_index: 0
explanation: "128 + 9 = 137, and signal 9 is SIGKILL, which a process cannot catch. No cleanup runs and no traceback is written, which is exactly why the log ends without explanation."
Hands-on exercise
Watch processes being born, scheduled, signalled and reaped. Worked through fully in the Day 7 lab directory.
| What you are observing | Command |
|---|---|
| The family tree | ps -o pid,ppid,stat,command |
| Live state and per-core load | htop (or top) |
| Start a background job | sleep 300 & echo $! |
| List your shell’s children | jobs |
| Ask politely, then insist | kill $! then kill -9 $! |
| Read the exit code | echo $? |
Expected output
PID PPID STAT COMMAND
4812 4801 S sleep 300
[1]+ Terminated sleep 300
143
STAT Sis sleeping — waiting, and costing nothing.143is 128 + 15: it ended on SIGTERM.- Repeat with
kill -9and the code becomes 137.
Validate your work
- You can read a PID and its parent PID and explain the relationship
- You started a background job and retrieved its PID from
$! - You produced exit code 143 and exit code 137 deliberately
- You found a process in the waiting state and can say what it waits for
-
bash tests/run_tests.shreports 0 failures
Troubleshooting
-
htopnot installed. Usetop; the columns are less friendly but the same data is there. -
$!is empty. It holds the most recent background PID only. Capture it immediately after the&. -
killsays no such process. It already exited. Use a longersleep. -
Exit code is 0 when you expected 143. You read
$?after another command overwrote it. It reflects only the last thing that ran.
Common mistakes
-
Reaching for
kill -9first. It skips cleanup, and cleanup is often the part that saves your work. -
Assuming a sleeping process is a problem. Sleeping is what healthy processes mostly do.
-
Passing secrets as command-line arguments, where
psshows them to everyone. -
Spawning more busy workers than cores and expecting a speedup.
Practice assignment
-
Start three background jobs of different durations and use
waitto collect each exit code in turn. -
Kill one with SIGTERM and one with SIGKILL and record both codes.
-
Write a script that traps SIGTERM, prints a cleanup message, and exits — then prove that SIGKILL never prints it.
-
Run
htopwhile starting a CPU-heavy loop and watch a single core saturate. -
Explain in three sentences why the load stayed at one core.
Extension challenge
-
Build a race condition on purpose. Two threads, one shared counter, a million increments each. Run it ten times and record how often the total is wrong.
-
Then fix it with a lock and measure what the lock costs in time.
-
Find the crossover. Write the same CPU-bound work using threads and using processes, and find the worker count where processes overtake threads. Explain the number you found using the GIL.
-
Create a zombie deliberately: fork a child, let it exit, and have the parent sleep without reaping. Find it in
pswith stateZ.
Quiz
Q1. What is the difference between a program and a process?
- A program is a file on disk; a process is a running instance of it
- They are two words for the same thing
- A program runs in RAM while a process runs on the disk
- A process is the source code and a program is the compiled version
Show answer
Answer: A. A program is a file on disk; a process is a running instance of it
A program is just a file until the operating system loads it into memory, assigns it a PID, gives it private memory and file handles, and starts executing it — that running instance is a process, and one program can run as many processes at once.
Q2. A process asks to read a block from disk. Which lifecycle state does it enter while the disk works, and what does that cost in CPU time?
- Running — it keeps its core until the read completes
- Waiting — it is blocked until the event arrives and consumes no CPU at all
- Ready — it goes back into the scheduler queue and keeps consuming its slice
- Zombie — it is dead until the disk revives it
Show answer
Answer: B. Waiting — it is blocked until the event arrives and consumes no CPU at all
A process blocked on input or output moves to the waiting state, where the scheduler ignores it entirely; when the event arrives it returns to ready. Blocked processes cost no CPU, which is why one core can host hundreds of mostly-idle processes.
Q3. What exactly is a zombie process?
- A process whose parent died before it did
- A process consuming 100 percent CPU that cannot be killed
- A process that has exited but whose exit code is uncollected
- A background process detached from any terminal
Show answer
Answer: C. A process that has exited but whose exit code is uncollected
When a process exits, the kernel keeps a small process-table entry holding its exit code until the parent reads it with a wait call; until then the dead process is a zombie. A few momentary zombies are normal — accumulating zombies indicate a parent that never reaps its children.
Q4. Why are threads described as cheap compared with processes — and what makes them dangerous?
- Threads run on special low-power cores, but overheat easily
- Threads copy the whole process memory at creation, which is fast but wastes RAM
- Threads are cheaper because the scheduler never preempts them, which risks freezing the machine
- Threads share the process's memory, which is why races are possible
Show answer
Answer: D. Threads share the process's memory, which is why races are possible
Creating a thread only adds a stack and registers inside an existing address space, so it is far lighter than creating a process. But because all threads of a process share one memory, simultaneous unsynchronized updates can silently corrupt data — the race condition.
Q5. Your laptop has 14 cores and 480 processes. How do they all appear to run at once?
- Most are waiting; the scheduler time-slices too fast to notice
- The operating system compresses processes so 480 fit onto 14 cores simultaneously
- Each core secretly contains about 34 smaller cores
- Only 14 processes ever exist at a time; the rest are swapped to disk
Show answer
Answer: A. Most are waiting; the scheduler time-slices too fast to notice
At any instant at most 14 threads truly execute. Nearly all processes are blocked in the waiting state, and the few runnable ones take rapid preempted turns — time slices separated by context switches — creating the illusion of simultaneity.
Q6. What is the practical difference between kill (SIGTERM) and kill -9 (SIGKILL)?
- SIGTERM works only on background jobs; SIGKILL works on any process
- SIGTERM can be caught and handled; SIGKILL cannot be refused
- SIGKILL politely asks the process to stop; SIGTERM forces it
- They are identical; -9 just makes it faster
Show answer
Answer: B. SIGTERM can be caught and handled; SIGKILL cannot be refused
SIGTERM (signal 15, the default) lets a process save files and shut down gracefully — a process killed by it exits with code 143. SIGKILL (signal 9) is enforced by the kernel, runs no cleanup, and yields exit code 137; use it only after SIGTERM fails.
Q7. Which statement correctly distinguishes concurrency from parallelism?
- Parallelism is possible on a single core; concurrency requires at least two cores
- Concurrency is the hardware feature; parallelism is the software feature
- Concurrency overlaps tasks on one core; parallelism needs several
- They are synonyms used by different operating systems
Show answer
Answer: C. Concurrency overlaps tasks on one core; parallelism needs several
A single core running three processes in time slices is concurrent but never parallel; simultaneous execution needs multiple cores (or a GPU's thousands). All parallelism on a shared machine also involves concurrency, but not vice versa.
Q8. Why do machine-learning frameworks load training data with several worker processes rather than threads in Python?
- Threads cannot read files on most operating systems
- Worker processes can share the GPU while threads cannot
- Processes are always faster than threads for every workload
- The GIL lets only one thread run Python, so parallelism needs processes
Show answer
Answer: D. The GIL lets only one thread run Python, so parallelism needs processes
Under the GIL, Python threads provide concurrency (overlapping waits) but not parallel execution of Python code. Spawning worker processes gives each its own interpreter and memory, so many cores can genuinely preprocess batches simultaneously to keep the GPU fed.
Glossary
- process
- A running instance of a program: the program's instructions loaded into private memory, plus the PID, file handles, owner, parent, and state the operating system tracks to run and clean up after it.
- PID
- Process identifier — the unique number the operating system assigns to each process, used to observe it (ps -p) and signal it (kill).
- thread
- An independent stream of instructions inside a process, with its own program counter and stack but sharing the process's memory and file handles with its sibling threads.
- scheduler
- The part of the operating system that decides which runnable thread gets which CPU core, and for how long, balancing responsiveness, throughput, and fairness.
- context switch
- The act of saving one thread's complete CPU state and loading another's so a core can change what it is running — pure overhead that pays for the illusion of simultaneity.
- time slice
- The brief interval, typically a few milliseconds, that the scheduler lets a thread run before preempting it and giving the core to another.
- preemption
- The scheduler forcibly interrupting a running thread when its time slice expires or something more urgent arrives — the thread is not asked, it is stopped.
- signal
- A small numbered notification the kernel delivers to a process, such as SIGTERM ("please shut down"), SIGINT (Ctrl+C), or SIGKILL (unrefusable termination).
- exit code
- The number a process leaves behind when it terminates: 0 means success, other values signal failure, and 128 plus a signal number marks death by that signal (143 for SIGTERM, 137 for SIGKILL).
- zombie process
- A process that has exited but whose exit code has not yet been collected by its parent, leaving a dead entry in the process table until it is reaped.
- concurrency
- Structuring many tasks so they make progress over overlapping time periods — achievable on a single core through time slicing, with no true simultaneity required.
- parallelism
- Executing more than one task at the same physical instant, which requires multiple execution units such as several CPU cores or a GPU's thousands of cores.
- background job
- A command started with & so the shell does not wait for it: the process runs while the prompt returns immediately, its PID captured in $! and listed by jobs.
- race condition
- A bug in which the result depends on the timing of threads touching shared data — for example two threads both incrementing a counter and silently losing one update.
Sources and further reading
- Process (computing) — Wikipedia (accessed 2026-07-12)
- Thread (computing) — Wikipedia (accessed 2026-07-12)
- Scheduling (computing) — Wikipedia (accessed 2026-07-12)
- Operating Systems: Three Easy Pieces — Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau (accessed 2026-07-12) — Free online textbook; the virtualization chapters cover processes, the process API, and scheduling in depth.
- Crash Course Computer Science — PBS Digital Studios (accessed 2026-07-12) — Free video series; the operating-systems episode covers multitasking and scheduling visually.
Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.