Computing Foundations › Inside the Machine › Day 5

Day 5: Text, Images, and Sound as Data

Day 5 of 365 — Text, Images, and Sound as Data

After this lesson you will understand how text, images, and sound are all reduced to numbers — and why that is exactly the form modern AI models consume.

Course
Computing Foundations
Category
Inside the Machine
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-005-text-images-and-sound-as-data

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/computing-foundations/day-005-text-images-and-sound-as-data
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

A model never sees your file. It sees the numbers your preprocessing produced from it.


The idea in plain language

Everything is a number, and the only question is what agreement turns the thing into numbers.

Diagram: how text, images and sound each become arrays of numbers


Historical background

Diagram: sample rate compared with bit depth

WhenWhat arrivedWhy it mattered
1963ASCII7 bits, 128 characters, English only
1987GIFLossless images, 256 colours
1991Unicode 1.0One code point per character, every script
1992JPEGLossy compression tuned to human vision
1993UTF-8 publishedASCII-compatible, variable width, now dominant
1996PNGLossless, full colour, patent-free
2010Unicode adds emojiMulti-byte sequences reach everyday text

What it is — and what it is not

It is a set of agreements for turning real-world things into numbers.

It is not:


Why it was created and what problems it solves


How it works

Text: catalogue numbers, then bytes

Diagram: from a character to its code point to UTF-8 bytes to bits

Diagram: how UTF-8 encodes characters in one to four bytes

Images: a grid of measured light

Diagram: a two-by-two pixel image as a grid of RGB numbers

Diagram: lossless compression compared with lossy

Sound: slicing a wave into numbers

Video: images, sound and time

File formats: conventions with a signature

What models actually eat


An everyday analogy

Diagram: a sound wave measured at intervals and stored as numbers

A library catalogue, a photograph and a recording, all reduced to numbers on index cards.

The real thingThe agreementThe numbers
The letter AUnicode code point65
A shade of redRGB triple220, 20, 60
A moment of soundOne sample-8,412
A file’s identityMagic bytes0x89 0x50 0x4E 0x47

Examples in practice

Diagram: the size of video before and after compression

Diagram: identifying a file by extension compared with by magic bytes


Implications: security, privacy, performance, scalability, and cost

Diagram: the stages of turning a stored image into a training batch


Alternatives: free, open source, and commercial

ResourceTypeWhat it gives you
The Day 5 labFreeBytes, code points and signatures on your own files
Joel Spolsky, The Absolute Minimum About UnicodeFree essayThe clearest short explanation there is
unicode.org chartsFree referenceEvery code point, authoritative
xxd, file, iconvFree toolsAlready installed; answer most questions
Pillow / libvipsFree librariesImage decoding, with libvips far faster at scale
ffmpegFree toolAudio and video conversion, and inspection

ConceptWhat it doesWhat it is not
EncodingMaps characters to bytesNot encryption; no secret, no key
CompressionMakes the same data smallerNot encoding; the meaning is unchanged
EncryptionMakes data unreadable without a keyNot compression; size roughly holds
HashingFixes a fingerprint of the dataNot reversible; you cannot get the file back
SerialisationTurns a structure into bytesNot about characters; about shape

When to use it — and when not to

Think about representation when text renders as é, a model performs worse in production than in training, a pipeline is slower than the maths explains, or you are accepting files from users.

Ignore it when the library handles it and the results look right. Modern tooling gets this correct by default.

The tell: output that is subtly wrong rather than absent — wrong accents, wrong colours, audio at the wrong pitch, a model that was fine yesterday.


The AI connection

Diagram: matching compared with mismatched preprocessing between training and serving


Knowledge check

quiz:
  question: "A dataset of names loads without error, but a model trained on it performs worse than expected on European customers. What is the most likely cause?"
  options:
    - "The file was read with the wrong encoding, so accented names became mojibake"
    - "The model architecture is too small for the number of names"
    - "The names were stored in a column that was too narrow"
    - "European names are inherently harder to learn"
  answer_index: 0
  explanation: "Reading UTF-8 as Latin-1 turns é into é silently. Nothing raises, the rows all load, and the model learns from corrupted tokens for exactly the subset of the data that had accents."

Hands-on exercise

Look at the bytes behind text, an image and a file’s identity. Worked through fully in the Day 5 lab directory.

What you are checkingCommand
Characters vs bytespython3 -c "s='café'; print(len(s), len(s.encode()))"
A character’s code pointpython3 -c "print(hex(ord('é')))"
A file’s real formatxxd yourfile | head -1
Image dimensions and sizefile image.png
Break an encoding on purposeiconv -f latin1 -t utf8 file.txt

Expected output

4 5
0xe9
00000000: 8950 4e47 0d0a 1a0a  .PNG....

Validate your work

Troubleshooting

Common mistakes


Practice assignment

  1. Measure a string. Take a word with an accent and print both its character count and its byte count. Explain the difference.

  2. Identify a file by its bytes. Use xxd | head -1 on three files of different types and match each signature to its format.

  3. Break an encoding deliberately. Read a UTF-8 file as Latin-1 and record what happens to the accented characters.


Extension challenge

Find an emoji that is several code points joined together — a family, or one with a skin-tone modifier.

  1. Count three ways: characters, code points, and UTF-8 bytes.
  2. Slice it in half in your language of choice and record what you get.
  3. Write the conclusion: why “string length” is an ambiguous question, and which of the three numbers a text box’s character limit should use.

Quiz

Q1. What does a Unicode code point represent?

  1. A number assigned to a character, whatever its storage
  2. The number of bytes the character occupies on disk once saved
  3. The color value of a single pixel
  4. The compression ratio used to store a file
Show answer

Answer: A. A number assigned to a character, whatever its storage

A code point (like U+0041 for A) is the identity Unicode gives a character; encodings such as UTF-8 then decide how to store it as bytes.

Q2. Why did UTF-8 become the dominant text encoding on the web?

  1. It uses exactly one byte for every character in every alphabet
  2. It is ASCII-compatible and stores common characters compactly
  3. It stores text without using any bytes at all
  4. It only works for English and drops other alphabets
Show answer

Answer: B. It is ASCII-compatible and stores common characters compactly

UTF-8 keeps the original ASCII bytes for English text and uses two to four bytes for everything else, so old ASCII files stay valid while all of Unicode is supported.

Q3. A file renamed from photo.png to photo.jpg still opens as a PNG. Why?

  1. The extension always determines the real format
  2. PNG and JPEG are internally identical formats
  3. Its magic bytes identify the true format, not the extension
  4. The operating system converted the file automatically on download
Show answer

Answer: C. Its magic bytes identify the true format, not the extension

Programs read the leading magic bytes (PNG starts with 89 50 4E 47) to detect the real format; the filename extension is only a hint.

Q4. How is a single pixel most commonly represented in a color image?

  1. As a single bit that is either on or off
  2. As a Unicode code point per pixel
  3. As an audio sampling rate
  4. As red, green, and blue channel values
Show answer

Answer: D. As red, green, and blue channel values

A color pixel is typically three numbers — its red, green, and blue intensities — often 0 to 255 each.

Q5. What is the key difference between lossless and lossy compression?

  1. Lossless restores the original exactly; lossy discards detail
  2. Lossy can reconstruct the original exactly; lossless cannot
  3. Lossless files are always smaller than lossy ones
  4. They are two names for the same thing
Show answer

Answer: A. Lossless restores the original exactly; lossy discards detail

Lossless (PNG, ZIP) restores the exact original; lossy (JPEG, MP3) throws away detail to save space and cannot be perfectly reversed.

Q6. What does the sampling rate of digital audio measure?

  1. How loud the sound is at its peak
  2. How many times per second the wave is measured
  3. How many distinct colors the audio signal contains
  4. The size of the finished file in bytes
Show answer

Answer: B. How many times per second the wave is measured

Sampling rate (for example 44,100 Hz) is how often the continuous waveform is measured to turn it into a sequence of numbers.

Q7. Why does an emoji often take four bytes in a UTF-8 file?

  1. Emoji are stored as images inside the text
  2. UTF-8 always uses four bytes per character
  3. Its code point is high, so UTF-8 needs several bytes
  4. Emoji cannot be stored in UTF-8 at all
Show answer

Answer: C. Its code point is high, so UTF-8 needs several bytes

UTF-8 uses more bytes for higher code points; emoji sit high in the Unicode range and so require a three- or four-byte sequence.

Q8. How do AI models most directly consume images and audio?

  1. As filenames the model resolves at run time
  2. As Unicode text regardless of medium
  3. As compressed archives the model unpacks
  4. As arrays of numbers (tensors and waveforms)
Show answer

Answer: D. As arrays of numbers (tensors and waveforms)

Because all media is already numbers, a model receives an image as a tensor of pixel values and audio as an array of samples — the same representations this lesson describes.

Glossary

ASCII
An early 7-bit character encoding that maps 128 codes to English letters, digits, punctuation, and control characters.
Unicode
A universal standard that assigns a unique number (a code point) to every character in every writing system, plus emoji.
code point
The unique number Unicode assigns to a character, written like U+0041 for the letter A.
UTF-8
The dominant variable-length encoding of Unicode: it stores common characters in one byte and others in two to four bytes, staying backward-compatible with ASCII.
mojibake
Garbled text that appears when bytes are decoded with the wrong character encoding.
pixel
The smallest element of a raster image, holding a single color value.
RGB
A color model that represents a pixel as red, green, and blue channel values, commonly 0-255 each.
resolution
The number of pixels in an image, usually given as width by height.
raster image
An image stored as a grid of pixels (for example PNG or JPEG), as opposed to a vector image.
vector image
An image described as shapes and coordinates (for example SVG) that scales to any size without losing sharpness.
lossless compression
Compression that lets the original data be reconstructed exactly (for example PNG or ZIP).
lossy compression
Compression that discards some detail to shrink data further, which cannot be perfectly reversed (for example JPEG or MP3).
sampling rate
How many times per second a sound wave is measured when digitizing audio, in hertz (for example 44,100 Hz).
bit depth
How many bits represent each audio sample or color channel, setting how finely a value can be recorded.
magic bytes
A short signature at the start of a file that identifies its true format regardless of the filename extension.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.