In the last lesson the computer only ever saw each example as two numbers: how far across and how far up. That was not a shortcut for the demo. It is how every model works. A model takes numbers in and gives numbers out, so before a computer can learn anything about photos, fruit, songs, or sentences, those things have to become numbers.
This lesson is about that step, and about a surprising fact: which numbers you choose matters as much as how clever the model is.
A picture is a grid of numbers
A digital picture is a grid of tiny squares called PixelOne tiny square of a digital image. A grayscale pixel is stored as one brightness number; a color pixel as three (red, green, and blue).Open in glossary. In a black-and-white picture, each pixel is stored as one number for its brightness. Below, 0 is black and 255 is white.
A picture is a grid of numbers
Each square is one pixel, stored as a brightness from 0 (black) to 255 (white). Paint to change the numbers.
A color photo stores three numbers per pixel (red, green, and blue), so a 12-megapixel phone photo is about 36 million numbers.
Try this
- Paint a few squares white and watch their numbers change to 255. The picture and the numbers are the same thing, shown two ways.
- Turn off Show the numbers. You see a face. The computer never does: it only has the hundred numbers.
- Paint with gray. Brightness is not just on or off; every value from 0 to 255 is allowed.
To you, the picture is obviously a smiling face. To the computer, it is a list of 100 numbers, mostly 235 with some 20s. Recognizing the face means finding a pattern in those numbers, and that is exactly the kind of thing models learn.
Sound works the same way: a recording is a long list of numbers measuring air pressure, typically 44,100 of them for every second of audio. Text needs a few more steps, which you will see in the language track, but it also ends up as numbers.
Describing things with features
Not everything starts as a picture. Often you describe each example with a handful of measurements. Each measurement is called a FeatureOne measurable property of an example, given to a model as a number, such as a fruit's weight or a pixel's brightness.Open in glossary, and the answer you want the model to learn is called the LabelThe answer a model should learn to give for an example, such as "apple" or "orange". Labeled examples are what supervised learning learns from.Open in glossary.
Here are a few fruit described that way. Each row is one example. The first four columns are features; the last is the label.
| Weight (g) | Width (cm) | Skin color | Skin bumpiness (0 to 10) | Label |
|---|---|---|---|---|
| 167 | 7.2 | red | 0.4 | apple |
| 234 | 8.0 | green | 0.4 | apple |
| 150 | 7.4 | orange | 7.4 | orange |
| 172 | 7.6 | orange | 8.2 | orange |
Skin color is written as a word here so you can read it, but the model gets a number: the color’s position on a color wheel, where red is near 0, orange near 30, yellow near 60, and the yellowish green of an unripe apple near 90.
Some features are better than others
Below are sixty fruit, thirty apples and thirty oranges, each measured four ways. The fruit are made up for this demo, but their measurements follow realistic ranges. Pick two features to plot. Each dot is one fruit, placed by its two numbers.
Which measurements tell apples from oranges?
Sixty fruit, measured four ways. Choose two measurements to plot and see how well a model can tell the fruit apart using only those.
Try this
- Start with Weight across and Width up. Apples and oranges are mixed together, and the model gets only about half of them right.
- Change Up to Bumpiness. The two kinds of fruit pull apart into separate groups, and the score jumps.
- Try Color with Weight. Color helps, but notice that apples form two groups, red and green, with oranges in between.
- Before switching, predict the score for Width and Color. Then check.
The model never changed. It is the same nearest-neighbors method from the last lesson every time. What changed was the information it was given. With weight and width, apples and oranges look the same, so the model is guessing. With skin bumpiness, the difference is right there in the numbers.
That is the first lesson of practical machine learning: a model can only find patterns that are present in the numbers it receives. No amount of cleverness recovers information you left out.
For decades, people chose features by hand, a job called feature engineering. A big change brought by Deep learningMachine learning with neural networks that have many layers. It powers modern image recognition, speech recognition, translation, and chatbots.Open in glossary is that models now often start from raw numbers, like every pixel of a photo, and learn useful features on their own. You will see how in the neural network lessons.
Why the demo rescales the numbersOptional
Weight is measured in grams and runs from about 110 to 250. Bumpiness runs from 0 to 10. If you measured distance between fruit using the raw numbers, a 20-gram difference in weight would swamp the biggest possible difference in bumpiness, and the model would effectively ignore bumpiness.
So before measuring distances, the demo rescales each feature: it subtracts that feature’s average and divides by how spread out it is (its standard deviation). After that, every feature has a similar range, and each one gets a fair say. This step, called standardization, is routine in machine learning.
The score in the demo is measured fairly too: each fruit is guessed from its five nearest other fruit, so no fruit can vote for itself.
Key ideas
- Models only see numbers. Pictures become grids of brightness values; sounds become lists of measurements.
- An example is described by features (the measurements) and a label (the answer to learn).
- A model can only learn patterns that are present in its features. Good features make learning easy; useless ones make it impossible.
- Modern deep learning often learns its own features from raw data instead of relying on hand-picked ones.