What is machine learning?

Most software is a list of instructions a person wrote: if the basket total is over 500,000, apply free shipping; if the password is wrong three times, lock the account. Machine learning is a different way to get a program. Instead of writing the rule, you collect examples of the inputs together with the answers you want, and an algorithm finds a rule that reproduces those answers. This lesson explains what that means in practice, introduces the three words you will meet in every later lesson (model, training, prediction), shows where machine learning fits among the other things people call "AI", and helps you judge when it is the right tool and when a few lines of ordinary code would do better.

The intuition: rules you could not write down

Imagine you have to build a spam filter. You start writing rules. Messages containing "you have won" are spam. Messages with more than five links are spam. Messages in capital letters are spam. Then the complaints arrive: a newsletter with six links lands in the spam folder, and a scam that says "you are a winner" gets through. You add exceptions, then exceptions to the exceptions. After a month you have 200 rules, nobody understands how they interact, and spammers have already changed their wording.

Now try a different approach. You take 10,000 messages your users have already marked as spam or not spam, and you give them to a program whose job is to find the pattern that separates the two piles. It might discover that a high share of capital letters matters, that links matter mostly when combined with certain words, and that messages from people you have replied to are almost never spam. Nobody typed those rules in. They came out of the examples. When spammers change tactics, you collect fresh examples and run the program again.

That is machine learning: the programmer supplies examples and a way of measuring mistakes, and the computer supplies the rule. You still decide what the inputs are, what counts as a right answer and how to check the result. What you hand over is the tedious, error-prone job of turning thousands of examples into precise numbers.

The chart below is a spam filter with one input: the share of capital letters in a message. Drag the cut-off to write your own rule, then press "Learn from examples" to let the program choose it. Watch the two scores: one on the messages used to choose the rule, one on next month's messages that nobody looked at.

How it works

Traditional programming versus machine learning

The difference fits in two lines:

Traditional programming:   rules + input      →  answer
Machine learning:          inputs + answers   →  rules   (then: rules + new input → answer)

In traditional programming a person turns knowledge into rules. In machine learning a training algorithm turns examples into rules, and those rules are then used like any other program. The learned rule is usually not a tidy sentence like "spam if more than five links". It is a set of numbers: weights in an equation, thresholds in a tree of questions, or millions of connection strengths in a neural network. Those numbers are called the model's parameters.

Model, training and prediction

Three words carry most of this course:

• Model: the learned rule. Technically, it is a function that takes an input and returns an output, with its behaviour set by its parameters. A spam model takes a message and returns "spam" or "not spam"; a house-price model takes a description of a house and returns a price.
• Training (also called fitting or learning): the process that chooses the parameters. The training algorithm looks at the examples, measures how wrong the current parameters are, and adjusts them to be less wrong. In scikit-learn this is the .fit(X, y) call.
• Prediction (also called inference): using the trained model on a new input. In scikit-learn this is .predict(X_new). Prediction is usually fast and cheap; training is where the computing time goes.

Before training you choose a model family, the shape the rule is allowed to take. A linear model can only learn a weighted sum of the inputs (lesson 13). A decision tree learns a sequence of yes/no questions (lesson 17). A k-nearest neighbours model keeps the examples and answers by looking up the most similar ones (lesson 15). Training then finds the best parameters within that family. Choosing the family is your job; filling in the numbers is the algorithm's.

Generalisation: the whole point

A model is useful only if it works on inputs it has never seen. Getting the training examples right is easy: a program that memorises every example and its answer gets 100% on them and is useless on anything new. What you want is a model that has picked up the real pattern, so that it is right on next month's messages, next year's houses and the patient who walks in tomorrow. This ability is called generalisation.

Two consequences follow, and they shape the rest of this course:

1. You always measure a model on data it did not train on. A score on the training examples measures memory, not skill. Lesson 4 shows how to split data so that the score you report is honest.
2. A model can be too simple or too flexible. Too simple, and it misses the pattern (a straight line through data that curves). Too flexible, and it learns the noise in the training examples as if it were pattern. Lesson 6 calls this the bias–variance trade-off.

A quick map of the field

"AI", "machine learning", "deep learning" and "large language models" are used loosely. They nest inside each other:

This course is about classical machine learning. It is not old-fashioned: on tabular data, which is most business data, gradient-boosted trees (lesson 20) still regularly match or beat deep learning, train in seconds to minutes on an ordinary CPU, and are easier to inspect. The ideas you learn here (training versus test data, features, loss, overfitting, evaluation metrics) carry over unchanged to deep learning and language models.

Worked example: a learned rule versus a hand-written one

The script below creates a small, made-up inbox. Each message is described by two numbers, how many links it contains and what share of its letters are capitals, plus a label: spam (1) or not spam (0). There are 200 labelled messages to learn from and 1,000 "next month" messages to check on. It compares three rules:

1. A hand-written rule: five or more links means spam.
2. A rule learned by a decision tree that may ask at most two questions.
3. A tree with no limit, which can keep asking questions until it gets every training message right.

import numpy as np
from sklearn.tree import DecisionTreeClassifier, export_text

rng = np.random.default_rng(0)

def make_messages(n):
    """Fake inbox: links per message, share of capital letters, and spam (1) or not (0)."""
    spam = rng.random(n) < 0.4
    links = np.where(spam, rng.poisson(4, n), rng.poisson(1, n))
    caps = np.where(spam, rng.normal(0.30, 0.10, n), rng.normal(0.10, 0.06, n)).clip(0, 1)
    X = np.column_stack([links, caps])
    return X, spam.astype(int)

X_old, y_old = make_messages(200)   # messages we have already labelled
X_new, y_new = make_messages(1000)  # messages that arrive next month

# 1. Traditional programming: a person writes the rule
def hand_rule(X):
    return (X[:, 0] >= 5).astype(int)   # "five or more links means spam"

# 2. Machine learning: the program finds the rule from labelled examples
model = DecisionTreeClassifier(max_depth=2, random_state=0)
model.fit(X_old, y_old)

print("Rule the model learned:")
print(export_text(model, feature_names=["links", "caps_share"]))

# 3. A model with no limit on rule length memorises the examples instead
memoriser = DecisionTreeClassifier(random_state=0).fit(X_old, y_old)

for name, predict in [("Hand-written rule", hand_rule), ("Learned rule", model.predict),
                      ("Memorised rules", memoriser.predict)]:
    old_acc = (predict(X_old) == y_old).mean()
    new_acc = (predict(X_new) == y_new).mean()
    print(f"{name:18s} accuracy on known messages {old_acc:.3f}, on new messages {new_acc:.3f}")

print(f"Learned rule has {model.get_n_leaves()} leaves, memorised rules have {memoriser.get_n_leaves()}")

message = np.array([[3, 0.42]])   # 3 links, 42% capital letters
print("New message [3 links, 42% caps] ->", "spam" if model.predict(message)[0] else "not spam")

Output:

Rule the model learned:
|--- caps_share <= 0.22
|   |--- links <= 4.50
|   |   |--- class: 0
|   |--- links >  4.50
|   |   |--- class: 1
|--- caps_share >  0.22
|   |--- class: 1

Hand-written rule  accuracy on known messages 0.790, on new messages 0.746
Learned rule       accuracy on known messages 0.975, on new messages 0.932
Memorised rules    accuracy on known messages 1.000, on new messages 0.929
Learned rule has 3 leaves, memorised rules have 13
New message [3 links, 42% caps] -> spam

What the output shows

• The learned rule is readable. The tree found that a capital-letter share above 22% is the strongest sign of spam, and that among the remaining messages more than 4.5 links (so five or more) also means spam. Nobody gave it the number 0.22; it came from the examples.
• The learned rule beats the hand-written one. The hand-written rule looked only at links and scored 74.6% on new messages. The learned rule used both inputs and scored 93.2%. A person could have found a similar rule by staring at the data, but the algorithm did it in milliseconds and would do the same with 50 inputs instead of two.
• New messages score lower than known ones, for every rule. That gap is normal. The known messages are the ones each rule was chosen to fit, so their score flatters it. The score on new messages is the one that predicts how the filter will behave in use.
• Perfect training accuracy bought nothing. The unlimited tree used 13 leaves to get every known message right (1.000), yet it was no better on new messages (0.929 against 0.932). The extra questions fitted quirks of those 200 messages, not the real pattern. On noisier data this gap gets much larger; lesson 6 explains why, and lesson 17 shows how to limit a tree.
• Prediction is one call. Once trained, the model labels a message it has never seen (3 links, 42% capitals) as spam, because its capital-letter share is above 0.22.

When machine learning is a good fit, and when it is not

Machine learning tends to work well when:

• The rule is hard to write but easy to demonstrate. People can label a photo as "cracked" or "fine", or a review as positive or negative, far more easily than they can write the rule they use. Examples are cheaper than rules.
• There are many inputs that interact. Credit risk depends on income, debt, history, age of accounts and dozens of other signals in combination. A model weighs them consistently; a person cannot hold 40 factors in mind.
• The pattern changes over time. Fraud, spam and demand shift. Retraining on fresh examples is easier than rewriting rules by hand.
• You have, or can collect, enough examples with reliable answers. How many depends on the problem: a few hundred rows can be enough for a simple pattern in a table, while recognising objects in photos needs many thousands.
• Being right most of the time is valuable, and the occasional mistake can be caught or tolerated: a product recommendation, a spam folder, a first-pass triage that a person reviews.

It is usually the wrong tool when:

• A clear rule already exists. Tax brackets, shipping fees and age checks are policies, not patterns. Write the if statement; it will be right every time and anyone can audit it.
• There is no data, or the answers in the data are unreliable. A model learns whatever the examples show, including their mistakes and biases. If past decisions were unfair, a model trained on them will repeat the unfairness at scale.
• Every mistake is unacceptable and cannot be checked. Every model is wrong some of the time. If a single error is catastrophic and no person or rule sits behind the model to catch it, machine learning should not be making that decision alone.
• You need the reason, not only the answer, and the reason must be exact. Some models can be inspected (lessons 26–28 cover how), but a learned rule is a statistical summary, not a legal argument.
• The future will not look like the past. A model assumes new data resembles its training data. A demand model trained before a pandemic, a price change or a new competitor may be confidently wrong afterwards.

A good habit: before training anything, write down the simplest non-learning solution (a hand-written rule, or always predicting the most common answer) and its score. That is your baseline. A model has to beat it by enough to justify the extra work, and lesson 13 shows how much a baseline can tell you.

Key takeaways

• Machine learning produces a program from examples: you supply inputs with the right answers, and a training algorithm finds the rule.
• A model is the learned function, its parameters are the numbers training chooses, training fits them to examples, and prediction applies the model to new inputs.
• What matters is generalisation, performance on data the model has never seen; a perfect score on training examples can mean memorisation.
• Classical machine learning (this course) works on tables; deep learning and large language models are machine learning with large neural networks, mostly for images, audio and text.
• Use machine learning when the rule is hard to write but examples are plentiful and reliable; use ordinary code when the rule is known or every answer must be exact.

Try it yourself

1. Change the hand-written rule to X[:, 1] > 0.25 (more than 25% capital letters) and rerun. How close can you get to the learned rule by hand, and how long did it take you to find a good threshold?
2. Change make_messages(200) to make_messages(20) for the known messages. Print the learned rule and both accuracies. What happens to the gap between known and new messages when the model learns from fewer examples?
3. Add noise to the labels by flipping 15% of y_old (flip = rng.random(200) < 0.15; y_old = np.where(flip, 1 - y_old, y_old)) before training. Compare the two-question tree and the unlimited tree on new messages. Which one suffers more, and why?