Types of data

Welcome to Data Analytics: Statistics and Visualisation. This course teaches you to describe data with the right numbers, to check whether a pattern is real or just noise, and to draw charts that tell the truth. It assumes you can read basic Python and have seen a pandas DataFrame before; it does not assume any statistics. Every idea comes with code you can run, and every number quoted in the text is the actual output of that code.

The course has sixteen lessons in four parts. Descriptive statistics (lessons 1 to 4) covers the kinds of data you will meet, averages, spread, percentiles and outliers. Distributions (lessons 5 to 8) looks at the shape of data: histograms and skew, the normal distribution, the binomial, Poisson and exponential distributions, and the central limit theorem, which is the reason so much of statistics works. Relationships and inference (lessons 9 to 12) covers correlation, association between categories, confidence intervals and hypothesis tests. Visualisation (lessons 13 to 16) covers choosing a chart, drawing it with matplotlib and seaborn, designing it honestly, and a final analysis that goes from a business question to a short written report.

This first lesson starts where every analysis should: before computing anything, work out what kind of data each column holds. The type of a variable decides which averages, charts and tests make sense.

Setting up

You need Python 3.11 or later and five libraries. Install them from a terminal:

pip install pandas numpy scipy matplotlib seaborn

pandas holds tables, NumPy does arithmetic on arrays and generates random numbers, scipy.stats supplies distributions and statistical tests, matplotlib draws charts, and seaborn builds statistical charts on top of matplotlib. The examples run as scripts or in a Jupyter notebook; they use print throughout so both behave the same. Check your installation:

import sys
import numpy as np
import pandas as pd
import scipy
import matplotlib
import seaborn as sns

print("Python", sys.version.split()[0])
print("pandas", pd.__version__, "| numpy", np.__version__, "| scipy", scipy.__version__)
print("matplotlib", matplotlib.__version__, "| seaborn", sns.__version__)

Output:

Python 3.12.12
pandas 3.0.6 | numpy 2.5.4 | scipy 1.18.1
matplotlib 3.11.2 | seaborn 0.13.2

Your versions may differ slightly; anything from pandas 2.2 and NumPy 2.0 upwards will run the examples. The outputs in this course came from pandas 3, which stores text columns with a dedicated string type (shown as str) instead of the older object. If you see object in your own output, that is the only difference.

The course dataset: a small online shop

Throughout the course we follow one business: a small online shop that sells electronics, homeware, clothing, books and beauty products. You have a year of its orders, 1,000 rows, and the questions a shop owner would ask. What does a typical order look like? Which categories get returned? Do returning customers spend more? Does slow delivery hurt ratings?

The data is synthetic: NumPy generates it with a fixed random seed, so you get exactly the same 1,000 orders on any machine, without downloading anything. Because we wrote the rules, we also know the right answers, so you can see whether a statistical method finds them. You do not need to understand every line of the function yet; lessons 5 to 7 explain the distributions it uses.

Every lesson in the course starts by pasting this same function, so each lesson runs on its own.

import numpy as np
import pandas as pd


def make_orders(n=1000, seed=42):
    """One year of orders from a small online shop, generated reproducibly."""
    rng = np.random.default_rng(seed)
    category = rng.choice(["Electronics", "Home", "Clothing", "Books", "Beauty"],
                          size=n, p=[0.15, 0.20, 0.30, 0.20, 0.15])
    customer_type = rng.choice(["new", "returning"], size=n, p=[0.4, 0.6])
    device = rng.choice(["mobile", "desktop", "tablet"], size=n, p=[0.55, 0.35, 0.10])
    shipping = rng.choice(["standard", "express", "next-day"], size=n, p=[0.6, 0.3, 0.1])
    items = rng.poisson(1.2, size=n) + 1                     # items in the basket, at least 1

    base = pd.Series({"Electronics": 4.3, "Home": 3.7, "Clothing": 3.5,
                      "Books": 2.9, "Beauty": 3.2})[category].to_numpy()
    log_mean = base + 0.15 * (items - 1) + 0.12 * (customer_type == "returning")
    order_value = np.round(rng.lognormal(log_mean, 0.45), 2)  # right-skewed, in GBP
    bulk = rng.choice(n, size=5, replace=False)               # five bulk orders (outliers)
    order_value[bulk] = np.round(order_value[bulk] * 8, 2)

    lag = pd.Series({"standard": 4, "express": 2, "next-day": 1})[shipping].to_numpy()
    delivery_days = lag + rng.poisson(0.8, size=n)
    session_minutes = np.round(rng.gamma(2.0, 4.0, size=n) + 2.0 * items
                               + 0.04 * np.minimum(order_value, 200), 1)
    customer_age = np.clip(np.round(rng.normal(38, 11, size=n)), 18, 80).astype(int)
    rating = np.clip(np.round(rng.normal(4.2 - 1.0 * (delivery_days > 5), 0.8)), 1, 5).astype(int)
    p_return = pd.Series({"Electronics": 0.08, "Home": 0.06, "Clothing": 0.20,
                          "Books": 0.03, "Beauty": 0.05})[category].to_numpy()
    returned = rng.random(n) < p_return
    order_date = pd.Timestamp("2025-01-01") + pd.to_timedelta(rng.integers(0, 365, size=n), unit="D")

    orders = pd.DataFrame({
        "order_date": order_date,
        "category": category,
        "customer_type": customer_type,
        "device": device,
        "shipping": pd.Categorical(shipping, categories=["standard", "express", "next-day"],
                                   ordered=True),
        "items": items,
        "order_value": order_value,
        "delivery_days": delivery_days,
        "session_minutes": session_minutes,
        "customer_age": customer_age,
        "rating": rating,
        "returned": returned,
    }).sort_values("order_date", ignore_index=True)
    orders.insert(0, "order_id", np.arange(1001, 1001 + n))
    return orders


orders = make_orders()
print(orders.shape)
print(orders.head(3).T)

Output:

(1000, 13)
                                   0                    1                    2
order_id                        1001                 1002                 1003
order_date       2025-01-01 00:00:00  2025-01-01 00:00:00  2025-01-01 00:00:00
category                      Beauty          Electronics                 Home
customer_type              returning                  new            returning
device                       desktop              desktop               mobile
shipping                    standard             standard             standard
items                              3                    1                    2
order_value                     49.4                50.57                31.75
delivery_days                      5                    4                    4
session_minutes                 12.0                 14.3                  9.5
customer_age                      21                   18                   23
rating                             4                    5                    5
returned                       False                False                False

The transposed view (.T) shows one order per column so all thirteen fields fit on the screen. Each row of the real table is one order: when it was placed, what was bought, by whom, on which device, how it was shipped, how much it cost in pounds, how long it took to arrive, how long the customer spent on the site, the customer's age, the 1-to-5 star rating they left, and whether the order was sent back.

The intuition

Imagine three columns on a spreadsheet: the number of items in a basket, the product category, and the star rating. All three could be stored as numbers. You could code the categories as 1 to 5 and the spreadsheet would happily compute an average category of 2.7. The arithmetic works; the meaning does not. There is no category halfway between Home and Clothing, because the codes are just labels.

The star rating sits in between. Five stars is better than four, so the order means something, but is the gap from one star to two the same as the gap from four to five? Nobody promised that. The number of items, by contrast, is a true quantity: a basket of four items holds exactly one more than a basket of three and twice as many as a basket of two. Every kind of arithmetic means something.

That is the whole idea of this lesson. Before you summarise a column, ask what the values mean, not how they are stored. The answer puts the column into one of a few types, and each type comes with a set of operations that make sense.

How it works

Numerical and categorical

The first split is between numerical (quantitative) data, where the values are amounts you can do arithmetic with, and categorical (qualitative) data, where the values are labels that put each row into a group.

Numerical data splits again:

• Discrete values are counts, so they come in whole steps: items in the basket, days to deliver, number of complaints. You cannot have 2.5 items.
• Continuous values are measurements that could, in principle, take any value in a range: order value, minutes on the site, weight, temperature. In practice they are rounded (to the penny, to a tenth of a minute), but the rounding is about the measuring, not the thing measured.

Categorical data splits too:

• Nominal categories have no order: product category, device, country, payment method. Electronics is not "more" than Books.
• Ordinal categories have a natural order but no fixed distance between steps: shipping speed (standard, express, next-day), star ratings, survey answers from "strongly disagree" to "strongly agree", T-shirt sizes.

A binary variable (returned or not, churned or not) is a categorical variable with two levels. It is worth singling out because, coded as 0 and 1, its mean is exactly the proportion of 1s, which is the most useful number you can report about it.

Levels of measurement

Statisticians often use a four-step ladder, from the psychologist Stanley Stevens, that says which operations are meaningful:

level      example                 equal / not equal   order   differences   ratios
nominal    category, device        yes                 no      no            no
ordinal    shipping speed, rating  yes                 yes     no            no
interval   order date, °C          yes                 yes     yes           no
ratio      order value, items      yes                 yes     yes           yes

The difference between interval and ratio is whether zero means "none". Zero pounds is no money, so £80 really is twice £40: order value is a ratio variable. A date has no natural zero, so 10 March minus 1 March is a meaningful nine days, but "10 March is ten times 1 March" is nonsense. Temperature in Celsius is the classic interval example: 20 °C is not twice as hot as 10 °C. In everyday analysis interval and ratio are usually treated together as numerical, but the ladder is a useful check on any statement of the form "X is twice Y".

What each type allows you to compute and plot

The type decides the summary statistics and charts that make sense. This table is the practical result of the lesson; later lessons fill in the details.

Storage type is not data type

pandas has its own types (dtypes): int64, float64, bool, str, category, datetime64. These describe how values are stored, and they are only a hint about the statistical type. A postcode stored as an integer is still a nominal label. An order ID is a number, but averaging IDs is meaningless: it identifies a row and is not a variable at all. A rating stored as int64 is ordinal. Part of your job is to tell pandas what you know, so that it helps rather than misleads.

The tools for that are the category dtype for nominal labels and pd.Categorical(..., ordered=True) for ordinal ones, which lets pandas sort and compare them properly.

Worked example: classifying the shop's columns

Start by looking at what pandas thinks each column is and how many distinct values it has. A column with only a handful of distinct values is often categorical even when it is stored as numbers.

summary = pd.DataFrame({
    "dtype": orders.dtypes.astype(str),
    "distinct": orders.nunique(),
    "example": orders.iloc[0],
})
print(summary)

Output:

                          dtype  distinct              example
order_id                  int64      1000                 1001
order_date       datetime64[us]       342  2025-01-01 00:00:00
category                    str         5               Beauty
customer_type               str         2            returning
device                      str         3              desktop
shipping               category         3             standard
items                     int64         6                    3
order_value             float64       955                 49.4
delivery_days             int64         9                    5
session_minutes         float64       251                 12.0
customer_age              int64        54                   21
rating                    int64         5                    4
returned                   bool         2                False

Reading down the list with the definitions above gives this classification:

• order_id: an identifier. Unique for every row, so it is neither numerical nor a useful category.
• order_date: a date (interval scale). Differences between dates are meaningful.
• category, customer_type, device: nominal. customer_type is also binary.
• shipping: ordinal, already stored as an ordered categorical.
• items, delivery_days: discrete numerical counts.
• order_value, session_minutes: continuous numerical.
• customer_age: age is continuous in reality but recorded in whole years, so it behaves like a discrete numerical variable with many values. Treat it as numerical.
• rating: ordinal, despite being stored as an integer.
• returned: binary.

Summarising nominal and binary columns

For a nominal column the only "average" is the mode, and the best summary is the share of each category. For a binary column the mean is the proportion.

print(orders["category"].value_counts())
print()
print(orders["category"].value_counts(normalize=True).round(3))
print("\nMost common category:", orders["category"].mode()[0])
print("Share of orders returned:", orders["returned"].mean())
print("Orders returned:", orders["returned"].sum())

Output:

category
Clothing       288
Books          200
Home           196
Electronics    165
Beauty         151
Name: count, dtype: int64

category
Clothing       0.288
Books          0.200
Home           0.196
Electronics    0.165
Beauty         0.151
Name: proportion, dtype: float64

Most common category: Clothing
Share of orders returned: 0.078
Orders returned: 78

Clothing is the most common category, with 28.8% of orders, and 78 orders (7.8%) were returned. value_counts(normalize=True) turns counts into proportions, which are easier to compare across datasets of different sizes. orders["returned"].mean() works because pandas treats True as 1 and False as 0, so the mean of the column is the proportion of returned orders and the sum is the count.

Ordinal columns: let pandas know the order

The shipping column was created as an ordered categorical, so pandas sorts it by speed rather than alphabetically, and comparisons such as "at least express" work:

print(orders["shipping"].dtype)
print(orders["shipping"].value_counts(sort=False))
fast = orders["shipping"] >= "express"
print("Orders shipped express or faster:", fast.sum())

# Ratings are stored as integers: make the ordinal nature explicit
rating = pd.Categorical(orders["rating"], categories=[1, 2, 3, 4, 5], ordered=True)
print("\nRating counts:")
print(pd.Series(rating).value_counts(sort=False))
print("Median rating:", orders["rating"].median())

Output:

category
shipping
standard    605
express     292
next-day    103
Name: count, dtype: int64
Orders shipped express or faster: 395

Rating counts:
1      1
2     31
3    209
4    480
5    279
Name: count, dtype: int64
Median rating: 4.0

Two things are worth noticing. value_counts(sort=False) keeps the categories in their defined order (standard, express, next-day) instead of sorting by frequency, which is what you want for an ordinal variable. And a plain string column sorted alphabetically would put "express" before "next-day" before "standard", which is the wrong order; declaring the order once fixes every sort, comparison and chart afterwards.

For ratings, the median is always safe because it only needs the order. The mean rating is a different matter, discussed under pitfalls below.

Numerical columns: discrete and continuous

For discrete counts with few distinct values, a frequency table is often more informative than any single number. For continuous values, describe() gives the standard summary, which lessons 2 to 4 take apart one statistic at a time.

print("Items per order:")
print(orders["items"].value_counts().sort_index())
print()
print(orders[["order_value", "session_minutes", "customer_age"]].describe().round(2))

Output:

Items per order:
items
1    305
2    401
3    190
4     74
5     26
6      4
Name: count, dtype: int64

       order_value  session_minutes  customer_age
count      1000.00          1000.00       1000.00
mean         54.88            14.30         37.93
std          48.05             6.31         10.87
min           5.90             3.00         18.00
25%          27.57             9.80         30.00
50%          41.10            13.00         38.00
75%          67.15            17.90         45.00
max         797.44            48.70         80.00

The item counts are whole numbers from 1 to 6, and 706 of the 1,000 orders contain one or two items. Order values range from £5.90 to £797.44, and the mean (£54.88) is well above the median (the 50% row, £41.10): a first sign of a skewed distribution with a few large orders, which lesson 2 explains.

The wrong answers that types prevent

To see why this matters, here are three summaries that pandas computes without complaint and that mean nothing:

codes = orders["category"].astype("category").cat.codes  # labels turned into 0-4
print("Mean category code:", round(codes.mean(), 2))
print("Mean order ID:     ", orders["order_id"].mean())
print("Sum of ratings:    ", orders["rating"].sum())

Output:

Mean category code: 2.06
Mean order ID:      1500.5
Sum of ratings:     4005

An average category code of 2.06 does not describe any category, and it would change if someone sorted the category names differently before coding them. The mean order ID is 1500.5 only because IDs run from 1001 to 2000. The sum of ratings is a number with no unit and no use. None of these raised an error. The software cannot know what your numbers mean; only you can.

Choosing a chart by type

The same reasoning picks the chart. A categorical column gets a bar chart of counts; a continuous column gets a histogram, which groups values into ranges (bins) and counts each range. Lesson 5 covers histograms properly, and lessons 13 to 15 cover chart choice and design in depth.

import matplotlib
matplotlib.use("Agg")          # draw to files; remove this line in a notebook
import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 3, figsize=(12, 3.5))

counts = orders["category"].value_counts()
axes[0].bar(counts.index, counts.values, color="#4C72B0")
axes[0].set_title("Nominal: orders by category")
axes[0].set_ylabel("Orders")
axes[0].tick_params(axis="x", rotation=30)

ship = orders["shipping"].value_counts(sort=False)
axes[1].bar(ship.index.astype(str), ship.values, color="#4C72B0")
axes[1].set_title("Ordinal: shipping, in speed order")

axes[2].hist(orders["order_value"], bins=40, color="#4C72B0")
axes[2].set_title("Continuous: order value")
axes[2].set_xlabel("Order value (GBP)")

fig.tight_layout()
fig.savefig("data_types.png", dpi=150)
plt.close(fig)
print("Saved data_types.png")

Output:

Saved data_types.png

The nominal bars are sorted by count, because there is no natural order and sorting makes comparison easy. The ordinal bars keep their natural order even though that is not sorted by size, because the order carries meaning. The histogram has no gaps between bars, because the values are continuous and every value falls in some bin.

Reference: pandas tools for data types

Common pitfalls

• Averaging codes. Categories stored as numbers (region 1 to 5, product type 10, 20, 30) will be averaged by any tool that sees numbers. Convert them to a categorical dtype as soon as you load the data.
• Treating IDs and postcodes as numbers. Customer numbers, phone numbers and postcodes are labels. Store them as strings; leading zeros disappear when a postcode is read as an integer.
• Sorting ordinal labels alphabetically. "High, Low, Medium" is the alphabetical order and the wrong one. Declare the order with pd.Categorical(..., ordered=True).
• The mean of a rating scale. Averaging star ratings assumes the steps are equally spaced. It is extremely common and often useful for comparing products, but report the distribution or the median alongside it, and do not read a difference between 4.1 and 4.2 as meaningful without checking (lesson 12).
• Binning a continuous variable too early. Turning age into "18–30, 31–45, 46+" throws information away and the cut-offs can change the conclusion. Keep the raw values and bin only for presentation.

Key takeaways

• Decide what each column means before you summarise it; the storage type is only a hint.
• Numerical data is discrete (counts) or continuous (measurements); categorical data is nominal (no order) or ordinal (ordered, unequal steps).
• Nominal: report the mode and proportions. Ordinal: median and the distribution. Numerical: mean or median plus a measure of spread. Binary: the proportion.
• Ratios ("twice as much") only make sense when zero means none.
• Use category and ordered pd.Categorical dtypes so pandas sorts, compares and plots labels correctly.
• The make_orders() function above recreates the course dataset exactly; each later lesson starts with it.

Try it yourself

1. Add a column month = orders["order_date"].dt.month_name(). Is it nominal or ordinal? Make it an ordered categorical in calendar order and print value_counts(sort=False).
2. Compute the share of orders returned for each category with orders.groupby("category")["returned"].mean(). Which category has the highest return rate, and why is the mean the right statistic here?
3. Create age_band = pd.cut(orders["customer_age"], bins=[17, 30, 45, 80]). What type of variable have you created, and what information did you lose? Compare orders["customer_age"].median() with the most common band.