Lists, dicts, and comprehensions

Why it matters

Before you reach for pandas, it pays to be fluent with plain Python lists and dictionaries. Data arrives as JSON from APIs, as rows from the csv module, or as small lookup tables you build yourself. All of these are lists of dictionaries. Knowing how to reshape them quickly, with comprehensions, makes you faster and helps you understand what pandas does for you later.

Records: a list of dictionaries

A common shape for tabular data in plain Python is one dictionary per row:

orders = [
    {"id": 1, "region": "East", "product": "Kettle", "units": 3, "price": 25.0},
    {"id": 2, "region": "West", "product": "Toaster", "units": 1, "price": 40.0},
    {"id": 3, "region": "East", "product": "Toaster", "units": 2, "price": 40.0},
    {"id": 4, "region": "North", "product": "Kettle", "units": 5, "price": 25.0},
]

Each dictionary is a recordOne row of tabular data stored as a dictionary, with keys playing the role of column names., and the keys play the role of column names. You read a field with square brackets: orders[0]["region"] is "East". Use .get(key, default) when a field might be missing: row.get("discount", 0).

List comprehensions

A list comprehensionA single expression that builds a new list from an existing one, optionally filtering with a trailing if. builds a new list from an existing one in a single expression:

revenues = [o["units"] * o["price"] for o in orders]
# [75.0, 40.0, 80.0, 125.0]

Read it left to right as "the value units * price, for each o in orders". It is equivalent to:

revenues = []
for o in orders:
    revenues.append(o["units"] * o["price"])

Add an if at the end to filter:

east_ids = [o["id"] for o in orders if o["region"] == "East"]
# [1, 3]

And use a conditional expressionA value choice written as x if condition else y, placed at the front of a comprehension to transform values. at the front to transform values differently:

sizes = ["large" if o["units"] >= 3 else "small" for o in orders]
# ['large', 'small', 'small', 'large']

Note the position: a filter if goes at the end and has no else; a value choice x if cond else y goes at the front.

Dictionary and set comprehensions

The same idea builds dictionaries, which is ideal for lookups:

price_by_id = {o["id"]: o["price"] for o in orders}
# {1: 25.0, 2: 40.0, 3: 40.0, 4: 25.0}

price_by_id[3]   # 40.0, found instantly without scanning the list

A set comprehension gives the distinct values:

regions = {o["region"] for o in orders}
# {'East', 'West', 'North'}  (sets have no fixed order)
sorted(regions)  # ['East', 'North', 'West']

Grouping and totals

Comprehensions are great for one-to-one reshaping. For totals by group, a plain loop with a dictionary is clearer:

units_by_region = {}
for o in orders:
    region = o["region"]
    units_by_region[region] = units_by_region.get(region, 0) + o["units"]

# {'East': 5, 'West': 1, 'North': 5}

The .get(region, 0) handles the first time a region is seen. The standard library's collections.Counter and defaultdict do the same with less code:

from collections import defaultdict

totals = defaultdict(float)
for o in orders:
    totals[o["product"]] += o["units"] * o["price"]
# {'Kettle': 200.0, 'Toaster': 120.0}

This "group, then aggregate" pattern is exactly what pandas' groupby does, which you will meet in module 3.

Sorting records

sorted takes a key function that says what to sort by:

by_units = sorted(orders, key=lambda o: o["units"], reverse=True)
[o["id"] for o in by_units]   # [4, 1, 3, 2]

Common mistakes

• Mutating records you did not mean to change. o["units"] = 0 inside a loop changes the original data. Build new dictionaries instead: {**o, "units": 0}.
• Huge, nested comprehensions. If a comprehension needs two fors and three conditions, write a loop. Clarity beats cleverness.
• KeyError on missing fields. Real JSON often omits keys. Use row.get("field") and decide what a missing value means.
• Relying on set order. Sets are unordered; wrap them in sorted() before showing them.
• Comparing floats with ==. 0.1 * 3 == 0.3 is False. Use math.isclose or round when checking money or measurements.

Quick recap

• A list of dictionaries is plain Python's table: one dictionary per record.
• [expr for x in items if cond] transforms and filters in one line.
• {key: value for x in items} builds fast lookups; {x for ...} gives distinct values.
• For totals by group, loop with dict.get or use defaultdict / Counter.

Try it

In the Do tab you will write three small functions over a list of order records: revenue per order, products from large orders, and total units per region.