Dictionaries and sets

Why it matters

Lists are great when position matters. Often, though, you want to find something by a name: the price of an item, the email for a user ID, the number of times each word appears. A dictionary (dict) stores key-value pairs and finds any value by its key almost instantly, however large it gets. A set stores unique values, which makes removing duplicates and comparing groups easy.

Creating and using a dictionary

prices = {"apple": 0.5, "bread": 2.25, "milk": 1.1}

print(prices["bread"])      # 2.25
prices["eggs"] = 3.0        # add a new key
prices["milk"] = 1.2        # update an existing key
del prices["apple"]         # remove a key
print(len(prices))          # 3
print("milk" in prices)     # True   (checks keys, not values)

Keys are usually strings or numbers, and each key appears only once. Values can be anything, including lists or other dictionaries.

Missing keys: KeyError and .get()

Looking up a key that does not exist raises an error:

print(prices["cheese"])          # KeyError: 'cheese'

.get() returns a default instead:

print(prices.get("cheese"))      # None
print(prices.get("cheese", 0))   # 0

Use [] when a missing key would be a bug you want to hear about, and .get() when missing is normal.

Looping over a dictionary

stock = {"bolts": 120, "nuts": 80, "washers": 0}

for name in stock:                 # loops over keys
    print(name)

for name, qty in stock.items():    # keys and values together
    if qty == 0:
        print(f"Reorder {name}")

print(list(stock.values()))        # [120, 80, 0]
print(sum(stock.values()))         # 200

Dictionaries remember the order in which keys were inserted.

Counting with a dictionary

Counting things is one of the most common jobs in data work. The pattern:

words = "the cat and the hat and the bat".split()

counts = {}
for word in words:
    counts[word] = counts.get(word, 0) + 1

print(counts)
# {'the': 3, 'cat': 1, 'and': 2, 'hat': 1, 'bat': 1}

counts.get(word, 0) is the current count, or 0 the first time a word is seen. The standard library also has collections.Counter, which does this in one line and adds handy methods:

from collections import Counter
counts = Counter(words)
print(counts.most_common(2))   # [('the', 3), ('and', 2)]

Learn the manual pattern first; it adapts to anything, such as summing amounts per category instead of counting.

Grouping with a dictionary of lists

readings = [("tank-a", 20.5), ("tank-b", 18.0), ("tank-a", 21.5)]

by_tank = {}
for tank, value in readings:
    if tank not in by_tank:
        by_tank[tank] = []
    by_tank[tank].append(value)

print(by_tank)   # {'tank-a': [20.5, 21.5], 'tank-b': [18.0]}

Sets: unique values

A set holds each value at most once, in no particular order:

tags = ["python", "data", "python", "beginner", "data"]
unique = set(tags)
print(unique)              # {'data', 'beginner', 'python'} (order may vary)
print(len(unique))         # 3
print(sorted(unique))      # ['beginner', 'data', 'python']

seen = set()
seen.add("sample-01")
print("sample-01" in seen) # True, and very fast even for millions of items

Sets support maths-style operations:

course_a = {"ada", "grace", "alan"}
course_b = {"grace", "katherine"}

print(course_a & course_b)   # {'grace'}                  in both (intersection)
print(course_a | course_b)   # all four names              in either (union)
print(course_a - course_b)   # {'ada', 'alan'}            in a but not b

Note that {} creates an empty dictionary. An empty set is set().

Choosing a structure

Common mistakes

• KeyError on a first count. counts[word] += 1 fails the first time. Use counts.get(word, 0) + 1.
• Checking values with in. "1.2" in prices checks keys. Use in prices.values() for values.
• Using a list as a key. Keys must be unchangeable (strings, numbers, tuples). A list raises TypeError: unhashable type.
• Expecting order from a set. Use sorted(my_set) when order matters.
• {} for an empty set. That is an empty dict.

Quick recap

• A dict maps keys to values: d[key], d[key] = value, del d[key].
• .get(key, default) avoids KeyError when missing is normal.
• Loop with .items() to get keys and values together.
• Count with counts[k] = counts.get(k, 0) + 1 or collections.Counter.
• Sets hold unique values and support &, | and -.

Try it

In the Do tab you will count words in a sentence, look up prices with a default, and find the unique tags in a list.