Learn to program in Python from zero: values, decisions, loops, functions, data structures, files, modules and a small data project.

Python Fundamentals

## Why it matters

This lesson pulls the whole course together. Turning raw text records into a clear summary is a task you will meet in almost every technical job: an engineer summarising sensor logs, a lab technician checking instrument output, a teacher totalling grades, a data analyst cleaning an export. The same five steps appear every time: **read, parse, validate, group, report**.

## The problem

A building's sensors write one line per reading:

```text
# date,sensor,value
2026-10-01,temp-a,21.5
2026-10-01,temp-b,19.0
2026-10-02,temp-a,22.5

2026-10-02,temp-b,n/a
2026-10-03,temp-a,20.0
2026-10-03,humid,55
garbage line
```

Real logs are messy. This one has a comment line, a blank line, a reading of `n/a` and a broken line. We want a report like:

```text
humid: n=1 min=55.0 max=55.0 mean=55.0
temp-a: n=3 min=20.0 max=22.5 mean=21.3
temp-b: n=1 min=19.0 max=19.0 mean=19.0
```

The same design works for grades: replace "sensor" with "student" or "assignment" and "value" with "score".

## Step 1: parse one line

Start with the smallest piece. A function that takes one line and returns a clean record, or `None` if the line is not usable:

```python
def parse_line(line):
    line = line.strip()
    if not line or line.startswith("#"):
        return None
    parts = line.split(",")
    if len(parts) != 3:
        return None
    day, sensor, text = parts
    try:
        value = float(text)
    except ValueError:
        return None
    return (day, sensor, value)
```

Every way a line can be bad is handled explicitly: blank, comment, wrong number of fields, value that is not a number. Returning `None` for "skip this" keeps the decision in one place.

Test it before moving on:

```python
assert parse_line("2026-10-01,temp-a,21.5") == ("2026-10-01", "temp-a", 21.5)
assert parse_line("# header") is None
assert parse_line("2026-10-02,temp-b,n/a") is None
assert parse_line("garbage line") is None
```

## Step 2: load every line

```python
def load_records(text):
    records = []
    for line in text.splitlines():
        record = parse_line(line)
        if record is not None:
            records.append(record)
    return records
```

`text.splitlines()` splits a block of text into lines. If the data were in a file, you would loop over the file object instead, exactly as in the files lesson. Keeping `load_records` working on a string makes it easy to test, and a one-line wrapper can read a file into that string.

## Step 3: group by sensor

This is the dictionary-of-lists pattern from the dictionaries lesson:

```python
def group_by_sensor(records):
    groups = {}
    for day, sensor, value in records:
        groups.setdefault(sensor, []).append(value)
    return groups
```

`groups.setdefault(sensor, [])` returns the existing list for that sensor, or creates an empty one first. It is a compact version of the `if sensor not in groups:` check.

## Step 4: summarise each group

```python
import statistics

def summarise(records):
    summary = {}
    for sensor, values in group_by_sensor(records).items():
        summary[sensor] = {
            "count": len(values),
            "min": min(values),
            "max": max(values),
            "mean": statistics.mean(values),
        }
    return summary
```

The result is a dictionary of dictionaries, easy to test and easy to turn into any output format.

## Step 5: format the report

```python
def format_report(summary):
    lines = []
    for sensor in sorted(summary):
        s = summary[sensor]
        lines.append(
            f"{sensor}: n={s['count']} min={s['min']:.1f} "
            f"max={s['max']:.1f} mean={s['mean']:.1f}"
        )
    return "\n".join(lines)
```

Sorting the sensor names makes the output predictable, which matters for testing and for readers comparing two reports. `"\n".join(lines)` glues the lines together with newlines between them.

## Putting it together

```python
report = format_report(summarise(load_records(LOG_TEXT)))
print(report)
```

Each function does one job, and the final line reads almost like the plan: load, summarise, format. When something is wrong, you can test each stage separately to see where the data stops looking right.

## Extending it

Once the core works, good next steps are:

- **Count what was skipped.** Report "3 lines skipped" so bad data does not vanish silently.
- **Flag out-of-range values.** Reuse your `reading_status` function from the conditions lesson.
- **Write the report to a file** with `with open("report.txt", "w")`.
- **Group by day as well as sensor**, using a tuple `(day, sensor)` as the dictionary key.
- **Grades version:** a CSV of `student,assignment,score`, with a letter grade per student from their mean.

## Common mistakes

- **Writing everything in one big loop.** It works for the first version, then becomes impossible to change. Split it into functions.
- **Silently dropping data.** Skipping bad lines is right, but count them so you know how much was lost.
- **Testing only the happy path.** Include a blank line, a comment, a non-number and a short line in your test data.
- **Unpredictable output order.** Sort keys before printing if a person or a test will compare the output.

## Quick recap

- Read, parse, validate, group, report: the shape of most data-summary programs.
- Parse one line in a function that returns a record or `None`.
- Group with a dictionary of lists, then summarise each group.
- Format the report last, in a predictable order.
- Test each function on its own with small, deliberately messy inputs.

## Try it

The Do tab gives you the log above and the five function names. Implement them so the checks pass, then print the report.


Parse messy text records, skip bad lines, group by key and report summary statistics.

Build a grade or sensor-log analyser

Python Tutorial: Real World Example - Parsing Names From a CSV to an HTML List

Write the problem in one or two specific sentences: who uses the program, what they give it and what they get back.

Write a small example input by hand (4 to 8 rows or lines), in the exact format the program will read.

Work out the expected output for that input by hand, deciding the details: decimals, sort order, labels.

List at least three edge cases (for example an empty file, a missing value, a single record) and write what the program should do for each.

List the functions you need, one line each: name, inputs, what it returns. Aim for 3 to 6 functions that each do one job.

Write pseudocode for the main flow and for the trickiest function. Break down any step that still feels vague.

Write at least four assert-style test cases from your example and edge cases, such as `assert parse_score("n/a") is None`.

Write a short "not in version 1" list of features you are deliberately leaving out.

## Why it matters

The most common reason a beginner project stalls is not a hard bug. It is starting to type before knowing what the program should do. Ten minutes of planning on paper saves hours of rewriting, and the plan doubles as a checklist that tells you when you are finished. Professional developers plan too; they just call it requirements, design and test cases.

## Step 1: write the problem in one or two sentences

Be specific about who uses it and what they get:

> A lab technician has a CSV of sample masses from three balances. The program should report the mean and range for each balance and flag any sample more than 0.5 g from its balance's mean.

If you cannot write this sentence, you are not ready to code. Vague goals ("analyse the data") lead to programs that never feel done.

## Step 2: pin down inputs and outputs

Write out a tiny, concrete example of each, by hand:

**Input** (`masses.csv`):

```text
balance,sample,mass_g
B1,S-01,12.40
B1,S-02,11.80
B2,S-03,12.10
B1,S-04,13.10
```

**Output:**

```text
B1: n=3 mean=12.43 range=11.80-13.10
B2: n=1 mean=12.10 range=12.10-12.10
FLAG B1 S-02 11.80 (0.63 from mean)
FLAG B1 S-04 13.10 (0.67 from mean)
```

Working out the expected output by hand is the single most valuable planning step. It forces you to decide the details (how many decimals? sorted how? what counts as "more than 0.5 g"?), and it becomes your first test.

## Step 3: list the edge cases

Ask "what could be weird about the input?":

- An empty file, or a file with only a header.
- A mass that is blank or `n/a`.
- A balance with only one sample (range is zero; can a sample be flagged?).
- Extra spaces or different capitalisation (`b1` vs `B1`).

Decide what the program should do for each. "Skip and count it" or "stop with a clear message" are both fine; pick one deliberately.

## Step 4: break it into functions

Each function should do one job you can describe in a sentence, with clear inputs and outputs. Write the signatures before any bodies:

```python
def read_rows(path):            # -> list of dicts from the CSV
def parse_mass(text):           # -> float, or None if unusable
def group_by_balance(rows):     # -> {balance: [(sample, mass), ...]}
def summarise(groups):          # -> {balance: {"n", "mean", "min", "max"}}
def find_flags(groups, summary, limit=0.5):  # -> list of (balance, sample, mass, diff)
def format_report(summary, flags):           # -> str
```

Notice the pattern from the previous lesson: read, parse, group, summarise, report. Most small data programs fit it.

## Step 5: write pseudocode

**Pseudocode** is the plan written in plain language, structured like code but without worrying about syntax:

```text
for each row in the file:
    mass = parse the mass text
    if mass is unusable:
        add 1 to the skipped count
        skip to the next row
    add (sample, mass) to that balance's list

for each balance:
    mean = average of its masses
    for each sample in the balance:
        if the distance from mean is more than 0.5:
            add it to the flags list
```

If a step in your pseudocode is fuzzy ("work out the flags somehow"), that is where you will get stuck while coding. Break it down further now.

## Step 6: write the tests first

Turn your hand-worked example into checks before writing the functions:

```python
assert parse_mass("12.40") == 12.4
assert parse_mass("n/a") is None
assert parse_mass("") is None

groups = {"B1": [("S-01", 12.4), ("S-02", 11.8), ("S-04", 13.1)]}
summary = summarise(groups)
assert round(summary["B1"]["mean"], 2) == 12.43
```

These fail at first, because the functions do not exist yet. That is the point: each one you make pass is visible progress, and when they all pass, you are done. This habit is the core of **test-driven development**, and it is exactly how the Do tab in this course works.

## Step 7: build one function at a time

Implement the smallest function first (`parse_mass`), run its tests, then move to the next. Resist writing everything and running it once at the end; when that fails, you will not know which part is wrong. Commit or save a copy each time a function works.

## Scope: what is in version 1?

Write down what you are **not** doing yet: no charts, no command-line options, no Excel files. A small program that works and is shared beats a big one that is half finished. Put extra ideas in a "later" list in your README.

## Common mistakes

- **Coding before you can state the problem.** Write the sentence first.
- **Skipping the hand-worked example.** Without it you have nothing to test against.
- **Giant functions.** If you cannot describe a function in one sentence, split it.
- **Ignoring edge cases until they crash the program.** List them in the plan.
- **Scope creep.** Finish version 1 before adding features.

## Quick recap

- State the problem in one or two specific sentences.
- Work out a small input and its expected output by hand.
- List the edge cases and decide how to handle each.
- Break the work into small functions with clear signatures, then write pseudocode.
- Write tests from your example first, then build and test one function at a time.

## Try it

In the Do tab you will plan a small program of your own (or the grades analyser) on paper or in a text file, following these steps.


Turn a vague idea into inputs, outputs, functions, pseudocode and test cases before writing code.

Planning a small program

What is pseudocode and how do you use it?

How I Plan My Coding Projects - 9 Steps

Copy your script into a file with a clear name (for example `analyse.py`), add a one-line comment or docstring at the top saying what it does, and wrap the main code in `if __name__ == "__main__":`.

Check every file you plan to share: remove passwords, API keys, real names and any data you do not own. Replace real data with a small made-up sample file.

Run the script once more from a fresh terminal (or the Do tab) and copy its exact output.

Write a README.md with these sections: what it does, requirements, how to run it, example output (pasted from the previous step) and input format.

Create a free GitHub account if you do not have one, then create a new public repository with a short lowercase name.

Upload your script, sample data and README.md (Add file > Upload files), and open the repository page to check that the README renders correctly.

Optional: add a LICENSE file from GitHub's templates (MIT is a common choice for small projects) and a `.gitignore` containing `__pycache__/` and `.venv/`.

Ask one person (a friend, classmate or a forum post) to follow your README from scratch, and note anything they found unclear.

## Why it matters

Code that only lives on your laptop helps nobody else, including your future self and future employers. Publishing a small project with a clear README shows that you can finish something, explain it and make it usable. For career changers in particular, a few small, well-documented projects on GitHub are often more convincing than a certificate, because a reviewer can see exactly how you work.

## What "shareable" means

A shareable project lets a stranger answer four questions in under two minutes:

1. What does this do?
2. How do I run it?
3. What does the output look like?
4. What do I need installed?

The **README** file answers them. Everything else (clean code, tests, a licence) builds trust.

## Tidy the code first

Before sharing, spend fifteen minutes on:

- **Names.** Replace `x`, `data2` and `temp_final_v3` with names that explain themselves.
- **A short comment or docstring at the top** of the file saying what it does.
- **A main guard,** so the file can be imported without running everything:

```python
def main():
    report = format_report(summarise(load_records(read_text("sensors.csv"))))
    print(report)


if __name__ == "__main__":
    main()
```

- **No secrets or personal data.** Remove passwords, API keys, real names, student records, patient data and anything your employer or school owns. Use a small made-up sample file instead. Once something is pushed to a public repository, assume it has been copied, even if you delete it later.
- **Run it one last time** from a fresh terminal, following your own README instructions exactly.

## A README that works

README files are usually written in **Markdown** (`README.md`), which GitHub renders as a formatted page. A good structure for a small project:

````markdown
# Sensor log analyser

Summarises a CSV log of sensor readings: count, min, max and mean per sensor.
Blank lines, comments and non-numeric values are skipped and counted.

## Requirements

- Python 3.10 or newer
- No third-party packages (standard library only)

## Usage

```bash
python3 analyse.py sample_data/sensors.csv
```

## Example output

```text
humid: n=1 min=55.0 max=55.0 mean=55.0
temp-a: n=3 min=20.0 max=22.5 mean=21.3
2 lines skipped
```

## Input format

One reading per line: `date,sensor,value`, for example `2026-10-01,temp-a,21.5`.

## Running the tests

```bash
python3 test_analyse.py
```

## Ideas for later

- Group by day as well as by sensor
- Write the report to a file
````

Keep it short and concrete. Example output is the part readers value most, so never skip it.

## Putting it on GitHub

GitHub is the most common place to share code. The basic flow in the browser, with no command line needed:

1. Create a free account at github.com.
2. Click **New repository**, give it a short lowercase name such as `sensor-log-analyser`, choose **Public**, and tick **Add a README file**.
3. Use **Add file > Upload files** to upload your `.py` file and the sample data.
4. Edit `README.md` in the browser and paste in your README.
5. Optionally add a licence (**Add file > Create new file**, name it `LICENSE`, and choose a template such as MIT) so others know they may reuse the code.

The Git & the Command Line course teaches the professional way to do this with `git add`, `git commit` and `git push`, which keeps a full history of every change. The browser upload is fine for a first project.

## Other files worth adding

- **`requirements.txt`** lists third-party packages (one per line, such as `pandas==2.2.3`) so others can run `pip install -r requirements.txt`. If you use only the standard library, say so in the README instead.
- **`.gitignore`** lists files Git should never upload, such as `__pycache__/`, `.venv/` and any real data files.
- **A test file,** even a few `assert` lines, shows that you check your work.

## Asking for feedback

Sharing is also how you improve. Post the repository link in a forum (such as this site's forums) or ask a friend to follow your README from scratch. Ask specific questions: "Is the README enough to run it?" or "Is there a simpler way to write `summarise`?" When someone reports a problem, thank them and fix the README or the code: if they were confused, the next reader will be too.

## Common mistakes

- **No README, or a one-line README.** Most visitors leave without trying the code.
- **Missing run instructions or example output.**
- **Committing secrets or private data.** Check every file before uploading.
- **Sharing code that only works on your machine,** for example with a hard-coded path like `C:\Users\me\Desktop\data.csv`. Use relative paths or a command-line argument.
- **Waiting until it is perfect.** Share version 1; improve it in public.

## Quick recap

- A README answers: what it does, how to run it, example output, requirements.
- Tidy names, add a main guard and remove secrets before sharing.
- A public GitHub repository with a README, the script, sample data and optionally a licence is a complete first project.
- `requirements.txt` and `.gitignore` help others run your code safely.
- Ask for specific feedback and treat confusion as a README bug.

## Try it

In the Do tab you will publish your mini project (or any small script) in a public GitHub repository with a complete README.


Publish a script on GitHub with a clear README so others can understand, run and reuse it.

Sharing your code

How to create your first GitHub repository: A beginner's guide | Tutorial

Getting started with Markdown on GitHub

Combine everything into a sensor-log analyser, learn to plan a small program before coding it, and share it on GitHub with a README.