Pipelines

Why it matters

Without automation, releasing software depends on someone remembering every step: run the tests, build the package, copy it to the server, restart the service. People skip steps when they are busy, and that is when outages happen. A pipelineAutomated stages (test, build, deliver, verify) that run the same way on every change, each gated on the one before. turns those steps into code that runs the same way on every change.

Continuous integrationBuilding and testing every push and pull request automatically so problems show up within minutes. (CI) means every push and pull request is built and tested automatically, so problems show up within minutes, while the change is still fresh in the author's mind. Continuous deliveryKeeping every change that passes the pipeline packaged and ready to release. (CD) extends that: every change that passes is packaged and ready to release, and releasing is a routine, low-drama step. (Continuous deployment goes one step further and releases every passing change automatically.)

The core idea: stages that gate each other

A typical pipeline has stages, and each one runs only if the previous one passed:

1. Checkout the exact commit.
2. Test: unit tests, linting, type checks. Fast feedback first.
3. Build an artifact: a container image, tagged with the commit SHA.
4. Deliver or deploy that artifact, usually only from the main branch.
5. Verify: a smoke test against what was deployed.

A failure at any stage stops the run and marks the commit red, so a broken change never reaches users.

GitHub Actions vocabulary

GitHub Actions runs pipelines defined in YAML files under .github/workflows/.

Cost: standard GitHub-hosted runners are free for public repositories, and private repositories on the Free plan get a monthly quota of included minutes. Quotas change, so check GitHub's billing docs for Actions (checked on 2026-10-23) and watch the usage page in your account settings.

A worked example

Add a test for the greeting function from the Docker lesson:

# test_app.py
import unittest

from app import greeting


class GreetingTest(unittest.TestCase):
    def test_greeting(self):
        self.assertEqual(greeting("CI"), "Hello, CI!")


if __name__ == "__main__":
    unittest.main()

Then the workflow:

# .github/workflows/ci.yml
name: ci
on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python -m unittest -v

  build:
    needs: test
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4
      - name: Build image
        run: |
          OWNER="${GITHUB_REPOSITORY_OWNER,,}"
          echo "IMAGE=ghcr.io/$OWNER/hello-devops:${GITHUB_SHA}" >> "$GITHUB_ENV"
          docker build -t "ghcr.io/$OWNER/hello-devops:${GITHUB_SHA}" .
      - name: Push image (main only)
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
          echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
          docker push "$IMAGE"

  smoke-test:
    needs: build
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: read
    steps:
      - name: Run the published image and check it answers
        run: |
          OWNER="${GITHUB_REPOSITORY_OWNER,,}"
          IMAGE="ghcr.io/$OWNER/hello-devops:${GITHUB_SHA}"
          echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
          docker run -d --name app -p 8000:8000 "$IMAGE"
          curl --fail --retry 10 --retry-delay 1 --retry-all-errors http://localhost:8000

How it fits together:

• on runs the workflow for pull requests and for pushes to main.
• needs: test means the image is built only after tests pass; needs: build chains the smoke test.
• The image tag is the commit SHA, so you always know exactly which code an image contains.
• The if: conditions make pull requests run tests and a build, but only main publishes. That stops unreviewed code from being delivered.
• GITHUB_TOKEN is a short-lived token GitHub creates for every run. The permissions blocks give it the least access each job needs. No password is stored anywhere.
• GitHub Container Registry requires lowercase image names, hence ${GITHUB_REPOSITORY_OWNER,,}.

In a real system the last job would deploy the image to a cloud service (for example by updating a Cloud Run service or a Kubernetes Deployment) and then smoke-test the live URL. The structure stays the same.

Secrets in pipelines

• Never commit credentials, keys or .env files. Store them as repository or environment secrets; GitHub masks them in logs.
• Prefer short-lived credentials. GITHUB_TOKEN is one; for clouds, use OIDC federation so the workflow exchanges a GitHub-signed token for temporary cloud credentials instead of storing long-lived access keys.
• Pull requests from forks do not receive your secrets, by design. Be careful with the pull_request_target event, which does.
• Pin third-party actions to a version (@v4), or to a full commit SHA for stronger supply-chain safety.

Common mistakes

• Slow pipelines. If CI takes 40 minutes, people batch changes and stop trusting it. Run fast checks first and cache dependencies.
• Flaky tests ignored. A test that fails randomly trains everyone to click "re-run". Fix or quarantine it.
• Deploying from any branch. Gate delivery on main (and on review via branch protection).
• Rebuilding per environment. Build once, then promote the same image; the next lesson explains why.

Quick recap

• CI tests every change; CD keeps every passing change releasable.
• A workflow has events, jobs, steps and runners; needs orders jobs.
• Tag artifacts with the commit SHA and publish only from main.
• Use GITHUB_TOKEN, least-privilege permissions and secrets, never committed credentials.

Try it

The Do tab has you push the hello-devops project to GitHub and watch this pipeline go red and then green.