Facilitate live and virtual sessions, build onboarding programs and an LMS course, and prove what your training changed.

Training Delivery & Evaluation

## Why it matters

"Did the training work?" is the question every sponsor eventually asks. Most training is evaluated only with an end-of-session satisfaction form, which tells you whether people liked it, not whether it changed anything. Evaluation models give you a structure for gathering better evidence, and planning evaluation before delivery (backward design again) makes it far easier.

## Kirkpatrick's four levels

Donald Kirkpatrick introduced four levels of training evaluation in a series of articles in 1959. They remain the most widely used framework.

| Level | Question | Typical measures | Incident-response example |
|---|---|---|---|
| 1. Reaction | Did they find it engaging and relevant? | End-of-session survey | "How useful will this be on your next shift?" (1 to 5) |
| 2. Learning | Did they gain the knowledge, skills and confidence? | Tests, practical tasks, pre/post | Severity scenario test before and after; sandbox escalation task |
| 3. Behavior | Do they apply it on the job? | Observation, work samples, manager check-ins, system data | Severity accuracy on real incidents in the first 60 days |
| 4. Results | Did the targeted business outcomes improve? | Business metrics | Mean time to resolve customer-impacting incidents |

### The New World Kirkpatrick Model

James and Wendy Kirkpatrick updated the model (*Kirkpatrick's Four Levels of Training Evaluation*, 2016). Key changes:

- **Start planning at Level 4** and work backwards: what result does the business need, what behaviors drive it, what must people learn, and how should training feel?
- **Required drivers** at Level 3: processes and systems that reinforce, monitor and reward the new behavior on the job (manager coaching, job aids, follow-up). Without them, learning rarely turns into behavior.
- **Leading indicators** at Level 4: short-term signs that results are on track (for example, escalations reaching the right team first time).
- Level 1 adds **engagement** and **relevance**, and Level 2 adds **confidence** and **commitment**.

## Phillips ROI: a fifth level

Jack Phillips added **Level 5: Return on investment**, comparing monetary benefits with program costs:

```text
ROI (%) = (net program benefits / program costs) x 100
net program benefits = monetary benefits - program costs
```

Example: the onboarding program costs 40,000 (design time, trainer time, learners' time). Faster incident resolution is estimated to save 70,000 in the first year. Net benefits = 30,000, so ROI = 30,000 / 40,000 x 100 = 75%. The hard part is **isolating the effect of training** from everything else that changed (a new tool, a quieter quarter). Phillips recommends methods such as control groups, trend-line analysis and conservative estimates, and being explicit about assumptions.

## Criticisms and alternatives

- **The levels are not a chain.** George Alliger and colleagues' 1997 meta-analysis found that reaction measures were only weakly related to learning and transfer. People can enjoy training and learn little, or dislike challenging training that works. Do not use Level 1 as a proxy for Level 2 or 3.
- **Most effort stays at Level 1.** Surveys are easy; behavior and results take planning and stakeholder support.
- **Will Thalheimer's LTEM** (Learning-Transfer Evaluation Model, 2018) offers eight tiers, separating attendance and activity from knowledge recall, decision-making competence, task competence, transfer and effects of transfer, to discourage treating weak measures as strong evidence.
- **Robert Brinkerhoff's Success Case Method** studies the most and least successful participants in depth to find out what helped or blocked transfer, which is often more persuasive to stakeholders than averages.

## Planning a four-level evaluation

For the incident-response onboarding:

1. **Level 4 (start here):** target is to reduce mean time to resolve SEV2 incidents handled by new engineers; leading indicator: correct first escalation rate.
2. **Level 3:** severity accuracy on a weekly sample of real incidents for 60 days; required drivers: buddy reviews, matrix linked in the tool, manager check-in at day 30.
3. **Level 2:** pre/post scenario test and the sandbox escalation task; confidence rating.
4. **Level 1:** three-question survey (relevance, confidence, one improvement) at the end of week 1.
5. **Timing and owners:** who collects what, when, and where the data lives.
6. **Baseline:** collect the same measures for the previous cohort, or before launch.

Agree this plan with the sponsor *before* the program starts, including which results would count as success.

## Common mistakes

- Only surveying satisfaction.
- Measuring behavior with no baseline.
- Claiming all improvement was caused by training.
- Ignoring the work environment that supports or blocks new behavior.
- Choosing measures after the program, based on what looks good.

## Quick recap

- Kirkpatrick: reaction, learning, behavior, results.
- Plan from Level 4 backwards; build required drivers for Level 3.
- Phillips adds ROI; isolate the training's effect and state assumptions.
- Reaction does not predict learning; consider LTEM and success cases too.

## Try it

Answer the quiz on the Do tab, then draft the Level 4 and Level 3 measures for a training you know.


Plan an evaluation across reaction, learning, behavior and results, starting from the business outcome.

Kirkpatrick evaluation

The Kirkpatrick Model of Training Evaluation

The Kirkpatrick Evaluation Model in a Nutshell

## Why it matters

Training teams sit on a lot of data: enrollments, completions, quiz scores, survey answers, time spent. Stakeholders ask for it constantly ("What percentage of the support team has finished?"). Being able to pull and summarize that data yourself, and to explain what it does and does not show, makes you far more effective than waiting for a report someone else builds. You do not need to be a programmer: a few lines of SQL go a long way.

## Common training metrics

| Metric | What it tells you | Watch out for |
|---|---|---|
| Enrollments | Reach | Says nothing about learning |
| Completion rate | Share of enrolled learners who finished | Easy to game with "viewed" completion rules |
| Assessment score | Performance on the test at the end | Depends on test quality |
| Pre/post gain | Change from before to after | Needs the same or equivalent tests |
| Time to proficiency | How fast people reach the standard | Needs a clear standard |
| On-the-job measures | Behavior and results (Kirkpatrick 3 and 4) | Many other causes |

## Reading training data with SQL

Many LMSs let you export reports or query a reporting database. A typical export has one row per learner per course. Here is a small table called `enrollments`:

| learner | team | course | completed | pre_score | post_score |
|---|---|---|---|---|---|
| Ana | Support | Incident basics | 1 | 55 | 85 |
| Ben | Support | Incident basics | 0 | 60 | (empty) |

`completed` is 1 for finished and 0 for not finished. `post_score` is empty (NULL) for people who have not finished.

Four SQL ideas cover most training reports:

```sql
SELECT course,                                  -- 1. choose columns
       COUNT(*) AS enrolled,                    -- 2. count rows in each group
       ROUND(100.0 * SUM(completed) / COUNT(*), 1) AS completion_pct
FROM enrollments
GROUP BY course                                 -- 3. one row per course
ORDER BY completion_pct DESC;                   -- 4. sort the result
```

- `COUNT(*)` counts learners in each course.
- `SUM(completed)` adds up the 1s, which is the number who finished. Multiplying by `100.0` (not `100`) makes the division a decimal rather than a whole number.
- `AVG(...)` gives an average and **ignores empty (NULL) values**. That means `AVG(post_score - pre_score)` is the average gain of the people who finished, because the others have no post score.
- `WHERE team = 'Support'` would filter to one team before grouping.

## Interpreting the numbers

Suppose the query shows:

| course | enrolled | completion_pct | avg_gain |
|---|---|---|---|
| Escalation paths | 5 | 80.0 | 22.5 |
| Incident basics | 6 | 66.7 | 26.3 |
| Writing summaries | 5 | 40.0 | 7.5 |

Questions a good trainer asks:

- **Why is "Writing summaries" completion so low?** Is it at a busy time, too long, or broken? Look at where people stop.
- **Is the small gain a problem with the course or the test?** A pre-test that is already high leaves little room to improve (a ceiling effect).
- **Survivorship:** average gain covers only completers. If the people who struggled dropped out, the gain looks better than it really is.
- **Small numbers:** with five learners, one person changes the completion rate by 20 points. Report the counts alongside the percentages.
- **Completion is not competence.** A high completion rate with a weak test tells you little; connect to Level 3 and 4 measures where you can.

## Reporting to stakeholders

- Lead with the question the stakeholder cares about ("Are new engineers ready for the rotation?"), not with every metric.
- Show counts and percentages together.
- Compare with a baseline or target.
- Say what the data cannot show, and what you will do next.
- Use simple charts: a bar chart of completion by course beats a pie chart or a dense table.

## Privacy and ethics

Learning data is personal data. Share individual results only with people who need them, aggregate where possible, follow your organization's privacy rules (and laws such as GDPR), and be careful about using training scores in performance reviews without telling learners in advance. People answer practice quizzes honestly only when it is safe to be wrong.

## Common mistakes

- Reporting completion as if it were learning.
- Dividing whole numbers and getting 0 instead of 66.7.
- Ignoring that averages skip empty values.
- Percentages from tiny groups with no counts.
- Sharing named results widely.

## Quick recap

- Know what each metric shows and its trap.
- `COUNT`, `SUM`, `AVG`, `GROUP BY` and `ORDER BY` answer most training questions.
- `AVG` ignores NULL, which affects who is included.
- Report counts with percentages, compare with a baseline, protect learner data.

## Try it

On the Do tab, finish a query that reports completion and average score gain per course.


Analyze completion and score data with a short SQL query and report it honestly.

Training data and reporting

Training evaluation | How to measure learning impact [Tutorial isEazy]

Measuring Learning Return on Investment (ROI)

## Why it matters

Technical trainers are hired mainly for two things: real expertise in what they teach and evidence that they can teach it. Credentials can help with the second, but the landscape is confusing: association certifications, vendor trainer programs, degrees, and certificates of attendance all look similar on a CV. This lesson helps you decide what is worth your time. Requirements and fees change often, so always check the official pages linked here. Checked on 2026-10-23.

## What employers actually look for

For most technical trainer roles (O*NET groups them with training and development specialists, 13-1151.00), employers ask for:

1. **Subject expertise**, often shown by working experience and by the technical certifications learners themselves are taking (a cloud trainer usually holds the cloud certifications).
2. **Evidence of teaching ability**: a portfolio, a recorded teach-back, references, and sample materials.
3. A bachelor's degree is common but not universal; a master's in instructional design, learning sciences or adult education helps for design-heavy and leadership roles.
4. Credentials, as a tiebreaker or requirement for some partner programs.

A **portfolio** is often more persuasive than any certificate: a short recorded lesson, a needs analysis summary, an objectives-and-assessment plan, a rubric and an evaluation report. Several activities in this course and the Instructional Design course produce exactly these.

## Association credentials: ATD

The **Association for Talent Development (ATD)** runs a certification body, the ATD Certification Institute, with two credentials:

- **APTD (Associate Professional in Talent Development):** for earlier-career practitioners. ATD has listed eligibility as about three years of professional work experience plus 28 hours of talent development professional development in the last three years.
- **CPTD (Certified Professional in Talent Development):** for experienced practitioners. ATD has listed about five years of experience plus 60 hours of professional development in the last five years, with alternative routes for APTD holders and ATD Master Series graduates.

Both are exam-based, cover a broad talent development capability model (instructional design, training delivery, evaluation, and more), and require recertification. They signal breadth across the L&D field rather than expertise in one technology. Check the current handbook at [td.org/certification](https://www.td.org/certification).

ATD also offers shorter **certificate programs** (for example, training and facilitation certificates). These are certificates of course completion, which are different from certifications with an independent exam.

## Vendor trainer programs

Technology vendors certify instructors who deliver their official courses through training partners. Typical requirements are a current technical certification in the product, proof of instructional skills, and an annual renewal.

- **Microsoft Certified Trainer (MCT):** Microsoft's enrollment page lists agreeing to the program agreement, holding a current qualifying Microsoft role-based (Associate or Expert) certification, and providing verified proof of instructional skills; membership is renewed annually. See [learn.microsoft.com/credentials/certifications/mct-enrollment](https://learn.microsoft.com/credentials/certifications/mct-enrollment).
- **AWS, Google Cloud, Cisco, Salesforce** and other vendors run their own authorized instructor programs, usually tied to working for or with an authorized training partner. Check each vendor's training site.

These matter most if you want to teach official vendor courses, or if your employer is a training partner.

## A retired credential: CompTIA CTT+

You will still see **CompTIA CTT+** (Certified Technical Trainer) in job ads and older advice. CompTIA retired it in 2023 and stopped selling exam vouchers that year. People who earned it remain certified for life, but it is no longer available to new candidates, and vendor programs that once accepted it as proof of teaching skills have moved to other options. Check [CompTIA's list of retired certifications](https://help.comptia.org/hc/en-us/articles/10830728209300-Which-CompTIA-Certifications-Have-Been-Retired) if you see it required.

## Comparing options

| Option | Best for | Signals | Typical cost in time |
|---|---|---|---|
| Portfolio | Everyone | Actual teaching and design skill | Ongoing; build as you go |
| Technical certifications in your subject | Everyone teaching a technology | Subject credibility | Weeks to months each |
| Vendor trainer program (e.g. MCT) | Teaching official vendor courses | Authorized to deliver vendor content | Requires the technical cert first, plus a skills check |
| ATD APTD | Early-career L&D generalists | Breadth of talent development knowledge | Months of study |
| ATD CPTD | Experienced L&D professionals moving into senior roles | Breadth and experience | Months of study; experience required |
| Master's degree | Instructional design, L&D leadership, academic roles | Deep theory and research | One to two years |

## Building a plan

1. Decide your direction: product or technology trainer, L&D generalist, or instructional designer.
2. Keep your technical certifications current; they are your credibility with learners.
3. Build a portfolio from real work, with confidential details removed.
4. Add one teaching credential that matches your direction: a vendor program if you teach that vendor's courses, ATD if you are moving towards L&D roles.
5. Keep learning in public: present at a meetup, write up an evaluation you ran.

## Common mistakes

- Collecting certificates instead of building a portfolio.
- Treating a certificate of attendance as an industry certification.
- Pursuing a retired credential because an old job ad mentioned it.
- Copying fees and requirements from blogs instead of the official handbook.

## Quick recap

- Subject expertise and a teaching portfolio come first.
- ATD's APTD and CPTD are broad talent development certifications with experience requirements.
- Vendor programs such as MCT require current technical certification plus proof of teaching skills.
- CompTIA CTT+ was retired in 2023; check official pages, as requirements change.

## Try it

Answer the quiz on the Do tab, then list the three pieces of evidence you would put in your trainer portfolio first.


Compare ATD's APTD and CPTD, vendor trainer programs and portfolios, and pick what fits your direction.

Trainer credentials

Keys to ATD CI’s Certification Eligibility

How to Become A Microsoft Certified Trainer (MCT)

Plan four-level evaluation, analyze completion and score data, and choose the trainer credentials worth your time.