Iterating after launch

Why it matters

Launch is not the end of design. It is the first time you see your design used by real people, at real scale, with real data and real distractions. Some of your assumptions will be wrong. Teams that keep learning after launch improve the product steadily; teams that move straight to the next project leave known problems in place for years. Planning post-launch research is a normal part of a mid-level designer's job.

The core idea: build, measure, learn, repeat

Iterative design is a loop: ship a version, measure what happens, learn why, and improve. After launch, you have three kinds of evidence:

None is enough on its own. A funnel shows that 40% of people drop off at step 3; it does not say why. A usability session shows why, but not how common the problem is. Combine them: use analytics to find where to look, research to understand why, and analytics again to check whether the fix worked.

Planning a post-launch study

Write a short plan before launch (or as soon as possible after):

1. Questions. What do you need to learn? Start from the risks and assumptions you made during design. "Do people find 'Change amounts' without help?" "Are reminders seen as helpful or annoying?"
2. Metrics to watch, with the baseline and target you set (previous lesson), plus guardrails.
3. Methods and timing. For example:
   • Week 1: watch analytics daily for errors and big drop-offs; read every support ticket mentioning the feature.
   • Week 2: a short in-app survey after the first use (one or two questions).
   • Weeks 2 to 3: five moderated usability sessions with real users on the live product, focused on the riskiest task.
   • Week 4: review all evidence and decide on changes.
4. Participants. Who you will recruit and how (for example, an opt-in link in the app for people who used the feature).
5. Decision rules. What would make you change, roll back or keep the design. "If sent splits stay below 80% after four weeks, revisit the edit-amount step."
6. Sharing. How findings reach the team: a short readout, a prioritised list in the backlog.

Worked example: a post-launch plan

Feature: Bill split v2 ("Change amounts" + auto reminders). Launch: 3 March.
Questions:
  Q1 Do people find "Change amounts" without help?
  Q2 Do reminders reduce chasing, or annoy housemates?
Metrics: splits sent / started (baseline 71%, target 85%);
         reminder opt-outs per 100 houses (guardrail: below 10);
         30-day house retention (guardrail: not lower than 58%).
Methods:
  Wk 1   Daily funnel check; tag support tickets "split" and "reminder".
  Wk 2   One-question survey after first split: "How easy was it to split this bill?" (1-7)
  Wk 2-3 5 moderated sessions on the live app with opted-in users who split a bill
         in the last week; tasks: unequal split, turn off reminders.
  Wk 4   Synthesis workshop; decide changes.
Decision rules: below 80% sent at week 4 -> redesign edit step;
                opt-outs above 10 per 100 -> make reminders gentler or less frequent.
Privacy: only aggregated analytics; session participants give consent; no recordings
         kept beyond 90 days.

Prioritising what you learn

After a study you will have more problems than time. A simple way to prioritise is to rate each issue by impact (how many people, how badly) and effort (how hard to fix), then do high-impact, low-effort fixes first. Keep a visible list so issues are not forgotten, and repeat the loop.

Common mistakes

• No plan until something breaks. Decide what you will measure before launch.
• Only looking at numbers. You will see problems you cannot explain.
• Only reading reviews. People who write reviews are a loud, unrepresentative minority.
• Changing several things at once. You cannot tell which change helped.
• No decision rules. Without them, every result is "interesting" and nothing changes.
• Collecting more data than needed. Respect privacy and your organisation's data policies.

Quick recap

• Launch is the start of learning; iterate with a build, measure, learn loop.
• Combine analytics (what), feedback (what people say) and research (why).
• A post-launch plan lists questions, metrics, methods and timing, participants, decision rules and how findings are shared.
• Prioritise fixes by impact and effort, and keep the loop going.

Try it

On the Do tab you will write a post-launch study plan for a feature from your own project.