Turn a vague analysis request into specific, answerable questions with defined measures and scope.
Why it matters
Analysis requests rarely arrive as clear questions. They sound like "Can you look into sales?", "Why are the results worse lately?" or "Is the new process working?" If you start coding straight away, you will produce lots of numbers and no answer. The most valuable skill in this module is turning a vague request into a few specific, answerable questions before touching the data.
What makes a question good
A good analysis question is:
- Specific about the measure. "Revenue", "on-time rate", "mean yield", not "performance".
- Specific about the population and time. Which customers, sites or samples, over which period?
- Comparative. Against what? Last year, a target, another group, before and after a change.
- Answerable with data you have, or can get.
- Tied to a decision. If the answer were X, what would someone do differently?
| Vague | Specific |
|---|---|
| How are sales doing? | Did Q3 revenue per region change compared with Q3 last year, and which region changed most? |
| Is the new process working? | Is mean batch yield on line L2 higher in the 8 weeks after the October change than in the 8 weeks before? |
| Why are deliveries slow? | Which cities and hours have a median delivery time over 40 minutes in September, and what share of orders do they cover? |
A method: from request to questions
1. Ask about the decision. "What will you do with the answer?" or "What prompted this?" The reply often narrows the work by half. A manager who asks "how are sales doing" may really be deciding where to put two new staff.
2. Name the stakeholder and the deadline. Who reads the result, how technical are they, and when do they need it? A rough answer on Friday may beat a perfect one next month.
3. Define the measures precisely. Write the formula: "on-time = delivered within 30 minutes of the promised time; rate = on-time orders / all delivered orders, excluding cancellations". Ambiguous definitions are the most common source of arguments about results.
4. Write 2 to 4 questions, each in the specific form above. Order them: the headline question first, then the ones that explain it.
5. Check the data exists. For each question list the table, the columns, the date range and any known problems. If a question needs data you do not have, say so early.
6. Write down assumptions and what is out of scope. "Refunds excluded; online orders only; no causal claim, this is a before/after comparison."
7. Play it back. Send the questions and definitions to the requester in a few lines and ask "Is this what you need?" Ten minutes here saves days of rework.
Association is not causation
Many requests contain a hidden causal question: "Did the campaign increase sales?" A before/after comparison or a correlation can show that two things moved together, not that one caused the other. Seasonality, a price change, or a different customer mix could explain the same pattern. Be honest in how you phrase the question ("Did sales differ...?") unless you have a proper experiment, which the regression-experimental-design course covers.
A worked example
Request from an operations manager: "Deliveries feel slow lately. Can you look into it?"
After a five-minute conversation:
- Decision: whether to add drivers in some cities for the evening peak.
- Stakeholder: operations manager, needs a one-page summary by Thursday.
- Measure: delivery time = delivered timestamp minus order timestamp, in minutes; use the median because a few very late orders skew the mean.
- Questions:
- Has the median delivery time in September changed compared with July and August?
- Which city and hour combinations have a median over 40 minutes in September?
- What share of September orders fall in those slow combinations?
- Data:
orders.csv(order and delivered times, city), July to September. Known issue: about 2% of rows lack a delivered time. - Out of scope: driver-level performance; causes (traffic, weather) unless obvious.
That is a plan someone can say yes or no to, and an analysis you can finish.
Common mistakes
- Starting with the data. Exploring without a question is fine for 30 minutes, not for a week.
- Undefined measures. "Active user" or "late" without a formula leads to two analysts getting two answers.
- Too many questions. Four focused questions beat fifteen shallow ones.
- Promising causes from observational data.
- Not playing the questions back to the person who asked.
Quick recap
- Ask what decision the analysis supports, who reads it and by when.
- Define each measure as a formula, including exclusions.
- Write 2 to 4 specific, comparative, answerable questions and check the data exists.
- Record assumptions and scope, then confirm with the requester.
Try it
The Do tab gives you a vague request and walks you through turning it into a one-page analysis plan.