Compare building, buying and configuring a product, and score vendor options with a weighted matrix.
Why it matters
"Should we build it ourselves or buy a product?" is one of the most expensive decisions a project makes, and it is often made on gut feeling or on the best sales demo. Analysts bring structure: clear requirements, a fair comparison and a recorded rationale that someone can revisit in two years when the decision is questioned.
The options are wider than two
- Buy (SaaS or packaged software): use the product largely as it comes. Fast to start, the vendor handles upgrades, but you adapt your process to the product.
- Configure: buy a platform and set it up with settings, workflows and fields (most CRM and ERP projects).
- Customise / extend: add code on top of a product. Powerful, but every customisation can break at upgrade time.
- Build: write your own. Full control and fit, but you own every bug, upgrade and security patch forever.
- Hybrid: buy the commodity parts and build the parts that differentiate you.
A useful rule of thumb: buy what is commodity, build what is differentiating. Payroll, email and accounting work the same way for nearly everyone; buy them. A unique pricing engine that wins you customers may be worth building.
Total cost of ownership
Compare costs over several years, not just the first invoice:
Buy: licences/subscriptions x users x years
+ implementation partner + data migration + integrations
+ training + internal admin time
Build: development team x duration
+ hosting + security and compliance work
+ ongoing maintenance (often 15-25% of build cost per year)
+ the opportunity cost of what that team is not building
The weighted scoring matrix
- Agree the criteria before you see demos, from the requirements: functional fit, integration (APIs available), security and compliance, total cost, vendor viability, usability, time to deliver.
- Weight them so they sum to 100%. Stakeholders agree the weights; this is where priorities become explicit.
- Score each option 1 to 5 against each criterion, using evidence: scripted demos (give vendors your scenarios), reference calls, a trial or proof of concept.
- Multiply and sum. The highest total is the recommendation, unless an option fails a must-have gate, which disqualifies it whatever its score.
Worked example: Harbor Outfitters CRM
Must-have gates: stores customer consent; has a documented REST API; can import 180,000 legacy customers.
| Criterion | Weight | Vendor A (SaaS CRM) | Vendor B (CRM suite) | Build in-house |
|---|---|---|---|---|
| Functional fit (loyalty, segments) | 30% | 4 | 5 | 5 |
| Integration with NetSuite and Shopify | 20% | 5 | 3 | 3 |
| 5-year total cost | 20% | 4 | 2 | 2 |
| Time to go live | 15% | 4 | 3 | 1 |
| Usability for store staff | 15% | 4 | 3 | 4 |
Weighted totals (score x weight, summed):
Vendor A: 4(.30)+5(.20)+4(.20)+4(.15)+4(.15) = 1.2+1.0+0.8+0.6+0.6 = 4.20
Vendor B: 5(.30)+3(.20)+2(.20)+3(.15)+3(.15) = 1.5+0.6+0.4+0.45+0.45 = 3.40
Build: 5(.30)+3(.20)+2(.20)+1(.15)+4(.15) = 1.5+0.6+0.4+0.15+0.6 = 3.25
Vendor A wins clearly, even though Vendor B has the best features. The recommendation document records the scores, the evidence behind each, the risks (Vendor A's loyalty module is newer) and the mitigations (a 30-day proof of concept on loyalty before signing).
Beyond the score
Also check: data ownership and export (can you leave with your data?), where data is hosted, the vendor's security certifications (such as SOC 2 or ISO 27001), contract length and exit terms, and the product roadmap. Sensitivity test the result: if changing one weight by 5 points flips the winner, the decision is close and needs more evidence.
Common mistakes
- Choosing criteria after the demos, which bends them toward the favourite.
- Letting vendors run their own demo script instead of your scenarios.
- Comparing year-one licence price against a build estimate without maintenance.
- Heavy customisation of a bought product, which gives you the costs of both options.
- No recorded rationale.
Quick recap
- Options range from buy to configure, customise, build and hybrid; buy commodity, build differentiators.
- Compare total cost of ownership over several years.
- Use must-have gates plus a weighted matrix agreed before demos, scored with evidence.
Try it
The Do tab asks you to build a weighted matrix for a real decision.