How to Build a Software Shortlist

Updated September 2026.

Most software buying guides start at the scorecard: how to score three finalists against a weighted rubric. That's useful, but it skips the harder problem. How did you get to three finalists, out of the thirty or forty vendors that show up when you search a crowded category like CRM, help desk, or project management?

That earlier step rarely gets documented. Teams either shortlist by whoever the loudest person already likes, or grab the top three logos from the first comparison article Google shows them. Both feel decisive and are actually unexamined. This guide is the missing step: how to go from "we need software for this" to a defensible list of three, before you score anything.

Key Facts

• 6sense's 2025 B2B Buyer Experience Report found buyers evaluate an average of 5.1 vendors and fill about 3.6 shortlist spots on "Day One," mostly with vendors they already know. Skip a deliberate search and brand recognition fills the list for you.
• Forrester's B2B Vital Signs research puts the average number of interactions across a buying group at 27 per purchase cycle, up from 17 in 2019, one reason an undisciplined process burns more collective time than people expect.
• G2's own documentation confirms paid "Sponsored" listings occupy the third position on category pages, separate from the organic Grid ranking in its scoring methodology. What loads first isn't automatically what scored best.

Write the requirements brief before you look at a single vendor

The single biggest predictor of a bad shortlist is skipping straight to vendor names. Someone says "we need a help desk," and within an hour there's a spreadsheet of logos with nobody having agreed on what the software actually needs to do.

Write the requirements brief first, with zero vendor names in it. The brief forces the team to describe the problem in its own words before judgment gets anchored by a demo or a friend's recommendation. It also becomes the yardstick you hold every candidate against later, instead of comparing vendors to each other in a vacuum.

Two people writing this brief independently, then comparing notes, almost always surfaces disagreement that would otherwise show up mid-evaluation, after six demos. It's a cheap way to find the fight early, when it costs an afternoon instead of a quarter.

Must-haves versus nice-to-haves: the walk-away test

The brief above splits requirements into must-haves and nice-to-haves, and that split is where most shortlists quietly fall apart. Teams write "must-have" lists of fifteen or twenty items, which means nothing on the list is actually a must-have. A real must-have is something you'd walk away from a vendor over, even if everything else about it were excellent.

Apply a simple test to every candidate must-have: if this vendor did everything else brilliantly but failed on this one thing, would you still say no? If the honest answer is "we'd probably still consider it," it's a nice-to-have wearing a must-have's badge. Move it down.

Five to eight real must-haves is a healthy number. If your list runs past ten, run it through the walk-away test again. A bloated must-have list doesn't protect you from a bad choice, it just makes every vendor fail somewhere and gives the room an excuse to argue about which failure matters most, exactly the gut-feel debate a shortlist process is supposed to prevent.

Sometimes the honest answer, once must-haves are pinned down, is that no vendor clears the bar and the real decision is build versus buy. If that's where you land, our build versus buy guide walks through that fork for CRM, and the logic carries to most categories.

Where to source candidates, and the bias baked into each source

Once the brief exists, the next question is where the candidate list comes from. Most teams default to a Google search or whatever a colleague mentioned last week, without noticing that every sourcing channel has a built-in slant. None of these sources is untrustworthy on its own; the mistake is treating any single one as neutral.

The fix is to pull candidates from at least two source types before treating anyone as a finalist. A shortlist built entirely from one review site, one analyst grid, or one colleague's stack reflects that source's blind spot, not the market.

The disqualification pass: forty vendors to about eight

With a brief and a source-diverse candidate list, most categories hand you twenty five to forty names. Scoring all of them wastes everyone's time, so the next step is a fast, cheap pass built entirely on binary yes/no questions pulled from your must-haves list.

This pass should take an afternoon, not a week. You're not evaluating fit yet, just checking for disqualifying gaps using public information: pricing pages, security pages, integration directories, and G2 or Capterra category filters.

Category-specific evaluation-criteria guides in this collection feed directly into this pass, since each lists the hard gates for that category: see the CRM evaluation criteria checklist, accounting software evaluation criteria, and project management software evaluation criteria.

Run this pass against your full list and you'll typically land at six to ten survivors. That's where the process gets more expensive per vendor, so a hard cap earns its keep here. If more than ten survive, tighten the must-haves list rather than carrying extra candidates forward.

The deeper cut: eight to three

The disqualification pass removes anyone who obviously fails. The deeper cut finds meaningful separation among vendors who all look reasonable on paper, a slower exercise you now run only on the survivors, which is the point of doing the cheap pass first.

Eight vendors through this process is manageable in one to two weeks with a small team. Three or four survivors is the right output, not because three is magic, but because that's roughly the ceiling for what stakeholders can hold in their heads and still compare properly instead of forming a vague impression. From here, the SaaS vendor evaluation scorecard turns these finalists into a weighted, defensible score.

Timebox it, or the process dies of exhaustion

Shortlisting has no natural stopping point. There's always one more vendor to check, one more review to read. Left open-ended, the process either drags for months or gets abandoned when the original urgency fades and everyone quietly goes back to the old way of working. Set a calendar, not a vague target, before sourcing begins.

A mid-market purchase should move from "we need this" to a signed shortlist inside four to six weeks. Mission-critical systems (ERP, core CRM) can run longer, but phases should still have hard end dates, not open-ended extensions. A blown deadline signals the brief was too vague, not a reason to keep going.

Who belongs in the room

Shortlisting fails almost as often from the wrong people in the room as from a bad process. Too few voices and the list reflects one person's preferences. Too many and nobody agrees, so the loudest opinion wins by default.

Notice the executive sponsor is deliberately kept out of sourcing. A senior leader mentioning a vendor by name early carries outsized weight, intended or not, and quietly narrows the search before it starts. Bring them in at brief sign-off and the final decision, not before.

Shortlisting mistakes that quietly wreck the process

Shortlisting by brand recognition. The vendor with the biggest marketing budget shows up first in search results and gets mentioned by the most peers, none of which means it fits your requirements brief. Recognition and fit are different things, and a disqualification pass built on your must-haves catches this; a gut-feel list doesn't.

Letting one vocal stakeholder seed the entire list. If a senior leader names three vendors in the kickoff meeting, those three become the shortlist by default, and everyone else's research quietly turns into confirmation of a decision already made. Separate sourcing from any one person's prior opinion, and write the brief before any names get mentioned out loud.

Building the list from one comparison article. A single "Best CRM Tools" listicle reflects that publication's affiliate relationships and editorial choices, not your requirements. Pull candidates from multiple source types instead.

Skipping the disqualification pass to save time. It feels faster to jump straight to demos with whichever vendors seem fine. It isn't. A binary pass on ten vendors costs an afternoon; six demo cycles on vendors that fail an obvious must-have cost weeks.

Treating nice-to-haves as tiebreakers late in the process. By the time you're down to three finalists, it's tempting to let a minor feature difference decide because it's the most recent thing anyone talked about. Go back to the weighted scorecard instead of letting recency bias make the call.

Where this collection picks up from here

This guide covers the part of buying that happens before evaluation criteria and scoring start. Once you have a shortlist, these go deeper on category-specific criteria and adjacent decisions:

• Help desk evaluation criteria
• ERP evaluation criteria, the higher-stakes version of this process
• Email marketing evaluation criteria
• CRM vs spreadsheet, if you're not sure the category needs dedicated software yet
• Help desk migration guide, for after the shortlist wins
• How to choose a CRM, a worked example of this whole process in one category

Once your finalists are locked, the SaaS vendor evaluation scorecard is where the scoring happens.

Frequently asked questions

How many vendors should actually make the first candidate list?

Somewhere between 25 and 40 for a typical mid-market category, pulled from at least two source types so no single channel's bias dominates. Fewer than that and you've probably sourced from one place; more usually means the brief wasn't specific enough to narrow the field on its own.

What if the disqualification pass eliminates almost everyone?

That's useful information, not a failure. It usually means a must-have is more restrictive than the market can support at your budget or size. Check whether that requirement is a genuine walk-away item or a nice-to-have that got misclassified. If it holds up, the category just has fewer viable options than you assumed, better to know now than after a demo cycle.

Can we skip the requirements brief if the team already knows what it wants?

You can, but it's usually a false economy. "The team already knows" is often three or four unstated assumptions held by different people, and they surface as disagreement mid-evaluation instead of in a one-hour brief-writing session.

Should the shortlist include an incumbent vendor we're already unhappy with?

Yes, if it can still theoretically meet your must-haves. Excluding it by default just means you never find out whether the problem was the tool or how your team configured it. Run it through the same disqualification pass as everyone else and let the process answer the question.

How is this different from just reading a "best software for X" listicle?

A listicle reflects one publication's research and the vendors that pitched them hardest. This process builds a list from your own requirements brief and multiple independent sources, then filters it against your own must-haves. A listicle is one legitimate input into sourcing, not a finished shortlist.

Related reading

• SaaS vendor evaluation scorecard
• CRM evaluation criteria checklist
• Accounting software evaluation criteria
• Project management software evaluation criteria
• How to choose a CRM
• CRM build versus buy
• Help desk evaluation criteria
• ERP evaluation criteria
• Email marketing evaluation criteria
• CRM vs spreadsheet
• Help desk migration guide
• How to run a software trial