Key takeaways
- →A ranking exercise gives you an ordering, not just a winner - which is what you need when the top item turns out to be blocked.
- →Cap the list at seven to nine items of roughly equal size, or people rank by fatigue rather than by judgement.
- →Everyone ranks in silence and simultaneously; a visible running tally turns the second half of the room into followers.
- →The aggregate order is a summary, not a verdict - look at the spread before you treat a two-point gap as a mandate.
- →The item that half the room ranks first and half the room ranks last is the most useful result in the exercise, not an error.
- →A ranking that ends without an owner and a date for the top item was a survey, not a decision.
A live ranking exercise asks everyone in the room to put the same shortlist in priority order at the same time, then shows the group its aggregate ordering. It is the right instrument when you need more than a winner - when you will fund the top three, or sequence work across a quarter, or want to know what the group falls back to if the obvious first choice turns out to be blocked.
Most ranking exercises fail in one of two places. Either the list going in is unrankable - overlapping items, one enormous item sitting next to five trivial ones, wording that argues for its own option - or the result coming out is read as far more precise than it is, and a two-point gap between third and fourth place gets treated as a mandate. Both failures are set up in the design, and neither can be repaired in the debrief.
What follows is the full mechanic: when ranking beats a straight vote, how to build a list that can actually be ordered, the two-round structure that works for groups of eight to two hundred, how to stop the first visible result anchoring everyone else, how to read spread and ties honestly, and how to turn the final order into something with a name and a date attached.
When ranking beats a straight vote
A single-choice poll answers one question: which item has the most support right now. That is enough when you are picking a venue or settling a binary. It is not enough when you have nine candidate projects and capacity for three, because the poll tells you nothing about the order of the other eight or how close the fourth was to the third.
Ranking costs more. Asking someone to order nine items is a genuinely harder cognitive task than asking them to pick one, and it takes two to four minutes rather than fifteen seconds. Spend that cost when the ordering will change what you do - roadmap sequencing, budget allocation, which three of eleven risks get mitigation plans, which problems the next quarter's work attacks first.
Do not spend it when the group has no real information to distinguish the items, or when you already know the answer and are running the exercise for legitimacy. Groups can tell the difference between being consulted and being managed, and a ranking whose result you overrule is worse than no ranking at all - it teaches people that the next one is decoration too.
Ranking, rating and dot voting are different instruments
The rating trap is the one most teams fall into, because a 1-5 scale per item feels easier to answer. It is easier, and that is the problem: rating imposes no scarcity. A person can mark six items as 'very important' without ever making the trade-off you actually needed them to make. Ranking forces the trade-off into the response itself, which is the whole point of running it.
Dot voting sits in between and is genuinely better than ranking when the list is long and rough - twenty sticky notes after a brainstorm, where you want to cut to a shortlist rather than order one. Use dots to get from twenty items to seven, then rank the seven. That two-stage move is faster than either method alone and produces a far better list going into the ordering round.
| Method | What it gives you | Group size | Where it breaks |
|---|---|---|---|
| Full ranking (order every item) | A complete order, including second and third fallbacks | 6-60 | Lists past nine items - people order the tail by fatigue |
| Top-three pick, unordered | A shortlist with no internal order | 10-500 | You need to know which of the three starts first |
| Dot voting | Rough intensity spread across a wide set | 8-200 | Dots are visible while people are still spending them |
| Single-choice poll | One winner and nothing else | Any size | The top two finish within a few points of each other |
| Pairwise comparison | A robust order built from many small choices | 6-40 | Item count grows - the comparisons multiply fast |
| 1-5 rating per item | Absolute enthusiasm, not relative priority | Any size | Everything comes back a four and nothing separates |
Build a list that can actually be ranked
The list is where a ranking exercise is won. A facilitator who spends ten minutes tightening nine items will get a usable order from a three-minute round; one who ranks whatever came out of the brainstorm will spend twenty minutes debating the result and still leave without a decision.
Cap the list at seven to nine items
Beyond nine, people stop comparing and start pattern-matching: they place the two they care about, dump the rest in the order they appear on screen, and the bottom half of your result becomes noise dressed as data. If you have fourteen candidates, cut to nine first with dots or a quick show of hands.
Make the items roughly the same size
'Rewrite the onboarding emails' and 'replatform billing' cannot sit in the same ranking. The big item wins on ambition or loses on fear, and either way the group is voting on scope rather than value. Split or merge until every item is plausibly the same order of magnitude of work.
Strip the advocacy out of the wording
An item written by its sponsor arrives pre-argued: 'fix the critical checkout bug losing us customers' will beat 'search improvements' regardless of merit. Rewrite every item into the same flat grammatical shape - a noun phrase of five to eight words, no adjectives that do persuasion.
Deduplicate out loud, in front of the group
Two items that overlap by seventy per cent split the vote between them and both finish mid-table. Read the list aloud before the round and ask directly: 'are three and seven the same thing?' Merging live takes forty seconds and saves you a meaningless result.
Name the ranking criterion in one sentence
'Rank these by what will most reduce support tickets in the next quarter' produces a different order from 'rank these by what you personally want to work on', and if you do not say which, you get a blend of both. Put the criterion on screen and leave it there for the whole round.
Freeze the list before anyone ranks
Once ranking starts, no additions. A late item added at position ten arrives with less consideration than the rest and distorts everything below it. Note it, park it, and tell the person it goes into the next round - which also stops the ranking becoming a negotiation about what is on the list.
The round structure that keeps the group moving
Fifteen minutes is the realistic budget for a nine-item ranking with a real challenge round in it. You can compress to six minutes by skipping the second round, and that is fine when the first result is clean - a clear top three and a wide gap to fourth. Skip the second round when the result is contested and you will spend the saved time arguing about it in the corridor instead.
The order of those blocks matters more than their durations. Silent individual ranking comes before any discussion, always. Discussion first produces a room that has already heard the two most confident opinions, and the ranking that follows measures those opinions rather than the group's.
0:00-1:00 State the criterion and the stakes
One sentence on what people are ranking by, and one on what happens to the result: 'the top three get engineering time in January, four to nine do not.' Consequence changes how carefully people rank.
1:00-4:00 Everyone ranks in silence, at once
No discussion, no visible tally, no going round the room. Three minutes for nine items is right; announce the response count as it climbs so the wait feels like progress rather than dead air.
4:00-5:00 Reveal the aggregate, not the individuals
Show the combined order and the spread. Keep individual rankings private unless the group has explicitly agreed otherwise - attribution changes how honestly people rank next time.
5:00-9:00 Challenge round on two items only
Ask for the strongest argument for one item to move up and one to move down. Two arguments, ninety seconds each. Open discussion at this point produces the same three voices and no new information.
9:00-12:00 Re-rank the contested middle
Positions one and two rarely move. Re-run the ranking on the three or four items in the disputed band only, so the second round is fast and the group can see what the argument actually changed.
12:00-15:00 Lock the top three and name owners
Read the final order aloud, state what happens to everything below the cut line, and put a name and a date against each of the top three before anyone leaves the room.
Sizing the exercise: items, time and people
These are structural defaults, not findings - start here and adjust once you have run the exercise with your own group twice. The one figure worth defending is the item cap. Working memory is the binding constraint in a ranking task, because ordering nine things means holding several comparisons in mind at once, and a list of eighteen does not produce a more considered result, only a more tired one.
Group size affects the reveal more than the ranking itself. Under about twenty people you can show the aggregate and discuss it as one room. Above sixty, the challenge round has to be structured - two nominated arguments, or a follow-up poll on a single contested pair - because open discussion at that scale is dominated by whoever is nearest a microphone.
Stop the first result anchoring everyone else
The most common way a live ranking gets corrupted is a running tally displayed while people are still submitting. The first twenty responses set a visible shape, and the remaining eighty rank towards it - not out of cowardice, but because a visible aggregate reads as information about what is reasonable. You end up measuring the first twenty people with a large margin of error attached.
Hide the results until the round closes. This is a one-click setting in most tools, including Session Flo, and it is the single highest-value default in the whole exercise. If your tool cannot hide a live tally, put the results screen away and rank on paper - the loss of polish is worth far less than the loss of an independent result.
The same logic applies to talking. If a senior person says 'obviously the migration has to be first' before the round, they have not expressed an opinion, they have set an anchor, and everything ranked afterwards is relative to it. Rank first, argue second. If the senior person wants to argue for the migration, they can do it in the challenge round like everyone else, and the group can see exactly what their argument moved.
Reading the result honestly
An aggregate ranking is a summary of disagreement, not a measurement of truth. Two results with identical top-three orders can mean entirely different things: one where almost everyone submitted a similar list, and one where two factions submitted opposite lists that averaged into something nobody actually wanted. Before you act on the order, look at how wide the spread is on each item.
Three checks take about ninety seconds. First, the gap between the last funded item and the first unfunded one - if third and fourth are within a couple of points, you do not have a top three, you have a top two and a coin toss. Second, the variance on each item: an item ranked consistently fifth by everyone is genuinely mid-priority, which is useful. Third, whether any item is bimodal, which is the finding worth stopping for.
Say all of this out loud when you reveal. 'Items one and two are clear. Three and four are effectively tied so I am going to ask about them. Six is the interesting one - some of you put it top, some bottom.' Reading the result with its uncertainty intact protects you later, when someone claims the group ranked something fourth as if that were a settled fact.
The split item is the most valuable result
When half the group ranks an item first and half ranks it last, the average puts it in the middle, which is the one place it definitely does not belong. That pattern almost always means the room is holding two different pieces of information - one group knows about a customer commitment, or a dependency, or a cost, that the other group does not.
Stop the exercise and ask directly: 'six is split. Whoever put it top, what do you know?' The answer to that question is frequently the most valuable two minutes of the whole session, and it is invisible in the aggregate number. Never resolve a bimodal item by averaging - resolve it by surfacing the disagreement, then re-ranking that item alone once everyone has the same information.
What to do with a near-tie
Top Tips
- Do not break the tie by re-running the same ranking. Two items that finish within a point of each other will finish within a point again, and a second identical round mostly measures who stayed engaged.
- Run a single head-to-head between the tied pair with the criterion sharpened: 'which of these two reduces support load faster?' A binary question on a narrow criterion separates items that a nine-way ranking could not.
- Change the criterion rather than the method. If both items score the same on value, rank the pair on effort, risk or reversibility. Items that tie on one dimension almost never tie on two.
- Ask whether the tie matters. If both items are getting done this quarter and only the order is in question, sequence by dependency and move on - you can spend twenty minutes settling a tie that changes nothing downstream.
- Let the owner break it, openly. Naming the decision-maker and having them choose in front of the group is faster and more honest than pretending a third round produced consensus.
- Record the tie in the recap. 'Three and four were level; we started with three because it unblocks two other things' is a sentence that prevents the argument being reopened in six weeks.
Ranking with a big room or a hybrid one
Above roughly sixty people the mechanics hold but the discussion does not. Keep the silent ranking exactly as it is - it scales perfectly, because everyone works in parallel - and replace the open challenge round with something structured: collect written arguments for a single contested item, show the three most-supported, then re-rank. The ranking is not the bottleneck at scale; unstructured conversation is.
In a hybrid room, everyone ranks on their own device including the people sitting together at the table. It feels odd to ask six colleagues in the same room to type instead of discuss, and it is the only way the remote half of the group gets an equal weight in the result. A ranking where the in-room people talk it through and the remote people submit individually is not one exercise, it is two.
Watch the reveal in hybrid especially. If the aggregate goes up on the meeting-room screen and the remote attendees see a shared screen a second and a half behind, the in-room reaction lands before they can read the result. Read the order aloud, item by item, before commenting on it - that costs fifteen seconds and puts everyone on the same footing.
Turning the order into a decision
The last item is the one that decides whether people take the next ranking seriously. Everything below the cut line needs a stated fate: parked until April, dropped permanently, or handed to another team. If items simply disappear, their sponsors learn that the exercise is where their idea goes to be quietly buried, and next time they will lobby before the round instead of ranking during it.
Send the full order in the recap, not just the winners. People who ranked item seven top want to see that it finished seventh across the group - that is what makes the outcome feel like a group decision rather than a facilitator's preference. A recap that lists all nine items with their final positions and the two sentences of context around the ties does more for buy-in than any amount of consensus language in the room.
- ✓Criterion stated in one sentence and visible for the whole round
- ✓List capped at nine items of comparable size, wording de-argued
- ✓Cut line announced before the vote, not after the result
- ✓Results hidden until every response is in
- ✓Spread and near-ties read out loud, not just the order
- ✓Split items surfaced and re-ranked rather than averaged
- ✓Top three carry a named owner and a date before anyone leaves
- ○The unfunded items get an explicit status, not silence
Running it again next quarter
A ranking exercise gets substantially better the second and third time a group runs it, for a simple reason: people learn what the criterion means in practice. The first time you ask a team to rank by 'impact on support load', half of them are guessing at your definition. By the third time they have seen what happened to the top three, and their rankings sharpen accordingly.
Keep the previous order visible when you re-rank. Showing last quarter's positions alongside the new list turns the exercise into a conversation about what changed - 'this was seventh in October and is second now, what moved?' - which is more useful than treating each quarter as a blank sheet. It also exposes the item that has been ranked fourth three times running and will never be done, which deserves killing rather than another appearance on the list.
Two practical habits make the repeat cheap. Keep the item wording stable between rounds so positions are comparable, and keep the same cut line unless capacity genuinely changed. Ranking the same nine things with the same rule twice tells you something real about how the group's judgement is moving; ranking a reshuffled list against a new criterion tells you almost nothing.
Frequently asked questions
How many items should a live ranking exercise have?
Seven to nine is the working range. Below five, a simple poll or a show of hands gets you the same answer faster. Above nine, people place the two or three items they care about and then order the rest by whatever sequence they appear on screen, so the bottom half of your result is noise. If you have fifteen candidates, run a quick dot vote first to cut to nine, then rank those - two fast rounds beat one long one.
Should people rank all the items or just their top three?
Rank everything when the list is nine items or fewer and the ordering below the cut line matters - for instance when you need to know what gets picked up if the top item is blocked. Ask for a top three only when the group is large, the list is long, or you genuinely only need a shortlist. The hybrid that works well is ranking the full list but only reporting positions to the cut line plus one, so people see what nearly made it.
Should a live ranking be anonymous?
Anonymous by default. Ranking is a statement about priorities, which means it is also a statement about whose project matters less, and named rankings quietly become polite. Anonymity gets you the honest order. The exception is a small leadership group of five or six who own the outcome jointly - there, visible rankings can be worth the loss of candour because the conversation about why someone ranked something last is the actual work.
What if the group's ranking disagrees with the decision-maker?
Say so plainly rather than quietly overruling it. 'The room ranked the migration fourth and I am going to start it first, because of a contract deadline you cannot all see' costs you nothing and preserves the exercise. Overruling without explanation is what teaches a group that ranking is theatre. If you already know the decision, do not run the ranking - ask a different question, like what would make the decided item succeed.
How do you run a ranking exercise with a large audience?
The silent ranking scales without modification, because everyone works in parallel on their own device - two hundred people take the same three minutes as twelve. What does not scale is the discussion. Replace the open challenge round with a structured one: collect short written arguments for the most contested item, display the two or three with the most support, then re-rank just that item. Session Flo runs the ranking and the follow-up from the same room code so nobody has to rejoin.
How is a live ranking exercise different from dot voting?
Dot voting measures intensity across a wide, rough list - people spend a fixed number of dots wherever they like, including several on one item. Ranking measures relative order across a tight list and forces a complete ordering with no ties. Use dots to reduce twenty raw ideas to a shortlist, and ranking to order that shortlist. Dots are also more prone to visible-vote bias, since people can see where the dots are landing while they place their own.
Keep reading
- how dot voting and ranked voting differ in practice — Choosing between the two before you design the round.
- running dot voting to cut a long list down — The step that gets twenty raw ideas to a rankable nine.
- why the first visible result anchors the room — The evidence behind hiding the tally until the round closes.
- turning workshop output into decisions with owners — What happens after the order is locked.
- run a live ranking with Session Flo — Ranking, dot voting and follow-up polls from one room code.