Selecting a Cohort You Can Defend a Year Later
Shortlisting, interviews, conflicts of interest and the waitlist - how to run selection so the decisions still make sense a year later.
The question that arrives later
Somewhere between six and eighteen months after a cohort starts, somebody asks how it was chosen. A governing body reviewing the programme. A funder. Occasionally an unsuccessful applicant, sometimes with more force than you would like.
The answer you want to give is boring: here was the rubric, here is how each application scored, here is who was shortlisted and why, here is what changed at interview, here is the decision. Boring is the goal. Boring means the process held.
Everything below is about making that answer available without archaeology.
Fix the process before you open applications
Decide and write down, before a single application arrives:
- How many places there are. A number, not "around eight".
- The rubric and weights (see the panel evaluation article for how to build one).
- The shortlist ratio - typically two to three times the places.
- Who sits on the panel, and who chairs.
- The conflict rule.
- The dates: close, shortlist, interview, decision, announcement.
Publishing the dates costs nothing and removes the single most common applicant complaint, which is not knowing when they will hear.
Stage one: eligibility, before anyone reads anything
Run the hard gates mechanically. Incorporated or not, founded within the window, sector, location, whatever your scheme requires.
Do it as a filter rather than as part of scoring. An ineligible application should never reach a panel member, because panel attention is your scarcest resource and you will need it again next round.
Record the reason for each exclusion. "Ineligible" is not a reason; "incorporated 2019, outside the three-year window" is.
Stage two: independent scoring
Every eligible application scored against the rubric, by at least three people, independently, before any discussion.
Two mechanics matter here.
Blind the first pass if you can. Hide names, institutions and photographs. It is a small effort and it measurably reduces the pull of a familiar college or a known founder. Reveal identity at shortlist, when you need it for diligence and conflict checks.
Handle conflicts before scoring opens. Circulate the applicant list, collect declarations, and remove those applications from the conflicted person's queue. A conflict declared after the scores are in is a much harder conversation, and it looks worse than it usually is.
Stage three: the shortlist, and the disagreements
Rank by weighted score. Take the top two or three times your places.
Then look specifically at the applications where the panel disagreed most - a wide spread of scores on the same application. Those are the interesting cases, and a ranked list alone buries them: an application scored 2, 3, 5 lands in the middle of the ranking looking unremarkable, when in fact your panel does not agree about it at all.
Surface those deliberately and discuss them before the shortlist is fixed. It is the highest-value twenty minutes in the whole process.
Stage four: interviews that add something
The interview should test what the form cannot: how the founders think when questioned, whether the team functions, and whether they can defend the claims they made in writing.
Three practices that keep interviews comparable:
- Same core questions for everyone, with room to follow up. Wholly freeform interviews cannot be compared across candidates, which puts you back where the rubric started.
- Two interviewers minimum, ideally including someone who did not score the written application.
- Score against the same rubric, as a separate scoring event. Do not overwrite the written score - keep both, because the gap between them is informative and it is the thing you will want when someone asks why a strong application did not convert.
Stage five: the decision meeting
Score to shortlist, discuss to select.
The discussion legitimately considers things a rubric cannot: portfolio balance, whether you have the right mentors for a sector this cohort, whether two applicants are effectively the same company. What it must not do is quietly re-litigate the scores.
Two rules keep it honest:
- Anyone can argue for a change, and the reason gets written down. "Moved up because the interview showed the technical depth the form did not" is a legitimate, recorded reason. Movement with no recorded reason is what makes a process indefensible.
- The chair does not score. Someone has to run the meeting without also arguing in it.
Stage six: the waitlist is a real list
Rank the near-misses in order and tell them they are on it.
Startups do drop out - funding arrives, a founder leaves, circumstances change - and a ranked waitlist means calling the next name rather than reopening applications, which is slow and looks disorganised.
It is also kinder. "You were fourth on the waitlist" is a materially different message from a form rejection, and those founders often apply again.
Tell people, and tell them something
Decisions go out on the date you published. Unsuccessful applicants get more than a form letter where you can manage it - even two sentences on the weakest criterion is remembered, and it is the cheapest goodwill an incubator can buy.
Do not offer detailed feedback you cannot deliver at volume. Promising individual feedback to ninety applicants and delivering it to twelve is worse than promising a short note to everyone.
What to keep from the round
When the question arrives a year later, you want to be able to produce:
- The rubric as it stood for that round, not the current version
- Every evaluator's individual criterion scores
- Conflict declarations and recusals
- The shortlist and the ratio used
- Interview scores, kept separately from written scores
- Any score changed after discussion, with the reason and who made it
- The final decision per applicant, including waitlist position
- The dates each stage actually happened
Most incubators can produce three or four of these. The two most often missing are the rubric version and the reasons for changes - and those are precisely the two that make the answer convincing.
The one-line version
Score before you discuss, write down why anything moved, and keep the rubric that was actually used. Everything else is detail.
Frequently asked questions
- How many applications should make the shortlist?
- Two to three times the number of places. Fewer and you have no room to lose people at interview; many more and you are interviewing candidates the scores already ruled out, which wastes panel goodwill you will need next round.
- Should the final decision be by score or by discussion?
- Score to shortlist, discuss to select. Ranking by score alone ignores portfolio balance and interview evidence. Discussion alone reintroduces exactly the inconsistency the rubric removed. The order matters: score first, so the discussion starts from a common baseline.
- How should conflicts of interest be handled?
- Declared before scoring begins, not when a case arises, and the rule written before you need it. A panel member with a conflict does not score that application and does not sit in its discussion. Record the declaration and the recusal alongside the round.
- Is a waitlist worth running?
- Yes, and it should be a real ranked list rather than a polite phrase. Accepted startups drop out more often than most programmes expect, and reopening applications is far worse than calling the next name.
