Assessing Fit Through Discovery Projects
4 hours of real work tells you more than 4 hours of interviews.
TL;DR
Core Insight: Resumes don't work. Proof of work should be the gold standard at every tech company.
Focus on These 3 Things:
- Real Problems: Use actual challenges from the role. No LeetCode, no trivia, no "gotcha" questions.
- Personalize It: Add their name, reference past conversations. Tiny effort = massive completion rate.
- Review as Conversation: Start with what they did well. Hype them up. 99% of people have never completed a proof of work project.
The Core Philosophy
- Sell while assessing. The project shows them what the work is like
- 2-4 hours max. Respect their time or they'll ghost
- Personalize every project. Reciprocity drives completion
- Conversation, not interrogation. Start with what they did well
- Skills are clear. Team decides on potential, obsession, and willingness to ship
The Discovery Project Template
Note: This is a Notion template used previously. For Wispr Flow, this could be elevated to a custom-branded webpage that matches their design language.
Template Structure
# [Company] Discovery Project - <insert role title>
Welcome to [Company]'s official project for the <insert role title>!
**[Insert company mission statement]**
Section 1: Who Are We?
[Company] is a [describe what the company does in one sentence]. We're a collection of [describe the team composition]. We've helped [social proof - customers, users, notable clients]. Our next step is to hire an incredible <insert role title>.
Section 2: You're a Great Fit If...
[Insert Success Vision from Job Scorecard]. [Insert second responsibility (what's non-negotiable?)]. You live and breathe <insert role>.
Section 3: What's the Purpose?
We're seeking an incredibly skilled <insert role> who has <insert filter>. If [Company] succeeds, [impact statement]. The Discovery Project eliminates bias and lets you showcase the best version of you.
Section 4: Are You Ready?
Each topic represents a critical skill needed to succeed. Start with <first step> and work your way down. We're keeping this as short as possible.
When done: Submit this form or let [Recruiter Name] know. We'll reach out in a few days to check in.
How to Personalize for Every Candidate
Purpose: A tiny bit of personalization goes a long way because the candidate appreciates perceived effort. This sells them via reciprocity.
The Personalization Elements
1. Project Title
Format: [Candidate Name] + Project Name + Emoji
Example: [Candidate Name] [Company] Discovery Project 🚀
2. First Line - Role Goal Reminder Remind the candidate what the goal of their role will be.
"Your goal will be to help [Company] scale [outcome] to reach [impact]."
3. Second Line - Why This Project Inform the candidate why you sent them this DP + big impact statement.
"We created this project to see how you think about [domain] and give you a glimpse into [Company]."
Critical: Role Language Consistency
Just because you are hiring for a CMO, does not mean that language resonates with your audience.
| What the role actually is | What you might call it |
|---|---|
| CMO | Marketing Leader |
| VP Sales | Revenue Partner |
| Head of Engineering | Technical Co-Builder |
Why this matters: Especially for agency/business owners who might resist "Full Time" language.
Email Template to Send Discovery Project
Hey <candidate name> - <previous interviewer> enjoyed speaking with you!
[Company] is in the perfect position to [big goal] and we're confident
this would be an incredible case study for you.
So we're pumped to see your [role-specific skill] in action with a
[one/two]-part discovery project.
This will help us understand how you think about [their domain], and
will give you an introductory glimpse into [Company].
**Please cap this at [X] hours max (make it as great as you can).**
If you can get this back to us by the end of week, that would be great.
Once complete, you'll hop on a call with <person>, talk about how we
can work together and help scale our vision of [impact statement].
Talk soon,
[Your Name]
PS
Discovery Project LINK
Why Each Line Works
| Line | Purpose |
|---|---|
| Short opener | Builds on momentum from last call |
| Two strong reasons | Based on what THEY said they're looking for |
| Why the project | We want to see their genius in action |
| Time constraint (BOLD) | Set expectation, respect their time |
| Soft deadline | Creates gentle urgency |
| Next step | Guide them into the future |
| PS with link | People look at PS first |
After Sending
Set a follow-up reminder for 24 hours. Don't wait and wonder.
Follow-Up After Sending the Project
Note: This entire cadence should be automated in Ashby (or whatever internal tool you build) to make this seamless.
The Cadence
| Timing | Action |
|---|---|
| Day 0 | Send discovery project email |
| Day 1 (24hr) | Follow up if no response |
| Day 3 (48hr after Day 1) | Check-in on progress |
Automate all of this. Don't rely on memory.
24 Hour Follow-Up (If No Response)
Short email under 40 words to bump them:
Hey [Name],
Just wanted to make sure you got the discovery project I sent yesterday.
Let me know if you have any questions before diving in!
[Your Name]
If the Candidate Responds
Let them do their thing!
Every email you send must be outcome driven. If they responded, gave a self-imposed deadline, and seem excited, don't respond immediately with "great! LMK how I can help" which moves nothing forward.
Better to leave it silent, let them work, and follow up in 2 days.
48 Hour Check-In
Hey [Name],
Just checking in - how's the discovery project going?
No rush, just wanted to see if anything came up or if you have questions.
[Your Name]
When They Submit: Review Invite Email
Once the candidate sends their project, respond within 4 hours:
Hey [Name] - thank you and amazing job getting this to us with such speed.
I liked your [specific approach they mentioned] approach to the project.
These discovery projects are a fun way to get a taste of [Company]
and remind yourself of how much you know too.
What does your availability look like next week? Shoot me over 3-4 times
and I'll help set up the next call.
If it is easier, I can use your previous calendly link to coordinate.
What works best?
[Your Name]
Why Each Line Works
| Line | Purpose |
|---|---|
| Acknowledge speed | They spent free time on this. Give them credit. |
| Mirror their approach | If they said "holistic approach" in their submission, use that language |
| Remind them why it was valuable | For seniors: this reminds them how good they actually are |
| Schedule next step | Give them the next step as promised. ASAP! |
| Use their Calendly | Don't go back and forth on times if you have direct access to their calendar |
How to Review Discovery Projects
Part 1: Internal Review (Before Candidate Call)
Step 1: Initial Read-Through (10-15 min)
Read entire submission without taking notes. Get a gut feel for quality, effort, and thinking. Note initial impression (strong/weak/mixed).
Step 2: Score Against Non-Negotiables
| Criteria | Score (1-5) | Notes |
|---|---|---|
| [Non-negotiable 1] | ||
| [Non-negotiable 2] | ||
| [Non-negotiable 3] |
Step 3: Identify Discussion Points
| Identify | Purpose |
|---|---|
| 2-3 things they did well (be specific) | Start the call positive |
| 1-2 areas you want them to explain | Dig deeper |
| Any concerns or questions | Address before deciding |
Part 2: The Review Call (30-45 min)
Structure:
- Opening (2 min): "This should be a conversation, not an interrogation."
- Start With What They Did Well (5 min): Be genuine and specific
- Walk Through Their Approach (15-20 min): Ask open questions
- Address Concerns (5-10 min): Curious, not accusatory
- Let Them Ask Questions (5 min): Red flag if no questions
- Closing (2 min): Clear next steps
Evaluation Framework
| Pass Signals | Fail Signals |
|---|---|
| Quality of thinking | Minimal effort |
| Ownership - can defend decisions | Can't explain their work |
| Communication clarity | Defensive when questioned |
| Coachability | Generic (any company) |
| Engagement - asks good questions | No questions |
Wispr Flow Application: ML Engineer Discovery Project
Note: I created this as an example. I'm not an ML engineer. This would need to be refined with Sahaj and the team to determine what Wispr Flow actually wants to see. Everything here is optional.
Purpose: 4 hours of real work tells you more than 4 hours of interviews. This project tests whether candidates can actually solve Wispr's constraint-based problems.
| What We're NOT Testing | What We ARE Testing |
|---|---|
| LeetCode skills | Can they think about latency? |
| Ability to memorize algorithms | Do they understand the accuracy/speed tradeoff? |
| How well they interview | Can they solve problems with engineering, not just compute? |
| Trivia questions | Do they communicate technical tradeoffs clearly? |
The Project (2-3 Hours, Max 4)
| Format | |
|---|---|
| Take-home, async | 2-3 hours expected, max 4 (honor system) |
| Submit written doc + any code | 30 min follow-up call to discuss |
Pay: Based on seniority. Fair for their time, fair for Wispr Flow.
Task 1: The Latency Problem (60 min)
Scenario:
Your voice-to-text pipeline runs at 650ms end-to-end.
Users complain it feels "laggy." Target: <500ms.
Current pipeline:
1. Audio capture: 50ms
2. Preprocessing: 100ms
3. Speech model: 350ms
4. LLM cleanup: 100ms
5. Display: 50ms
Total: 650ms
Instructions:
- Identify where you'd focus optimization first and why
- Propose 3 different approaches to get under 500ms
- For each: What you'd do, expected savings, tradeoffs
- Which would you try first? Why?
Task 2: Accuracy vs Speed (90 min)
Scenario:
Latency optimized to 480ms. Great!
But accuracy dropped from 98% to 94%.
Product says: "Zero edit rate is our north star."
You can't go back to slow. You need BOTH.
Instructions:
- Explain why reducing latency might have hurt accuracy
- Propose 2-3 approaches to recover accuracy WITHOUT sacrificing speed
- How would you measure success?
- How would you A/B test your solution?
Task 3: Real-World Debugging (60 min)
Scenario:
You've shipped optimizations. Most users are happy.
But complaints:
- User A: "Accuracy is terrible"
- User B: "It's too slow"
Metrics:
- P50 latency: 420ms (good!)
- P50 accuracy: 97% (good!)
- P99 latency: 1200ms (bad!)
- P99 accuracy: 82% (bad!)
Instructions:
- What hypotheses for why P99 is so bad?
- What data would you want to debug this?
- For User A (accuracy issues), what might be wrong?
- For User B (latency issues), what might be wrong?
- How would you prioritize which to fix first?
Evaluation Rubric
| Dimension | 5 (Strong Yes) | 3 (Maybe) | 1 (No) |
|---|---|---|---|
| Technical Depth | Deep understanding, specific techniques | Surface level, knows buzzwords | Doesn't understand production ML |
| Constraint Thinking | Clearly gets on-device, latency, privacy | Knows constraints but ignores them | Keeps suggesting cloud solutions |
| Communication | Clear, structured, explains tradeoffs | Understandable but messy | Can't communicate technical ideas |
| Problem-Solving | Creative, sees angles others miss | Basic solutions, nothing special | Can't approach systematically |
Hire Bar: 18-20 = Strong Yes | 14-17 = Yes | 10-13 = Maybe | <10 = No
Red Flags in Submissions
| What They Say | What It Means |
|---|---|
| "Just use a bigger model in the cloud" | Doesn't understand on-device constraint |
| No mention of tradeoffs | Binary thinking, not engineering thinking |
| Copy-pasted generic answers | Didn't engage with the specific problem |
| Only one solution per task | Can't generate multiple approaches |
| No metrics or measurement plan | Doesn't think about validation |
Green Flags in Submissions
| What They Say | What It Means |
|---|---|
| Mentions distillation, quantization, pruning | Technical depth |
| Considers device constraints throughout | Gets the problem |
| Proposes ways to measure success | Scientific mindset |
| Acknowledges uncertainty ("I'd need to test this") | Intellectual honesty |
| Shows enthusiasm for the problem | Culture fit |