← Thumbtack Interview Insights
This was essentially six questions wrapped into one.
Start by aligning on the business goal: increase booking volume and liquidity while maintaining trust and quality. Then define a north-star metric that captures successful instant bookings, and build a hierarchy of secondary and diagnostic metrics. Finally, outline the instrumentation, rollout, success criteria, and risks with mitigations, emphasizing experimentation and guardrails.
Pro tip: Tie your north-star metric to Thumbtack's marketplace health—e.g., 'Instant Book GMV' or 'Instant Book completed jobs'—and explicitly discuss how you'll measure cannibalization of the existing quote-based flow. This shows you understand two-sided marketplace dynamics and avoid optimizing a local metric at the expense of the whole.
Choose a north-star metric that reflects the core value of Instant Book (e.g., number of successfully completed Instant Book jobs) and list 2-3 secondary objectives like booking conversion rate, time-to-book, and customer retention.
Break down the funnel into diagnostic metrics (e.g., search-to-book rate, pro acceptance rate, cancellation rate) and define guardrails such as customer satisfaction (CSAT), pro utilization, and overall marketplace liquidity.
Specify the events to track (e.g., instant_book_impression, instant_book_click, booking_confirmed) and ensure proper attribution, user-level tracking, and data pipeline readiness for real-time monitoring.
Propose a phased rollout (e.g., A/B test in select markets) with a control group, define success criteria (e.g., statistically significant lift in north-star with no degradation in guardrails), and outline iteration based on results.
Identify key risks such as pro supply shortages, price sensitivity, quality issues, and cannibalization, and propose mitigations like dynamic pricing, pro incentives, quality monitoring, and holdout groups.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Buried inside the bigger question but they pushed on this specifically.
Start by defining a clear, mutually exclusive taxonomy of booking events (organic vs. Instant Book) and a unified sessionization logic that stitches user activity across devices and time. Then propose a deduplication rule that prioritizes the final conversion event and uses a deterministic event ID or user-level aggregation to avoid double-counting. Finally, outline how you would validate the instrumentation with QA checks and monitor for edge cases.
Pro tip: Emphasize that the metric definition must align with the business question—e.g., whether you're measuring conversion rate or total bookings—and that you'd socialize the definition with product and engineering to ensure consistent implementation. Also, mention the importance of logging both the quote request and Instant Book events with a shared correlation ID to enable accurate deduplication.
Classify each booking attempt as organic (e.g., request-a-quote) or Instant Book based on the user action and flow. Ensure the definitions are mutually exclusive and exhaustive, and document them clearly.
Use a user ID (e.g., logged-in user or deterministic identifier) to stitch sessions across devices. For anonymous users, employ probabilistic or cookie-based stitching with a lookback window, and handle identity resolution when users log in.
Assign a unique correlation ID to each booking attempt that spans quote and Instant Book events. When both occur, prioritize the Instant Book event as the conversion and mark the quote as non-converting, or use a rule like 'last-touch' within a short window.
Implement event logging with the correlation ID and user ID, and set up QA checks to ensure no double-counting. Validate with test cases simulating multi-device and dual-event scenarios.
Set up dashboards to track the ratio of organic to Instant Book conversions and monitor for anomalies. Periodically review edge cases and refine definitions as the product evolves.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
I got adverse selection and schedule conflicts but drew a blank on price anchoring.
Start by framing Instant Book as a product change that shifts the marketplace from a request-based to a commitment-based model, then systematically identify risks across supply, demand, and trust dimensions. For each risk, propose a concrete metric, monitoring plan, and mitigation strategy, emphasizing how you would use data to detect and address issues early.
Pro tip: Tie every risk to a measurable metric and a clear threshold for action, and mention the importance of running a controlled experiment (e.g., A/B test) before full rollout to quantify impact and catch unintended consequences.
Briefly restate what Instant Book entails and its intended benefits (e.g., faster bookings, higher conversion) to set context for risk analysis.
Brainstorm risks for customers (e.g., poor matches, cancellations), pros (e.g., lost flexibility, no-shows), and the platform (e.g., trust erosion, support load).
For each risk, specify a metric (e.g., cancellation rate, pro acceptance rate, customer satisfaction) and how you would track it (dashboards, alerts, experiments).
Suggest product, policy, or algorithmic interventions (e.g., smart matching, penalties, insurance, gradual rollout) to reduce risk impact.
Explain how you would prioritize risks based on severity and likelihood, and set up a process to learn and adapt post-launch.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.