Thumbtack·Data Scientist·Technical Phone Screen·Senior

Senior

Jul 2026

Summary

Thumbtack data scientist interview, technical phone screen focused entirely on a single dense pandas problem involving response rate calculations, confidence intervals, and data quality handling. Pretty involved for a phone screen but I guess that's how they filter.

Questions Asked (1)

Given a CSV with job posting data (job_id, job_category, invitations_sent, provider_responses, region, created_at), write pandas code that: computes a per-job response rate while treating zero-invitation rows as missing; calculates an invitation-weighted response rate per job category with 95% Wilson score confidence intervals; returns the top 5 categories ranked by weighted rate with CI lower bound as tiebreaker; drops and logs rows with impossible values like negative counts or provider_responses exceeding invitations_sent; and verifies that the job-level weighted average matches the aggregate rate within floating point tolerance.

Product Analytics & MetricsData ModelingTechnical Trade-offs

Author's notes

This was a lot to hold in your head at once.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by outlining a clear data validation and cleaning step, then compute job-level response rates with zero-invitation rows set to NaN. Next, aggregate to category level using invitation-weighted rates and Wilson score intervals, rank categories, and finally verify consistency between job-level and aggregate rates.

Pro tip: Mention that you would log dropped rows with reasons and counts, and that you'd use vectorized operations for efficiency. Also, note that the Wilson interval is preferred over normal approximation for proportions, especially with small sample sizes or extreme rates.

1. Data Validation and Cleaning

Identify and drop rows with impossible values (negative counts, provider_responses > invitations_sent), logging the dropped rows with reasons. Treat zero-invitation rows as missing for response rate calculation.

2. Compute Job-Level Response Rate

Calculate response rate per job as provider_responses / invitations_sent, setting rate to NaN where invitations_sent == 0. This ensures zero-invitation rows are excluded from rate calculations.

3. Aggregate to Category Level with Wilson CI

For each job_category, compute the invitation-weighted response rate as sum(provider_responses) / sum(invitations_sent). Calculate 95% Wilson score confidence intervals for each category's weighted rate.

4. Rank and Select Top Categories

Sort categories by weighted response rate descending, using the CI lower bound as a tiebreaker. Select the top 5 categories.

5. Verify Consistency

Compute the overall aggregate response rate (total provider_responses / total invitations_sent) and compare it to the job-level weighted average (weighted by invitations_sent) within a small floating point tolerance (e.g., 1e-9).

Key Points to Mention

Handling zero-invitation rows by setting response rate to NaN to avoid division by zero and to treat them as missing.
Using invitation counts as weights for category-level aggregation to reflect the volume of invitations per job.
Implementing the Wilson score interval formula for binomial proportions, which is robust for small samples and extreme proportions.
Logging dropped rows with reasons (e.g., negative counts, responses > invitations) for data quality monitoring.
Ensuring the job-level weighted average matches the aggregate rate by using the same weights (invitations_sent) and checking with np.isclose.
Using vectorized pandas operations for efficiency and readability, avoiding row-wise loops.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.