This one took more thought than I expected.
Start by clarifying the business questions and the grain of each DataFrame, then plan the join strategy to avoid row explosion or loss. Use pandas merge with validate to ensure correct cardinality, and groupby with named aggregations to compute metrics. Validate results by cross-checking totals, spot-checking edge cases, and ensuring no data loss or duplication.
Pro tip: Always validate your joins by checking row counts before and after, and use merge's validate parameter to catch unexpected duplicates. Also, consider the business context: for example, order completion rate might require filtering to only completed orders, and average basket size might need to exclude cancelled orders.
Identify the specific metrics needed (e.g., average sales per category, revenue by category, per-restaurant completion rate, average basket size) and understand the grain of each DataFrame (e.g., orders at order level, items at item level).
Determine the join keys and order (e.g., orders to items on order_id, then to restaurants on restaurant_id). Use merge with validate='one_to_many' or similar to ensure correct cardinality and avoid duplication.
Use groupby on the appropriate dimensions (e.g., category, restaurant_id) and apply aggregations like sum, mean, count, and custom functions for rates. Use named aggregation for clarity.
Calculate metrics like order completion rate (completed orders / total orders) and average basket size (total sales / number of orders) using the aggregated data, ensuring correct denominators.
Cross-check totals against raw data, spot-check specific groups, and ensure no data loss or duplication. Use sanity checks like sum of category revenue equals total revenue.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.