Atlassian·Data Scientist·Take-home Assignment·Intermediate

Intermediate

May 2026

Summary

Take-home assignment for a Data Scientist role at Atlassian. Pretty involved for a take-home, basically a full ML pipeline from data loading to model comparison to final predictions, all in one shot.

Questions Asked (1)

Given two CSVs (training and test sets with a binary outcome and six continuous features), write a complete ML pipeline: load the data, run diagnostics including missingness and collinearity checks, fit baseline logistic regression plus L1 and L2 regularized variants with cross-validated penalty selection, report ROC AUC and a threshold-dependent metric on the held-out set, compare models and justify your regularization choice, identify the most important features, and output final predicted probabilities with a confusion matrix.

Technical Trade-offsProduct Analytics & MetricsAlgorithms & Data Structures

Author's notes

This took way longer than I expected.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Walk through the pipeline end-to-end, emphasizing reproducibility and justification for each choice. Start with data loading and diagnostics, then model fitting with cross-validation, evaluation on held-out test set, and interpretation. Highlight trade-offs between L1 and L2 and how you would communicate results to stakeholders.

Pro tip: Always set a random seed and use stratified cross-validation to ensure stable results, especially with imbalanced data. When comparing models, consider not just AUC but also business implications of false positives/negatives to choose the threshold and regularization.

1. Data Loading and Diagnostics

Load training and test CSVs, check for missing values, and examine feature distributions and correlations. Use visualizations and summary statistics to identify potential issues like multicollinearity.

2. Preprocessing and Feature Engineering

Handle missing data via imputation (e.g., median) and scale features if needed for regularized models. Consider creating interaction terms or polynomial features if domain knowledge suggests, but keep it simple for baseline.

3. Model Fitting with Cross-Validation

Fit baseline logistic regression, then L1 and L2 regularized versions. Use grid search with cross-validation to select the penalty strength (C) for each, ensuring to use the same folds for fair comparison.

4. Evaluation and Model Comparison

Evaluate models on the held-out test set using ROC AUC and a threshold-dependent metric like F1-score or accuracy at a chosen threshold. Compare performance and justify regularization choice based on interpretability and predictive power.

5. Interpretation and Output

Identify most important features via coefficients (for L1, many will be zero). Output predicted probabilities and a confusion matrix at the selected threshold, and summarize findings for stakeholders.

Key Points to Mention

Use stratified k-fold cross-validation to maintain class balance in each fold.
Check for multicollinearity using correlation matrix or VIF; consider dropping or combining highly correlated features.
Standardize features before regularization to ensure fair penalty application.
L1 regularization performs feature selection by driving some coefficients to zero, while L2 shrinks coefficients but keeps all features.
ROC AUC is threshold-independent, but for business decisions, choose a threshold based on cost-benefit analysis and report metrics like precision, recall, or F1.
Communicate results in terms of business impact, e.g., how many false positives/negatives are acceptable.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.