Microsoft·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

Microsoft ML Engineer interview that went deep on data pipeline design, specifically around cleaning and preparing a messy OCR dataset for LLM training. One long technical question that branched into four or five sub-topics, so it felt more like a working session than a traditional interview.

Questions Asked (1)

You're given a raw OCR-extracted text dataset intended for LLM training or fine-tuning. Walk through your full pipeline: how do you diagnose OCR noise, filter and deduplicate, normalize the data, and ultimately validate dataset quality before and after cleaning?

System DesignTechnical Trade-offsData Modeling

Author's notes

This question is basically four questions glued together and I did not pace myself well.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer as a systematic pipeline: start with diagnosis of OCR noise using quantitative metrics and sampling, then apply filtering and deduplication techniques, followed by normalization steps, and finally validate quality with both automated metrics and human evaluation. Emphasize trade-offs between aggressive cleaning and data retention, and how you iterate based on validation results.

Pro tip: Highlight the importance of establishing a baseline by measuring model performance on the raw data before cleaning, so you can quantify the impact of each cleaning step and avoid over-cleaning that removes valuable signal.

1. Diagnose OCR Noise

Quantify noise types (e.g., character errors, garbled text, layout issues) using metrics like character error rate on a sample, and visualize common failure patterns to guide cleaning.

2. Filter and Deduplicate

Apply heuristic filters (e.g., length, language confidence, symbol ratio) and deduplication methods (e.g., MinHash, exact match) to remove low-quality and redundant data while preserving diversity.

3. Normalize Data

Standardize text through Unicode normalization, whitespace correction, and language-specific rules (e.g., fixing common OCR confusions like 'rn' to 'm') without altering semantic content.

4. Validate Quality Pre- and Post-Cleaning

Use automated metrics (e.g., perplexity, downstream task performance) and human evaluation on samples to compare raw vs. cleaned data, ensuring improvements and catching regressions.

5. Iterate and Document

Based on validation, refine cleaning steps, document decisions and trade-offs, and establish a repeatable pipeline for future datasets.

Key Points to Mention

Quantitative diagnosis: character error rate, noise pattern analysis, sampling
Filtering heuristics: length thresholds, language identification, symbol-to-word ratio
Deduplication techniques: exact match, fuzzy hashing (MinHash, SimHash), semantic dedup
Normalization: Unicode NFKC, whitespace normalization, OCR-specific corrections
Validation metrics: perplexity, downstream task performance (e.g., GLUE), human evaluation
Trade-offs: over-cleaning vs. data retention, computational cost, bias introduction

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.