LinkedIn·AI Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

LinkedIn AI Engineer system design round, one big open-ended question about building a job categorization system that can also invent new categories on the fly. Dense problem with a lot of moving parts and I'm not sure I covered everything they wanted.

Questions Asked (1)

Design an ML system that takes a new job posting and either maps it to an existing job category or proposes a brand-new category when nothing fits. Walk through text representation, classification approach, how you'd decide when to trigger a new category, how new categories get created and named, your labeling pipeline, inference strategy, evaluation, and how the taxonomy evolves over time.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

This is a sprawling question and I underestimated how much ground they actually wanted covered.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around the full ML lifecycle: start with text representation and classification, then address the open-set problem of triggering new categories, and finally cover taxonomy evolution. Emphasize trade-offs between precision and recall, and how you'd balance automation with human-in-the-loop for new category creation.

Pro tip: Frame the problem as open-set classification with a human-in-the-loop for taxonomy expansion, and highlight that the system should optimize for long-term taxonomy health, not just immediate accuracy.

1. Text Representation & Classification

Use a transformer-based encoder (e.g., BERT) fine-tuned on job postings to get embeddings, then train a classifier with a softmax over existing categories. Consider hierarchical classification if categories have structure.

2. Open-Set Detection & New Category Trigger

Use confidence thresholding or distance-based methods (e.g., Mahalanobis distance) to detect out-of-distribution postings. Trigger new category creation when confidence is low or when a cluster of similar postings emerges.

3. New Category Creation & Naming

Cluster low-confidence postings and use human experts to review and name new categories. Optionally, use LLMs to suggest names based on cluster keywords, but always validate with humans.

4. Labeling Pipeline & Inference

Implement active learning to label uncertain samples, and use a two-stage inference: first classify to existing categories, then if uncertain, route to a new category candidate pool for human review.

5. Evaluation & Taxonomy Evolution

Evaluate with precision/recall/F1 and track new category creation rate. Periodically retrain the model with updated taxonomy, and monitor for category drift or merging.

Key Points to Mention

Use of pre-trained language models for text representation
Open-set recognition techniques (e.g., confidence thresholding, OOD detection)
Human-in-the-loop for new category validation and naming
Active learning to efficiently label uncertain samples
Evaluation metrics that account for both existing and new categories
Taxonomy evolution: retraining, category merging/splitting, and monitoring

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.