← Atlassian Interview Insights

Atlassian·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
Jun 2026

Summary

System design round at Atlassian for a software engineer role. The whole thing was one big question about building an async web crawler with a polling API, and it went pretty deep into distributed systems territory.

Questions Asked (1)

Q1

Design a REST service where clients submit a list of URLs and the service crawls each URL and all its sublinks to extract image references. POST /jobs accepts URLs and returns a job ID; GET /jobs/{id} returns the status and discovered images once complete.

System DesignAPI & IntegrationsTechnical Trade-offs
Author's notes

This one sprawled in every direction.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then outline a high-level architecture that separates the API layer from asynchronous crawling workers. Focus on scalability, reliability, and trade-offs, and walk through the lifecycle of a job from submission to completion.

Pro tip: Emphasize idempotency and rate limiting to prevent abuse and ensure fair resource usage, and discuss how you would handle failures and retries in the crawling pipeline.

1. Clarify Requirements

Ask questions to understand scale, expected latency, depth of crawling, and whether authentication or politeness policies are needed.

2. Design API Contract

Define the POST /jobs and GET /jobs/{id} endpoints, including request/response schemas, status codes, and error handling.

3. Architect the System

Propose a distributed system with a job queue, worker pool for crawling, and a data store for job status and results.

4. Address Scalability and Reliability

Discuss partitioning, rate limiting, retries, idempotency, and monitoring to handle large-scale crawling.

5. Discuss Trade-offs

Compare synchronous vs asynchronous processing, storage options, and crawling strategies (e.g., BFS vs DFS).

Key Points to Mention

  • Asynchronous job processing with a message queue (e.g., RabbitMQ, Kafka) to decouple API from workers.
  • Idempotency of POST /jobs to handle duplicate submissions safely.
  • Rate limiting and politeness policies (robots.txt, crawl-delay) to avoid overwhelming target sites.
  • Data model for storing job status and discovered images, with indexing for efficient retrieval.
  • Scalability considerations: horizontal scaling of workers, partitioning by domain, and backpressure handling.
  • Error handling and retries with exponential backoff for transient failures.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.