← mercor Interview Insights

mercor·Software Engineer·Onsite - System Design / Architecture·Senior

Senior
Jun 2026

Summary

Technical round at Mercor for an Infrastructure Engineer role. The whole session basically revolved around Kubernetes networking internals, which I thought I knew well enough until they started drilling into the specifics.

Questions Asked (4)

Q1

Walk me through how traffic is routed inside a Kubernetes cluster, from kube-proxy and its iptables or IPVS modes all the way to how a request actually lands on a pod.

System DesignTechnical Trade-offs
Author's notes

I started with kube-proxy and iptables rules, explained how rules get programmed per Service, then pivoted to IPVS as the more scalable option for large clusters.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem: a client sends a request to a Service, which is a stable virtual IP. Then explain how kube-proxy programs the data plane (iptables or IPVS) to load-balance and DNAT the request to a backend pod. Finish by walking through the packet's journey, highlighting the role of conntrack and the differences between iptables and IPVS modes.

Pro tip: Mention that iptables mode uses random selection with O(1) complexity but can become slow with thousands of services, while IPVS uses hash tables and supports more load-balancing algorithms, making it more scalable. Also note that kube-proxy is not in the data path—it only programs rules.

1. Service and Endpoints

Explain that a Kubernetes Service provides a stable virtual IP (ClusterIP) and DNS name. The kube-proxy watches Services and Endpoints (or EndpointSlices) to know which pods back the service.

2. kube-proxy Modes

Describe the two main modes: iptables and IPVS. In iptables mode, kube-proxy creates NAT rules to randomly select a backend pod. In IPVS mode, it uses a virtual server with real servers and supports scheduling algorithms like round-robin.

3. Packet Flow: Client to Service

Walk through the packet: client sends to Service IP:port. The packet hits the host's network stack, where iptables/IPVS rules DNAT the destination to a chosen pod IP:port. Conntrack records the connection for return traffic.

4. Pod Ingress and Response

The packet is routed to the pod's network namespace (via veth pair, CNI). The pod responds, and the reply packet is un-DNATed by conntrack so it appears to come from the Service IP. The client sees a seamless connection.

5. Trade-offs and Edge Cases

Discuss trade-offs: iptables is simple but scales poorly with many services; IPVS is more scalable but adds complexity. Mention session affinity, externalTrafficPolicy, and how kube-proxy handles nodePort and loadBalancer services.

Key Points to Mention

  • kube-proxy is a controller that programs iptables/IPVS rules; it is not in the data path.
  • iptables mode uses random selection and is O(1) per packet but rule updates are O(n) and can be slow with many services.
  • IPVS mode uses hash tables, supports multiple load-balancing algorithms, and is more scalable for large clusters.
  • Conntrack (connection tracking) ensures return traffic is correctly un-DNATed.
  • CNI plugins (e.g., Calico, Flannel) handle pod-to-pod networking and often integrate with kube-proxy.
  • Service types (ClusterIP, NodePort, LoadBalancer) and externalTrafficPolicy affect how traffic reaches pods.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

What are the differences between ClusterIP, NodePort, and LoadBalancer service types, and when would you actually use each one?

System DesignTechnical Trade-offs
Author's notes

Pretty standard, covered the basics fine.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining each service type in terms of its scope (internal vs external) and how traffic reaches pods. Then explain the trade-offs in terms of exposure, cost, and operational complexity, and finally give concrete use cases for each. Emphasize that these are not mutually exclusive and often used together.

Pro tip: Mention that LoadBalancer typically provisions a cloud provider's external load balancer, which incurs cost and may require additional configuration for TLS termination or health checks. Also note that NodePort is often used for debugging or when you need direct access to a specific node, but it's rarely the final production solution.

1. Define ClusterIP

Explain that ClusterIP is the default service type, providing an internal virtual IP reachable only within the cluster. It's used for inter-service communication and is not exposed externally.

2. Define NodePort

Describe NodePort as a service that exposes the service on each node's IP at a static port (30000-32767). It allows external access via any node's IP and port, but lacks load balancing and requires managing node IPs.

3. Define LoadBalancer

Explain that LoadBalancer builds on NodePort and ClusterIP, and provisions an external load balancer (e.g., AWS ELB, GCP LB) that routes traffic to the service. It provides a single external IP and handles load balancing.

4. Compare trade-offs

Discuss trade-offs: ClusterIP is simple and secure but internal-only; NodePort is simple but exposes high ports and lacks LB; LoadBalancer is production-ready for external traffic but adds cost and cloud dependency.

5. Provide use cases

Give concrete scenarios: ClusterIP for microservice-to-microservice calls; NodePort for dev/test or when you need direct node access; LoadBalancer for exposing a public-facing web app or API.

Key Points to Mention

  • ClusterIP is the default and only accessible within the cluster.
  • NodePort exposes a static port on each node's IP, allowing external access but without load balancing.
  • LoadBalancer provisions an external load balancer from the cloud provider, giving a single external IP.
  • LoadBalancer typically requires cloud provider integration and incurs additional cost.
  • These service types are often used together: LoadBalancer routes to NodePort, which routes to ClusterIP.
  • Ingress is an alternative to LoadBalancer for HTTP/HTTPS routing and can reduce cost by sharing one LB.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How does DNS-based service discovery work in Kubernetes? What resolves a service name to a cluster IP?

System DesignAPI & Integrations
Author's notes

Talked through CoreDNS, the cluster DNS suffix, and how pods get their resolv.conf configured.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the role of CoreDNS as the cluster DNS provider and how it watches the Kubernetes API for Service and Endpoint changes. Then describe the DNS resolution flow: a pod's DNS query for a Service name is resolved by CoreDNS to the Service's ClusterIP (for ClusterIP type) or to individual pod IPs (for headless services). Finally, mention the different DNS record types (A, SRV, PTR) and how they map to Kubernetes objects.

Pro tip: Emphasize that DNS-based service discovery is not just about resolving names to IPs but also about load balancing and failover, as kube-proxy handles the actual traffic distribution. Also, mention that CoreDNS can be customized with plugins and that DNS caching (e.g., ndots) can affect resolution behavior.

1. Introduce CoreDNS and its role

Explain that CoreDNS is the default DNS server in Kubernetes, deployed as a Deployment and exposed via a Service. It watches the Kubernetes API for Service and Endpoint objects to maintain DNS records.

2. Describe DNS record creation

Detail how CoreDNS creates A records for Services (pointing to ClusterIP) and for headless Services (pointing to pod IPs). Mention SRV records for named ports and PTR records for reverse lookups.

3. Explain the resolution process

Walk through a pod's DNS query: the pod's resolv.conf points to the kube-dns Service IP, which routes to CoreDNS. CoreDNS resolves the query based on its records and returns the IP(s).

4. Cover service types and their DNS behavior

Differentiate between ClusterIP, headless, and ExternalName Services. ClusterIP resolves to a single virtual IP; headless resolves to multiple pod IPs; ExternalName resolves to an external CNAME.

5. Mention kube-proxy and load balancing

Clarify that DNS returns the ClusterIP, and kube-proxy (iptables/IPVS) handles load balancing to backend pods. For headless services, the client is responsible for load balancing.

Key Points to Mention

  • CoreDNS as the default DNS provider and its watch on the Kubernetes API
  • DNS record types: A, SRV, PTR and their mapping to Services/Endpoints
  • ClusterIP vs headless Service resolution differences
  • The role of kube-proxy in load balancing after DNS resolution
  • Pod's DNS configuration (resolv.conf, ndots, search domains)
  • Custom DNS configurations (e.g., dnsPolicy, dnsConfig) and CoreDNS customization

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

Explain how an Ingress controller like nginx or traefik fits into the traffic path. How does external traffic actually reach a pod end-to-end?

System DesignTechnical Trade-offs
Author's notes

This was the one I felt best about.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Walk through the end-to-end traffic flow from the external client to the pod, highlighting where the Ingress controller sits and what it does at each hop. Emphasize that the Ingress controller is both a reverse proxy and a Kubernetes controller that watches Ingress resources and configures itself dynamically. Use a concrete example (e.g., nginx) to make the explanation tangible.

Pro tip: Mention that the Ingress controller itself runs as a pod (often behind a LoadBalancer Service) and that it routes based on host/path rules defined in Ingress objects, not by directly managing iptables. This shows you understand the separation between control plane and data plane.

1. External request hits the edge

Start with the client sending a request to a domain name that resolves to an external IP (e.g., a cloud load balancer). Explain that this IP is typically provisioned by a Service of type LoadBalancer or NodePort that fronts the Ingress controller pods.

2. Ingress controller receives and inspects

The Ingress controller (e.g., nginx) terminates TLS, inspects the HTTP host header and path, and matches them against Ingress rules it has programmed from the Kubernetes API. It then selects the appropriate backend Service.

3. Routing to the Service and Pod

The controller forwards the request to the Service's ClusterIP (or directly to pod IPs if using a service mesh or endpoint slicing). kube-proxy or the CNI then load-balances to one of the healthy pod endpoints backing that Service.

4. Pod handles the request

The request arrives at the pod's container, which processes it and sends a response back along the same path. Mention that the Ingress controller may also add headers like X-Forwarded-For to preserve client IP.

5. Dynamic updates and health

Explain that the Ingress controller continuously watches Ingress, Service, and Endpoint objects, so when pods scale or change, it updates its configuration without manual reloads. This ensures high availability and zero-downtime deployments.

Key Points to Mention

  • Ingress controller is a reverse proxy and a Kubernetes controller that watches Ingress resources.
  • It runs as a pod (or set of pods) typically exposed via a LoadBalancer or NodePort Service.
  • Routing decisions are based on host and path rules defined in Ingress objects.
  • Traffic flows: Client → LoadBalancer → Ingress Controller Pod → Service → Pod.
  • The controller dynamically updates its configuration when Ingress/Service/Endpoints change.
  • TLS termination and header manipulation (e.g., X-Forwarded-For) are common responsibilities.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.