ByteDance·Machine Learning Engineer·Onsite - System Design / Architecture
- What are the main problems and failure modes you'd expect when building an LLM agent system that relies heavily on tool use in production?
- How would you model and persist an agent's working state across a long sequence of tool calls?
- How do you handle context window limits, partial failures, and rollback in a multi-step agent task?
- What does end-to-end evaluation look like for an agent that takes multi-step actions, and how do you attribute errors to the model versus the tools versus the orchestration layer?
- What offline benchmarks and online metrics would you use for an LLM agent system, and how do you deal with non-determinism when evaluating it?
“I started listing the obvious stuff like latency and cost, but they pushed back pretty fast and wanted me to go deeper.”