American Express·AI Engineer·Technical Phone Screen
- How would you design an end-to-end evaluation framework for an AI system that uses LLMs or agents? Walk through both offline and online approaches, how you handle agent-specific metrics, regression testing across model versions, and where human eval fits in. Also discuss what can go wrong with these setups.
“This was basically the whole interview packed into one question.”