Design an LLM-Based Contributor Evaluation Platform

Published 3 October 2026

System Design Round — LLM-Based Contributor Evaluation Platform

The system design question asked me to design a platform that used an LLM to evaluate tasks completed by contributors. Because evaluations could take time and needed to run independently from the original request, the execution model was expected to be asynchronous and resembled a distributed job-processing system.

Evaluation Quality

The interviewer asked how the system could maintain reliable output quality when the underlying LLM produced inconsistent or unstable responses.

Failure Handling

The discussion covered how evaluation jobs should behave when model calls, workers, or downstream services failed.

Rate Limiting

I was also asked how the platform should control request volume when communicating with the LLM service.