AI Infra Design

The infrastructure underneath models: GPU scheduling, inference serving, and moving weights and data around a fleet of accelerators.

• GPU inference scheduler — problem statement only so far.
• Document review inference system — problem statement only so far.
• Fast model distribution across GPU workers — problem statement only so far.