Inference Engineer

Hyperbolic Labs is democratizing AI by breaking down barriers to computing power via its Open-Access AI Cloud, a GPU marketplace and AI inference service built on idle compute across the globe. This role builds inference capabilities on top of "Forge", the company's unified control plane, so customers can consume model tokens without managing GPUs.

Responsibilities

• Deploy models across globally distributed, heterogeneous-hardware clusters on Forge/Kubernetes
• Evaluate inference frameworks and set up monitoring, gateways, and endpoints
• Optimize inference through autoscaling and KV-cache orchestration
• Debug customer-facing inference issues end to end

What we are looking for

• Strong general inference background across the full request-to-token stack
• Deep hands-on Kubernetes production experience
• Solid grasp of TTFT, disaggregated inference, speculative decoding, and KV cache internals
• Working knowledge of NVIDIA Dynamo and modern inference/serving frameworks

Additional Information

Based in San Francisco, CA. Preferred experience includes inference optimization (quantization, batching, kernel tuning) and prior GPU cloud/inference-provider work.