Software Engineer - Model API Platform

What you will do

Own a platform area end to end

• Own one or more major parts of the platform, such as the WebSocket streaming layer, a speech recognition pipeline, or text to speech pipeline: build, operate and improve them.

• Turn loosely defined goals into RFCs, milestones and shipped results, and write down the decisions and trade-offs.

• Be accountable after launch: on-call, incident response, root-cause analysis and follow-through on fixes.

• Make the call when a decision is needed, and own the outcome.

• Obsessed with latencies, dissecting existing services to understand the system, and tuning the right knobs to optimize performance.

• Bring services across without regressions in customer-facing APIs, latency or reliability.

Build real-time and model-serving pipelines

• Build and operate streaming infrastructure: WebSocket connections, long-lived sessions queue-based batch processing, backpressure and low-latency model invocation.

• Engineer audio pipelines: TTS chunking around model context limits, sample-rate conversion and format encoding, and VAD-based silence detection for ASR.

• Hold your services to sub-100ms latency SLOs in a multi-tenant environment, and know where every millisecond goes.

Think in systems and from first principles

• Reasons from constraints: latency budgets per hop, throughput, cost per request, failure modes and tenant isolation.

• Use back-of-the-envelope estimates to decide what to build, what to optimize and what to leave alone.

• Prefer simple designs you can fully explain over clever ones you cannot.

• Build in observability from day one: logging, metrics and tracing that let anyone debug production quickly.

Raise the bar for the team

• Strengthen the integration testing harness so the team can ship fast without breaking customer-facing APIs.

• Partner with the Inference and MLOps teams on making model serving fast, reliable and operable.

• Mentor junior engineers through RFC reviews, code review and pairing.

• Practice design-first, documentation-first engineering: RFCs before code, no tribal knowledge.

What we are looking for

• 4–6 years of building and operating production backend systems.

• At least one service you owned end to end under real load, including what broke and what you changed.

• Strong distributed-systems fundamentals: concurrency, networking, queues, caching, consistency and failure modes.

• Experience with low-latency or high-throughput services, and comfort reasoning about p95 and p99 latency.

• Solid fundamentals with a relational database and a cache, such as PostgreSQL and Redis: schema design, query performance and caching strategy.

• Fluency in at least one backend language and willingness to work in both Python and Golang.

• Experience running services with containers, CI/CD and production monitoring.

• Daily, hands-on use of AI tools in your engineering workflow, with concrete examples of what you automated or sped up.

• A bias for action: fast on reversible decisions, careful on irreversible ones.

• Clear writing, especially RFCs and design docs that others can act on.

Good to have (not required)

• Voice AI or real-time audio experience: WebSockets, WebRTC, streaming ASR or TTS, or audio processing with FFmpeg or pyav, codecs, sample rates and VAD.

• Production experience in Go, or in Python with FastAPI.

• Experience serving LLM, ASR, TTS or vision models in production.

• Experience building metering, billing, wallet or payments infrastructure.

• Running services on Kubernetes in production.

• Time at an early- or growth-stage startup.

Why Sarvam

• A fast-moving, high talent-density team building full-stack AI for India, with real population-scale impact.

• Researchers, engineers, builders and business leaders who move fast and hold each other to a very high bar.

• High ownership and high impact from day one.

• AI-first in everything, from how we build and ship to how we think about problems