Senior Applied ML Engineer, Evals & Data

You will study real agent runs, turn important failures into evaluation cases, and measure whether changes make the product better. You will also work with product and engineering to ship those improvements.

This is not a research-only, prompt-only, or QA role.

You’ll be working alongside a team of engineers who all care deeply about craft, including the founders. You like owning problems end to end, and you’d rather ship something great this week than something perfect next quarter

What you'll actually do

Define quality standards and build trusted evaluation datasets from real product usage.

Build offline and online evaluations, including automated checks and human review.

Analyze model and agent failure patterns, then improve quality through better data, evaluation methods, model selection, and, where useful, fine-tuning.

Add regression checks and release gates while tracking quality, latency, and cost.

Solve these hard problems.

What we are looking for

Experience shipping and operating an LLM or agent system used by real customers.

Strong software engineering skills in TypeScript or Python, with the ability to work across both.

Experience building evaluations, datasets, experiments, or AI quality systems.

Strong product judgment and the ability to turn unclear quality problems into measurable improvements.

You do not need a PhD or experience training foundation models. Evidence of building reliable AI products matters more than formal credentials or knowledge of a specific framework.

Nice to have

Experience with multimodal AI, video, media, or creative software.

Experience with human labeling, model graders, or fine-tuning.

Good knowledge of experiment design and statistics.

Within your first six months:

We have a trusted quality baseline for our main agent workflows.

Production failures regularly become new evaluation cases.

Important agent changes pass clear regression checks before release.

We can show measurable improvements in key editing workflows.

What you get

You'd be surrounded by people who are absurdly good at what they do. One started coding at 11 and shipped an app with 6M+ downloads in high school. One got into CS engineering at 14 and has been working on distributed systems for 8+ years. One's an ex-founder who took a company to 1.2M users and $300M+ in transactions. That's the team. We're looking for someone who'll raise the bar on technical craftsmanship and creative product quality. Apart from that you'd get:

Competitive salary and founding-team equity.

Unlimited tokens across every AI model. Use whatever you want, as much as you want.

A healthy budget for AI tools and any peripherals you need to do your best work.