Product Engineer, Agent Experience

Avante is the enterprise AI platform for benefits teams. Benefits is one of the most complex, high-stakes functions in any company, and Avante’s purpose-built agents take on end-to-end benefits work, from checking vendor contracts against claims to preparing leadership reports to helping employees use and understand their benefits. Samsung Semiconductor, Real Chemistry, Zscaler and News Corp use Avante today to keep costs in check, support employees across the globe, and get more from their benefits investment.

The Role

We are hiring a Product Engineer to own the quality of our Agents end to end. Carly delivers personalized benefits guidance to employees, grounded in plan designs, coverage rules, and each employee's context. Ava is the agentic intelligence that benefits teams use to run their benefits operations. In this role, you will review evals, collect qualitative and quantitative feedback, define what a high-quality outcome looks like, ship improvements, and measure their impact.

The role combines product judgment with hands-on implementation. You will own decisions such as whether a problem calls for a prompt change, a new skill, or a code change; whether an evaluation is rigorous enough to trust; and whether a new model is ready for production. You will then implement those changes yourself across evaluations, prompts, skills, model configuration, and our TypeScript codebase, using AI development tools as a core part of your workflow, and partner with our agent engineers on deeper changes to retrieval, tooling, or the platform.

You will join our Agent Experience team, embedded with agent engineering and working closely with product leadership, design, and Customer Success. We welcome candidates from either direction: product-minded engineers, and engineering-minded product managers who are already building. We place more weight on what you have shipped and how quickly you learn than on years of experience, and we are open to candidates early in their careers who show high agency, curiosity, and sound judgment.

Success in this role means evaluation results consistently translate into shipped improvements, model and prompt changes are validated against representative customer tasks, recurring failures become regression cases, and the team can clearly explain how each change affects usefulness, reliability, latency, and cost.

What You'll Do

• Define what good looks like for our agents. Review production evals, customer feedback, usage patterns, and evaluation results. Find where an answer or action falls short, including cases where it's factually correct but still doesn't help the person asking.
• Build evaluations the team can trust. Turn customer needs into representative scenarios, clear rubrics, and regression checks. Test whether automated scores agree with human judgment, and use the results to decide what to fix first.
• Ship improvements yourself. Update prompts, skills, and agent behavior. Build prototypes and targeted product changes, and contribute reviewed code. Partner with engineers when the problem sits in retrieval, tools, or the underlying platform.
• Make the call on new models. Compare models against real tasks for quality, safety, latency, and cost. Decide with engineering whether we upgrade, run controlled rollouts, inspect production results, and follow through on regressions.
• Own the feedback loop with customers. Work with CX and product teammates to understand what people are trying to get done. Turn recurring friction into product changes, then check whether those changes improve task completion, engagement, and trust.
• Use AI tools as a core part of how you work. Build with AI coding tools and agents every day, and be ready to explain how you checked what they produced.
• Keep the team current. Try new AI tools and approaches, form a view on where they help, and share what you learn. Turn the experiments that work into changes that hold up in real use, with clear evidence and appropriate guardrails.

What We're Looking For

• Experience shipping digital products in product management, product engineering, software engineering, or a similarly hands-on role. You can point to customer outcomes you personally improved.
• Strong product judgment. You can look at an agent interaction, explain what the user needed, and tell a useful result from one that only looks good in a demo or scores well.
• Active, practical use of AI coding tools and agents. You've built prototypes, tools, or product changes with them and can explain how you checked their behavior and handled mistakes.
• Enough coding fluency to read an unfamiliar codebase, make targeted changes, use version control, and contribute reviewed code. You're comfortable learning our TypeScript stack.
• Comfort with data and experiments: querying or scripting an analysis, defining metrics that mean something, and recognizing when a sample or automated evaluation gives false confidence
• Systems thinking and customer empathy. You spot where a skill, an automation, or a workflow change would remove friction, and you can talk with customers about it.
• High agency in an ambiguous environment. You pick a useful next step, ship, learn quickly, and communicate clearly with engineers, designers, and customer-facing teams.

Nice to Have

• Experience with LLM evaluations, prompt and context design, or agent tools and orchestration
• Running controlled experiments on a live product
• Working knowledge of SQL and TypeScript
• Familiarity with benefits, healthcare, HR technology, or enterprise software

What to Expect from Us

Interview Process:

• Initial screen with our recruiters
• Video Interview with our Engineering Manager
• Technical interview focused on real-world problem solving (no whiteboard algorithms)
• On-site interview loop:
• Team behavioral interview (60 min)
• Session with our Engineering Manager (60 min)
• Final conversation with our CTO (30 min)

The base salary for this role is $115,000–$165,000 . Where you land in that range depends on your experience, your skills, and how well your background fits what the role needs.

We also offer:

• Equity: Stock options, so you share in what we’re building
• Health coverage: Medical, dental, and vision, with employee premiums covered by Avante
• Unlimited PTO: Take the time you need to rest and recharge, plus paid company holidays
• Retirement: 401(k)
• Commuter benefit: $150/month toward however you prefer to get to work

How We Work

Our four values aren't aspirational. They translate into concrete operating tenets that shape how we make decisions every day.

Beat Yesterday. We re-evaluate our own toolkit every quarter, because the workflows that made you effective three months ago may already be outdated. We act on 70% of the information rather than waiting for certainty. We timebox, execute, and iterate, getting work in front of real people quickly instead of polishing in isolation. And we own our output regardless of how we produced it. AI can get you 80% of the way there. Everything that goes out with your name on it is your responsibility to review, refine, and stand behind.

Embrace Type 2 Fun. We take on hard problems and find meaning in the work, even when it's uncomfortable. We assume good intent and follow up directly rather than venting in side channels. We celebrate wins loudly and specifically, naming the actual behavior so the whole team learns what great looks like. We are on by default, because we owe it to each other and to our customers to be available beyond what a regular corporate job would demand. And we do unscalable things today that build scalable advantages tomorrow: every hand-built account play and late-night proposal turnaround earns us the right to scale later.

Stay Hungry, Stay Curious. We expect everyone to be the expert on AI in their function before anyone asks them to be. The landscape changes weekly. When you learn something, teach it. Learn enough about adjacent functions to ask good questions. Ask the question you think is too basic, because in complex domains like benefits, the cost of a wrong assumption can be a lost account. And remember: curiosity without urgency is a hobby. Learning has to translate into action on a timeline that matches the opportunity.

Act Like an Owner. We say the uncomfortable thing early, raising missed commitments or quality gaps directly with the person within 48 hours. We surface problems when they're small, not the week they become a crisis. Customer-facing work gets priority, always. And we close the loop every time, because over-communication is a feature at our stage, not a bug.