Recommender Assurance Lead

San Jose

Regular

R&D - Security

Job ID: A154898

to this job

Responsibilities

About the team: Content Assurance builds and operates the technical frameworks used to test whether high-impact content systems behave within declared safety, quality, and governance boundaries. The team works independently from the product engineering organizations that build the systems it evaluates and produces evidence that can withstand executive, regulatory, and audit scrutiny. Content Assurance team partners with Trust & Safety, Product, Engineering, Legal, Governance, Risk & Compliance, and independent validation teams. We do not set content policy or make product trade-off decisions. We translate approved objectives and obligations into repeatable tests, measurable controls, and defensible assurance outcomes. About the Role The Recommender Assurance Lead will own the assurance framework for recommendation systems, including retrieval, candidate generation, ranking, re-ranking, and related optimization layers. This leader will establish how USDS JV independently determines whether recommender systems operate within committed bounds for relevance, safety, diversity, freshness, fairness, consistency, and harm reduction before and after release. This is a senior technical leadership role. The successful candidate will set assurance strategy, direct the engineering and operational program, defend methodology, and issue clear assurance findings on high-risk recommender changes. The role does not own ranking algorithm design, content policy, engagement-versus-safety trade-off decisions, or production uptime. Responsibilities - Own the end-to-end recommender assurance framework, including control objectives, evaluation standards, release gates, evidence requirements, exception criteria, and post-release re-evaluation. - Define independent evaluation methods for retrieval and ranking systems, including counterfactual replay, ranking-quality benchmarks, harm-exposure sampling, cohort analysis, and policy-consistent ranking tests. - Establish measurable acceptance thresholds for relevance, safety, diversity, freshness, population fairness, consistency, and declared harm-reduction goals; require remediation or documented risk acceptance when systems fall outside committed bounds. - Lead pre-release assurance reviews for tiered recommender changes and issue evidence-backed verdicts that clearly state scope, findings, limitations, residual risk, and required actions. - Build the post-release assurance program, including scheduled re-evaluations, drift detection, harm-exposure monitoring, adverse-delta alerts, and incident forensics to determine whether ranking behavior remained within approved bounds. - Direct the development of evaluation code, benchmark datasets, replay workflows, sampling strategies, and assurance telemetry on shared testing platforms. - Set and report program measures, including assurance-gated release coverage, high-risk surface and cohort sampling coverage, evaluation freshness, and post-release harm-exposure deltas. - Consume intervention-effectiveness results where they affect ranking outcomes while maintaining a strict boundary between recommender behavior assurance and intervention-firing assurance. - Partner with Product Engineering, Trust & Safety, Legal, GRC, and independent validation teams to convert approved objectives into testable requirements without assuming ownership of product design or policy decisions. - Lead, develop, and hold accountable a multidisciplinary team of assurance engineers, applied scientists, and technical program owners; set technical direction and raise the quality of written findings and evidence. - Represent recommender assurance in executive reviews, audits, investigations, and regulatory inquiries, and defend the methodology and evidence with precision.

Qualifications

Minimum Qualifications - Bachelor’s degree in Computer Science, Machine Learning, Statistics, Data Science, or a related technical field, or equivalent practical experience. - 8+ years of relevant experience in machine learning, recommender systems, algorithm evaluation, AI assurance, integrity, or a closely related field, including experience leading senior technical teams or programs. - Deep understanding of modern recommendation architectures, including retrieval, ranking, re-ranking, embeddings, multi-objective optimization, experimentation, and online/offline evaluation. - Demonstrated ability to design statistically sound evaluation methods, sampling strategies, benchmarks, and monitoring for large-scale production ML systems. - Experience translating safety, policy, legal, or risk requirements into objective technical controls and defensible pass/fail criteria. - Strong executive communication and written judgment; able to make firm calls under ambiguity and explain evidence, limitations, and residual risk. Preferred Qualifications - Advanced degree in Machine Learning, Computer Science, Statistics, or a related field. - Experience with counterfactual evaluation, causal inference, fairness measurement, adversarial testing, or harm-exposure analysis. - Experience building assurance or governance programs subject to independent validation, regulatory review, or audit. - Hands-on proficiency with Python and large-scale data or ML evaluation systems; familiarity with production languages and distributed data platforms is a plus. - Experience in Trust & Safety, platform integrity, content governance, or high-risk consumer technology.

Job Information

【For Pay Transparency】Compensation Description (Annually)

The base salary range for this position in the selected city is $147200 - $374400 annually.

Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units.

Benefits may vary depending on the nature of employment and the country work location. Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, wellbeing benefits, among others. Employees also receive 10 paid holidays per year, 10 paid sick days per year and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

The Company reserves the right to modify or change these benefits programs at any time, with or without notice.

For Los Angeles County (unincorporated) Candidates:

Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state, and local laws including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Our company believes that criminal history may have a direct, adverse and negative relationship on the following job duties, potentially resulting in the withdrawal of the conditional offer of employment:

1. Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;
2. Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems; and
3. Exercising sound judgment.

About USDS

TikTok USDS Joint Venture LLC is dedicated to the safety and security of millions of Americans who create, discover, and connect with what they love on the apps we operate. The Joint Venture has been established in compliance with the Executive Order signed by President Trump on September 25, 2025. Our foundation is a comprehensive data privacy and cybersecurity program we operate under defined safeguards to protect national security and secure U.S. user data, apps and the algorithm. We safeguard the U.S. content ecosystem, holding decision-making authority for trust and safety policies and moderation. USDS Joint Venture helps ensure Americans can continue to express their creativity, discover new hobbies and interests, and build thriving communities and businesses on a global scale.

On-site presence across teams allows the company to operate with greater speed, alignment, and agility — especially in areas like real-time decision-making, team development, and integrated execution. As such, the company is shifting from a hybrid work model to a fully in-person schedule up to 5 days a week.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

USDS Reasonable Accommodation

USDS is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, pregnancy, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or a reasonable accommodation, please reach out to us at https://tinyurl.com/USDS-RA