К содержимому

Research Residency – Red Team

AI Security InstituteLondon, UK

  • Red Team

About the AI Security Institute

The AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We’re in the heart of the UK government with direct lines to No. 10 (the Prime Minister's office), and we work with frontier developers and governments globally.

We’re here because governments are critical for advanced AI going well, and UK AISI is uniquely positioned to mobilise them. With our resources, unique agility and international influence, this is the best place to shape both AI development and government action.

The deadline for applying to this role is Tuesday 8th December, end of day, anywhere on Earth

About the team

The Red Team conducts cutting-edge research to evaluate the safety measures of frontier AI systems, sharing our findings directly with leading AI companies, UK officials and allied governments. Our work shapes how the most powerful AI systems are deployed - you can read about it in the The New York Times , The Economist, and The Telegraph . Demand for this work is growing faster than we can currently meet - we're scaling to address that gap.

The Red Team’s work is crucial to ensuring advanced AI systems are developed and deployed safely. We are composed of three specialised sub-teams:

  • Alignment – developing methods to stress-test whether models reliably act as intended (e.g., our evaluation of GPT-6 Astra , and our research into cheating behaviour in frontier model evaluations )
  • Control – researching methods for stress-testing and evaluating the measures that identify and prevent agents taking harmful actions
  • Misuse - stress-testing safeguards against misuse of dangerous capabilities (e.g. our safeguard testing with Anthropic and OpenAI , and recent work on Boundary Point Jailbreaking , attacks on layered safeguard systems and principles for evaluating misuse safeguards )

While each team has its own focus, we share methodologies, infrastructure and tooling across the wider team.

Role Description

This is an exciting opportunity for motivated and talented researchers to join AISI's Red Team, contributing to priority research objectives within a fast-paced environment in government. We are looking for exceptional candidates currently studying for a PhD, or postdoctoral researchers who have submitted their PhD within the last 2 years, in machine learning, computer science, computer security or a related field. We will by default consider this a 6-month residency but are flexible to shorten or extend per candidate preferences and availability. Following an end of residency review, promising candidates may be offered the opportunity to join their preferred team in a full-time employee role.

We are particularly interested in candidates who have one or more of the following. You don't need all of them, and we'd encourage you to apply even if you only match a few:

  • Have research experience in adversarial machine learning, AI security, alignment, AI control or evaluations, or are familiar with this research
  • Have experience with optimisation methods such as reinforcement learning, fine-tuning or evolutionary methods
  • Are familiar with loss-of-control threat models, such as deceptive alignment, research sabotage or reward-seeking
  • Have hands-on experience red teaming or jailbreaking AI systems, or have taken part in capture-the-flag, bug bounty or other security challenges
  • Have built something with LLMs, such as an evaluation, an agent or a fine-tuned model
  • Have used ML frameworks such as PyTorch or Hugging Face
  • Have shared research or technical work publicly, for example a paper, preprint, open-source tool or blog post

What you’ll be doing

As a resident on one of our sub teams, you :

  • Work closely with research scientists to decide which questions matter most and how to test them
  • Work on a defined project with support from an experienced research engineer or scientist, for example new attack methods, automated red teaming, alignment evaluations, or attacks on AI monitors and agents
  • Help design and run attacks and evaluations on frontier AI systems to find weaknesses in their safety measures, such as jailbreaking safeguards, uncovering misaligned behaviour, or evading control monitors
  • Write code and help build tools to run experiments at scale, such as testing attack techniques across several models, with a focus on reliability, speed and reproducibility
  • Analyse experiment results carefully and turn them into clear findings for technical and policy audiences
  • Keep up with new attack techniques and the research literature, apply relevant insights to our work
  • Learn how frontier AI models are evaluated and red teamed in practice

Illustrative projects

The projects below are examples of the kind of work you might do. Our priorities change quickly, so your actual project may be different, and we're open to matching work to your interests and skills.

Misuse

  • Create and improve novel automated research approaches for developing better red-teaming algorithms, including evolutionary approaches like OpenEvolve and multi-agent approaches
  • Work on core misuse research, such as:
  • Jailbreaking and attack pipelines, for example Boundary Point Jailbreaking of Black-Box LLMs and STACK: Adversarial attacks on LLM safeguard pipelines
  • Backdoors and data poisoning, for example Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
  • Prompt injection and AI agent security, for example Breaking agent backbones

Alignment

  • Build methods that automate the identification of harmful or unsanctioned model behaviour, such as research sabotage or reward-seeking (see our work on research sabotage and cheating in evaluations )
  • Develop methods to improve the realism of simulated evaluations (for example, our simulated supply-chain attack evaluation of GPT-6 Astra )
  • Investigate, after the fact, what drove examples of misaligned behaviour in the wild (as in our investigation of AISI's July 2026 incident )

Control

  • Develop better ways to evaluate AI monitors , such as : working out how attacks on monitors can predict whether they’ll catch harmful agent behaviour before it happens in real deployments .
  • Develop our scientific understanding of the limitations of control evaluations: including both tractable problems with current evaluations, and fundamental challenges with the paradigm.

Person specification

You may be a good fit if you have some of the following skills, experience and attitudes:

  • Experience running experiments or analysing data, for example in coursework or projects
  • Good research habits, such as keeping work reproducible, recording what you tried and being honest about uncertainty
  • Curiosity about how systems fail, and a willingness to try unusual approaches
  • Clear written and spoken communication, including explaining technical work to non-specialists
  • Openness to feedback, and a willingness to ask for help when you're stuck
  • Motivation to reduce risks from advanced AI and to support government work on AI security

Highly Desirable

We additionally may factor in experience with particular areas like:

  • Published or co-authored research on adversarial ML, jailbreaks, data poisoning, AI safeguards, alignment, control or evaluations
  • Previous research experience in ML or AI safety, such as a research internship, a research assistant role or a substantial thesis project
  • A background in security research, such as finding and disclosing real vulnerabilities, or placing well in CTF competitions
  • Experience fine-tuning or modifying open-weight models, or working with model internals
  • Familiarity with the alignment or AI control literature, or with current methods for post-training and aligning LLMs
  • Experience with RL, supervised fine-tuning or other ML optimisation, including careful choices about what to measure
  • Experience in security analysis of agent scaffolds, sandboxes or ML infrastructure
  • Experience testing the security of AI agents, for example prompt injection or tool misuse

This might not be the role for you if:

  • You want to lead or set research strategy from day one. You'll have real influence over your project and will work with experienced research leaders, who set our agenda during this residency
  • You prefer well-defined requirements and long planning cycles. Our priorities shift as the landscape of frontier AI changes rapidly, and projects often run to tight deadlines
  • You are uncomfortable in a collaborative research environment and prefer extended periods of solo working

Motivated candidates are encouraged to apply even if they don't meet all of the above criteria.

Selection process

The interview process may vary from candidate to candidate, however, you should expect a typical process to include some technical proficiency tests, discussions with a cross-section of our team at AISI (including non-technical staff) and conversations with your team lead. The process will culminate in a conversation with members of the senior leadership team here at AISI.

Candidates should expect to go through  some or all of the following stages once an application has been submitted:

  • Initial assessment
  • Initial screening call
  • Technical assessment
  • Research interview
  • Final interview with members of the senior leadership team

What We Offer

Impact you couldn't have anywhere else

  • Incredibly talented, mission-driven and supportive colleagues.
  • Direct influence on how frontier AI is governed and deployed globally.
  • Work with the Prime Minister’s AI Advisor and leading AI companies.
  • Opportunity to shape the first & best-resourced public-interest research team focused on AI security.

Resources & access

  • Pre-release access to multiple frontier models and ample compute.
  • Extensive operational support so you can focus on research and ship quickly.
  • Work with experts across national security, policy, AI research and adjacent sciences.

Growth & autonomy

  • If you’re talented and driven, you’ll own important problems early.
  • 5 days off and annual stipends for learning and development, and funding for conferences and external collaborations.
  • Freedom to pursue research bets without product pressure.
  • Opportunities to publish and collaborate externally.

Life & family*

  • Modern central London office, or where applicable, option to work in similar government offices in Birmingham, Cardiff, Darlington, Edinburgh, Salford or Bristol.
  • Hybrid working, flexibility for occasional remote work abroad and stipends for work-from-home equipment.
  • At least 25 days’ annual leave, 8 public holidays, extra team-wide breaks and 3 days off for volunteering.
  • Generous paid parental leave (36 weeks of UK statutory leave shared between parents + 3 extra paid weeks + option for additional unpaid time).
  • Excellent pension provision, with a choice between Partnership (no employee contributions required and up to 14.75% employer contributions) or Alpha, a valuable defined benefit scheme that provides a guaranteed, inflation-linked retirement income for life. Find out what benefits a Civil Service Pension provides (opens in a new window).
  • Discounts and benefits for cycling to work, donations and retail/gyms.

*These benefits apply to direct employees. Benefits may differ for individuals joining through other employment arrangements such as secondments.

Salary

Annual salary is benchmarked to role scope and relevant experience. Most offers land between £65,000 and £145,000 made up of a base salary plus a technical allowance (take-home salary = base + technical allowance).

This role sits outside of the DDaT pay framework given the scope of this role requires in depth technical expertise in frontier AI safety, robustness and advanced AI architectures.

The full range of salaries are available below:

  • Level 3: £65,000–£75,000 (Base £40,850 + Technical Allowance £24,150–£34,150)
  • Level 4: £85,000–£95,000 (Base £48,540 + Technical Allowance £36,460–£46,460)
  • Level 5: £105,000–£115,000 (Base £63,165 + Technical Allowance £41,835–£51,835)
  • Level 6: £125,000–£135,000 (Base £76,475 + Technical Allowance £48,525–£58,525)
  • Level 7: £145,000 (Base £76,475 + Technical Allowance £68,525)

Additional Information

Use of AI in Applications

Artificial Intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance for more information on appropriate and inappropriate use.

Use of AI tools in recruitment

We may use AI tools to help our hiring team review applications, for example by summarising application materials and helping us prioritise them against the published criteria. We do not use AI tools to find information about you from other sources, including online. People make every decision. We do not make solely automated decisions about candidates. Find out more, including your rights, in our recruitment privacy notice: https://www.aisi.gov.uk/careers/privacy-policy.

Internal Fraud Database

The Internal Fraud function of the Fraud, Error, Debt and Grants Function at the Cabinet Office processes details of civil servants who have been dismissed for committing internal fraud, or who would have been dismissed had they not resigned. The Cabinet Office receives the details from participating government organisations of civil servants who have been dismissed, or who would have been dismissed had they not resigned, for internal fraud. In instances such as this, civil servants are then banned for 5 years from further employment in the civil service. The Cabinet Office then processes this data and discloses a limited dataset back to DLUHC as a participating government organisations. DLUHC then carry out the pre employment checks so as to detect instances where known fraudsters are attempting to reapply for roles in the civil service. In this way, the policy is ensured and the repetition of internal fraud is prevented. For more information please see - Internal Fraud Register.

Security

Successful candidates must undergo a criminal record check and get baseline personnel security standard (BPSS) clearance before they can be appointed. Additionally, there is a strong preference for eligibility for counter-terrorist check (CTC) clearance . Some roles may require higher levels of clearance, and we will state this by exception in the job advertisement. See our vetting charter here.

Nationality requirements

We may be able to offer roles to applicant from any nationality or background . As such we encourage you to apply even if you do not meet the standard nationality requirements (opens in a new window).

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants. The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window) . As part of the application process, we monitor statistics on D&I. You can see how we process this data here: Recruitment privacy notice - GOV.UK .

openqareer не кадровое агентство: вакансия собрана с открытого источника, отклик подаётся на сайте работодателя.