Principal AI Engineer

We are seeking an exceptional Principal AI Engineer to lead the technical architecture and engineering of production-grade AI systems, advanced AI infrastructure, and scalable backend platforms.

This is a senior individual contributor role for an experienced engineering professional with a strong track record of designing, building, and deploying enterprise-scale AI solutions. The ideal candidate will combine deep expertise in applied AI and Large Language Models (LLMs) with strong distributed systems, backend engineering, and cloud infrastructure capabilities.

You will establish technical standards, make critical architectural decisions, and develop scalable, reliable, and secure AI solutions for high-impact government applications.

Candidates must have at least 15 years of overall professional experience, including substantial hands-on experience delivering production AI systems and leading complex technical architectures.

Key Responsibilities

• Define and lead cross-system architecture for enterprise AI platforms, backend services, and distributed systems.
• Design and optimize GPU-based LLM inference and serving infrastructure using technologies such as vLLM, TGI, or TensorRT-LLM.
• Architect production-grade Retrieval-Augmented Generation (RAG) systems, vector search, reranking, grounding, and agentic AI solutions.
• Build scalable backend services, APIs, event-driven architectures, and high-performance data pipelines.
• Design and optimize PostgreSQL databases, Kubernetes deployments, and cloud-native solutions on Microsoft Azure.
• Establish AI evaluation frameworks, regression detection, observability, and production monitoring to ensure reliability and consistent model performance.
• Make strategic build-versus-buy decisions for models, AI frameworks, serving infrastructure, and platform technologies.
• Lead technical design reviews, resolve complex production issues, and establish engineering best practices.
• Mentor senior and staff engineers, strengthen technical capabilities, and contribute to senior-level technical hiring.
• Ensure AI systems meet stringent security, reliability, performance, and data sovereignty requirements.
• Support bilingual AI applications operating effectively in both Arabic and English.

Mandatory Requirements

• 15+ years of overall professional experience in software engineering, backend development, AI engineering, or related technical disciplines.
• Demonstrated experience designing and delivering complex, production-grade AI systems.
• Strong expertise in Python and its production engineering ecosystem.
• Hands-on experience with Generative AI, LLMs, RAG architectures, AI agents, and model evaluation.
• Strong understanding of distributed systems, microservices, event-driven architectures, API design, and system reliability.
• Production experience with PostgreSQL, including database performance optimization and large-scale data workloads.
• Experience with cloud platforms, preferably Microsoft Azure, and container orchestration using Docker and Kubernetes.
• Practical experience with LLM inference, model serving, performance optimization, or AI infrastructure.
• Working proficiency in at least one additional programming language, such as Go, Rust, Java, or Kotlin.
• Demonstrated ability to make architecture decisions, lead complex engineering initiatives, and influence senior technical teams without relying on formal managerial authority.
• Recent hands-on coding and system design experience, including the use of AI-assisted development tools.

Preferred Qualifications

• Experience deploying production RAG systems covering ingestion, embeddings, vector search, reranking, and retrieval evaluation.
• Familiarity with LangGraph, LangChain, Microsoft Agent Framework, OpenAI Agent SDK, or Semantic Kernel.
• Experience with vector databases such as pgvector or Qdrant.
• Experience with GPU infrastructure and inference optimization using vLLM, TGI, or TensorRT-LLM.
• Knowledge of Kafka, Azure Service Bus, or Azure Event Hubs.
• Experience with Redis, Azure Blob Storage, Terraform, and cloud infrastructure automation.
• Familiarity with observability tools such as Langfuse, Grafana, and Prometheus.
• Experience building Arabic-language AI, voice AI, or conversational AI systems.
• Experience delivering AI solutions in government, financial services, defence, or other regulated environments.
• Experience mentoring senior engineers and establishing engineering standards across multiple teams.

Important Note

This is a hands-on Principal-level engineering position, not a people-management or research-only role. Candidates must demonstrate both substantial engineering experience and practical expertise in production AI systems.

Applicants who do not meet the mandatory 15+ years of overall professional experience requirement will not be considered.

Pay: From AED15,000.00 per month

Application Question(s):

• Do you have strong hands-on Python development experience in production environments?

Experience:

• Software engineering, backend development, AI engineering?: 10 years (Required)

Work Location: In person