Senior Data Engineer – Data Products & AI
# Senior Data Engineer – Data Products & AI **HealthboxHR** · London · `On-site` 🕒 **Статус:** *Опубликовано: сегодня* · *Источник: Indeed* --- ### About the Role Employment Type Full Time Application Start Date 06-10-2026 Application End Date 06-11-2026 Location London Work Style Hybrid Industry Biotechnology/pharmaceuticals Description About Beacon Intelligence Beacon Intelligence delivers data and insights that help R&D scientists develop better pharmaceuticals, faster than ever before. Our focus is simple: we curate high-accuracy data enriched and transformed with AI. This is delivered through a range of platforms, increasingly via AI-native products and features that enable customers to extract insights faster than ever before. This is a rare opportunity to make a meaningful impact on patients’ lives worldwide. The Role As a Senior Data/AI Engineer, you will play a central role in building and scaling our data products. This is not a traditional back-end data engineering role. You will work closely with product, commercial and technical colleagues, transforming datasets into commercial, customer-facing solutions. You will combine deep technical expertise with a strong understanding of how data can be structured, enriched and operationalised through AI systems. What You’ll Be Doing Data Product Development (Core Focus) - Design and build scalable data products underpinning Beacon’s commercial offerings - Transform raw and third-party data into structured, enriched, product-ready datasets - Partner with product and commercial teams to define how data is packaged, accessed and monetised - Enable delivery via APIs, internal tools and customer-facing platforms AI & Data Enrichment - Apply AI and LLM capabilities to enrich and enhance data (e.g. classification, tagging, summarisation, insight generation) - Design and build LLM-powered product features - Integrate RAG into pipelines to improve data quality and unlock new features - Support development of AI-driven products such as recommendation engines, search and insight tools - Ensure systems are optimised, well-governed and resilient Platform & Engineering - Support deployment and lifecycle management of data and AI systems - Own and evolve the data platform architecture (Databricks, Azure, Airbyte) to support scalabilit - Own performance and cost optimisation across Azure Databricks and supporting Azure services, including compute selection, autoscaling, workload monitoring and resource-efficiency improvements - Build and maintain robust data pipelines (batch and streaming) delivering reliable, production-ready datasets - Ensure high standards in data quality, testing and observability - Improve efficiency through automation, CI/CD and engineering best practice Commercial & Stakeholder Impact - Partner with senior stakeholders to identify high-value data product opportunities - Translate business needs into practical, scalable data and AI solutions - Act as a bridge between technical teams and commercial/product stakeholders - Contribute to prioritisation based on business impact Leadership & Growth - Shape the evolution of our data product and AI strategy - Mentor junior engineers and promote a strong engineering and product mindset - Help build a high-performing, commercially aware data and AI engineering team What We’re Looking For: Core Engineering Experience - Deep, hands-on experience designing, building and operating production data platforms using Azure Databricks and Azure Cloud environments - Strong experience with Apache Spark, PySpark, Spark SQL and Delta Lake, including incremental processing, schema evolution, performance optimisation and batch and streaming workloads. - Strong experience implementing Unity Catalog, including catalog and schema design, access control, data lineage, discovery and governance of production data assets. - Proven experience designing lakehouse and Medallion architectures and converting raw and third-party data into governed, reusable data products. - Advanced Python and SQL, focused on production-quality, scalable solutions - Proven experience designing and building data pipelines and models - Experience working with APIs and data delivery mechanisms (critical for productisation) - Experience deploying Databricks workloads through automated CI/CD, preferably using Azure DevOps or GitHub Actions, automated testing and Databricks Declarative Automation Bundles. - Experience integrating Azure Databricks with ADLS Gen2, Azure identity, Key Vault, APIs and external data sources. Data Product Mindset (Key Differentiator) - Experience building or supporting data products or customer-facing data solutions - Strong understanding of how to structure, expose and scale data for end users or clients - Ability to think beyond pipelines and focus on commercial value - Experience working closely with product or commercial teams AI Engineering - Familiarity with generative AI / LLMs (Azure OpenAI, LangChain etc.) and experience applying them to real-world use cases - Experience evaluating, monitoring and optimising AI systems - Experience working with unstructured data (text, documents, web data) - Familiarity with embeddings, vector databases or semantic search - Understanding of AI governance and data ethics Desirable - Experience with Airbyte or modern data ingestion tools - Experience with Terraform or infrastructure-as-code - Exposure to LLM frameworks (e.g. LangChain, LlamaIndex) - Experience with experimentation or A/B testing frameworks - Background in building data-driven / AI-enabled products - Domain experience in life sciences - Exposure to Sigma, or similar cloud-based business intelligence and analytics platforms, would be advantageous. How You Work - Focused on shipping products, not just experimentation - Think in terms of products, not just pipelines - Commercially aware and motivated by impact - Combine technical excellence with pragmatism and pace - Collaborate effectively across disciplines and influence stakeholders - Curious about AI and emerging technologies, with a focus on reliable, scalable delivery Why Join Beacon Intelligence - Build products, not just infrastructure – directly helping scientists treat and cure diseases - Shape our AI future – define how we embed AI into products and workflows - High-impact role – influence product strategy and commercial performance - Modern, evolving stack – work with cutting-edge tools across data engineering and AI - Growth and ownership – join early and grow as we scale our capabilities - Collaborative environment – stakeholders who actively value data and insight Location & Working Pattern - London (hybrid working) – 2 days per week minimum in office - Flexible working arrangements available Time Spent (Directional) - 60% hands-on technical · 20% management and mentoring · 20% exploration Apply If you’re excited about turning data into products and building AI-enabled solutions that drive real commercial value, we’d love to hear from you.
Наблюдалась 2026-10-06, впервые 2026-10-06, источник — Indeed.