Databricks SME
# Databricks SME **Scicom Infrastructure Services, Inc.** · United States · `Remote` · `Contractor` 💼 **Уровень роли:** `Senior` 🕒 **Статус:** *Опубликовано: сегодня* · *Источник: Himalayas (JSON API)* --- ### Top Skills & Match 🎯 **Ключевой стек роли:** `[Databricks-Architecture]` `[Data-Platform-Engineering]` `[Cloud-Data-Architecture]` `[Lakehouse-Architecture]` `[Data-Engineering]` `[Databricks-SME]` `[Databricks]` `[Databricks-Specialist]` --- ### About the Role Key Responsibilities: Architecture & Platform Design - Design enterprise Databricks Lakehouse architectures aligned with the Databricks Well-Architected Framework - Define reference architectures for batch, streaming, analytics, and ML workloads - Select and standardize cluster, compute, and workspace architectures - Design multi-workspace strategies (dev/test/prod, shared vs. isolated) - Ensure architectures meet scalability, availability, and performance requirements Well-Architected Framework Alignment Apply Databricks best practices across all pillars, including: - Security & Governance (Unity Catalog, IAM, data access controls) - Reliability & Resilience (job retries, checkpointing, failure isolation) - Performance Efficiency (cluster sizing, autoscaling, caching) - Cost Optimization (compute policies, workload separation, monitoring) - Operational Excellence (monitoring, automation, CI/CD, runbooks) Implementation & Engineering - Lead Databricks workspace, cluster, and Unity Catalog implementations - Implement Delta Lake, Delta Live Tables (DLT), and Structured Streaming - Build and optimize ETL/ELT pipelines using Spark and SQL - Integrate Databricks with cloud services (S3/ADLS/GCS, IAM, Key Vault, networking) - Establish CI/CD pipelines for notebooks, jobs, and infrastructure Security, Governance & Compliance - Implement Unity Catalog for centralized governance - Define data classification, lineage, and audit strategies - Enforce least-privilege access and secure networking patterns - Support compliance requirements (HIPAA, SOC 2, PCI, GDPR as applicable) Operations & Optimization - Monitor platform health, performance, and cost - Troubleshoot production issues across jobs, clusters, and data pipelines - Perform workload tuning and cost-performance optimization - Define SLOs, alerts, and operational metrics Collaboration & Leadership - Partner with Data Engineering, Analytics, ML, Platform, and Security teams - Translate business requirements into technical architectures - Provide architectural guidance and technical mentorship - Communicate risks, tradeoffs, and recommendations to leadership Required Qualifications: Experience - 7+ years in data engineering, analytics, or platform architecture - 3–5+ years hands-on Databricks experience in production environments - Proven experience applying the Databricks Well-Architected Framework - Experience designing cloud-native lakehouse architectures - Experience supporting mission-critical data platforms Technical Skills - Databricks Lakehouse Platform - Apache Spark (PySpark / Scala / Spark SQL) - Delta Lake, Delta Live Tables, Structured Streaming - Unity Catalog (governance, lineage, access controls) - Cloud platforms: AWS, Azure, or GCP - Infrastructure as Code (Terraform strongly preferred) - CI/CD tools (GitHub Actions, Azure DevOps, GitLab, etc.) - Data formats and protocols (Parquet, JSON, Avro) Certifications Required: - Databricks Certified Data Engineer Professional - Databricks Certified Professional Architect (or equivalent advanced certification) Preferred / Additional Certifications - AWS Certified Solutions Architect (Associate or Professional) - Azure Solutions Architect Expert - Google Professional Data Engineer - Databricks Machine Learning Professional - Snowflake or other cloud data platform certifications Soft Skills - Strong architectural decision-making and documentation skills - Excellent communication with technical and non-technical stakeholders - Ability to lead design reviews and architecture governance forums - Strong troubleshooting and performance-tuning mindset Nice-to-Have Experience - MLflow and MLOps architectures - Real-time analytics and streaming pipelines - Multi-region or cross-account data architectures - Consulting or MSP delivery experience Originally posted on Himalayas
- Databricks-Architecture
- Data-Platform-Engineering
- Cloud-Data-Architecture
- Lakehouse-Architecture
- Data-Engineering
- Databricks-SME
- Databricks
- Databricks-Specialist
Наблюдалась 2026-09-29, впервые 2026-09-28, источник — Himalayas (JSON API).