Big Data Engineer (Databricks)
- Experience with Databricks Asset Bundles (DAB).
- Experience using Azure DevOps.
- Knowledge of incremental processing and schema evolution.
- Healthcare or Insurance industry experience.
- Experience with cloud-based data platforms.
- Develop and maintain data pipelines using Databricks.
- Support the design and optimization of Delta Lake tables.
- Work with Unity Catalog to support data governance and access management.
- Participate in CI/CD processes using Azure DevOps and Databricks Asset Bundles (DAB).
- Implement incremental data processing and support schema evolution requirements.
- Collaborate with team members to troubleshoot data quality and performance issues.
- Assist with data modeling activities and platform optimization efforts.
- Document solutions and follow engineering best practices.
- Ideally 5 YOE
- Strong expertise in Apache Spark, including delivery on Databricks
- Azure Databricks certifications.
- Exposure to Medallion Architecture.
- Data quality and monitoring framework
openqareer не кадровое агентство: вакансия собрана с открытого источника, отклик подаётся на сайте работодателя.