MS Data Engineer

Job Purpose:

The MS Data Engineer is responsible for maintaining, optimizing, and supporting data platforms, data pipelines, and relational database environments for managed services clients. The role combines data engineering with hands-on database administration across Oracle, Microsoft SQL Server, MySQL, and PostgreSQL, whether hosted on cloud IaaS (virtual machines) or on cloud-native PaaS services such as Amazon RDS, Azure SQL Database, and Azure SQL Managed Instance. The role focuses on operational excellence, system stability, continuous improvement, SLA adherence, and proactive monitoring of data and database environments across cloud platforms (Azure, AWS, GCP).

Key Accountabilities:

Data Engineering & Operations (Managed Service Focus):

• Monitor, maintain, and support existing data pipelines, ETL/ELT processes, and data platforms for managed services clients.
• Ensure high availability, performance, and reliability of data systems across cloud environments (Azure, AWS, GCP).
• Perform ongoing optimization and tuning of data workflows and workloads to improve efficiency, reliability, and cost-effectiveness.
• Troubleshoot and resolve data pipeline failures, integration issues, and performance bottlenecks within defined SLAs.
• Implement enhancements and minor change requests within client-managed environments, following agreed change management processes.
• Maintain secure data environments, ensuring compliance with data governance, regulatory, and cross-border data requirements.
• Manage incident response, root cause analysis (RCA), and preventive action planning for data platform incidents.
• Support backup, disaster recovery, and business continuity processes for data platforms.
• Collaborate with cloud, security, and infrastructure teams to ensure seamless operations.

Database Administration (Oracle, SQL Server, MySQL & PostgreSQL):

• Administer Oracle, Microsoft SQL Server, MySQL, and PostgreSQL databases hosted on cloud IaaS virtual machines (Azure, AWS, GCP) for managed services clients, covering installation, configuration, patching, version upgrades, and routine maintenance.
• Design, implement, and regularly test backup and recovery strategies (full, differential/incremental, and point-in-time recovery) in line with client RPO/RTO requirements.
• Configure and operate high availability and disaster recovery solutions, including SQL Server Always On Availability Groups and failover clustering, Oracle Data Guard / RAC, MySQL replication and clustering, and PostgreSQL streaming replication.
• Perform proactive performance tuning: query and execution plan analysis, index and statistics maintenance, wait and lock analysis, memory and storage configuration, and capacity planning.
• Manage database security: user and role administration, least-privilege access, encryption at rest and in transit (e.g., TDE), auditing, and vulnerability remediation.
• Automate routine DBA tasks (health checks, maintenance jobs, alerting) using T-SQL, PL/SQL, shell/PowerShell, and Python.
• Support database migrations, consolidations, and upgrades between cloud IaaS and PaaS platforms, and across cloud providers, with minimal downtime.

Cloud PaaS Database Services (Amazon RDS & Microsoft Azure)

• Provision, configure, and operate Amazon RDS instances (Oracle, SQL Server, MySQL, PostgreSQL) and Amazon Aurora, including Multi-AZ deployments, read replicas, parameter and option groups, maintenance windows, and engine version upgrades.
• Manage RDS backup and recovery (automated backups, manual snapshots, point-in-time restore, cross-region copies) and monitor health and performance using Amazon CloudWatch, Performance Insights, and Enhanced Monitoring.
• Administer Azure PaaS databases: Azure SQL Database (single databases, elastic pools, Hyperscale), Azure SQL Managed Instance, Azure Database for MySQL – Flexible Server, and Azure Database for PostgreSQL – Flexible Server.
• Configure Azure database resilience and continuity features such as zone redundancy, active geo-replication, auto-failover groups, long-term backup retention, and geo-restore.
• Optimize PaaS database performance and cost through service tier and compute sizing (vCore, DTU, serverless), Query Store, automatic tuning, and reserved capacity recommendations.
• Secure PaaS databases using Microsoft Entra ID / AWS IAM authentication, private endpoints and VNet/VPC integration, firewall rules, Microsoft Defender for SQL, auditing, and data masking.
• Plan and execute migrations to cloud PaaS using Azure Database Migration Service, AWS Database Migration Service (DMS) with Schema Conversion Tool (SCT), and native backup/restore methods.

Data Warehouse & Data Lake Support:

• Maintain and optimize data warehouses and data lakes in cloud environments (e.g., Azure Synapse Analytics / Microsoft Fabric, Amazon Redshift, Google BigQuery).
• Support schema updates, partition management, indexing strategies, and performance improvements.
• Ensure data integrity, accuracy, and consistency across source databases, pipelines, and analytical stores.
• Monitor storage utilization and implement optimization strategies.

Monitoring, Governance & Compliance:

• Implement and maintain monitoring and alerting for data pipelines, data platforms, and database instances (e.g., Azure Monitor, Amazon CloudWatch, and third-party tools).
• Create automated validation checks to ensure data quality and accuracy.
• Maintain documentation of data architecture, database inventories, workflows, runbooks, and operational procedures.
• Ensure compliance with client SLAs, internal policies, and security standards.

Reporting & Analytics Support:

• Support BI teams in maintaining dashboards (Power BI, Tableau, QuickSight, etc.).
• Troubleshoot data source connectivity and refresh issues.
• Assist in performance optimization of reporting queries and datasets.

Job Specification:

Education:

Bachelor’s degree in computer science, information

technology, data engineering, or any related field

Professional Certification:

• Microsoft Certified: Azure Database Administrator Associate (DP-300) – strongly preferred
• Microsoft Certified: Fabric Data Engineer Associate (DP-700) or equivalent cloud data engineering certification
• AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect – Associate
• Oracle Database Administration Certified Professional (OCP)
• ITIL 4 Foundation
• MySQL and PostgreSQL certifications (e.g., Oracle MySQL Database Administrator, EDB PostgreSQL) are an advantage

Working Experience:

• 5–8 years of combined experience in database administration and data engineering, ideally within a managed services or enterprise operations environment
• Minimum 3 years of hands-on DBA experience across Oracle, Microsoft SQL Server, MySQL, and PostgreSQL in cloud-hosted production environments (IaaS and PaaS)
• Demonstrable hands-on experience administering Amazon RDS (including Aurora) and Azure PaaS databases (Azure SQL Database, Azure SQL Managed Instance, Azure Database for MySQL and PostgreSQL Flexible Server)
• Strong hands-on experience with cloud data services (Azure Data Factory, Synapse / Microsoft Fabric, AWS Glue, BigQuery, etc.)
• Experience supporting multiple clients under SLA-driven managed services contracts

Knowledge:

• Advanced proficiency in SQL (T-SQL, PL/SQL, MySQL and PostgreSQL dialects) and scripting languages (Python preferred; PowerShell, Bash).
• Deep understanding of database internals across the supported platforms: storage engines, transaction isolation, locking and concurrency, indexing, execution plans, and memory management.
• Strong knowledge of backup/recovery, high availability, and disaster recovery architectures for both IaaS-hosted and PaaS databases.
• Working knowledge of cloud networking and security as applied to databases (VNet/VPC, private endpoints, Microsoft Entra ID / AWS IAM, encryption, key management).
• Experience with ETL/ELT processes and data pipeline troubleshooting.
• Familiarity with monitoring tools and incident management / ITSM systems (e.g., ServiceNow, Jira Service Management).
• Familiarity with infrastructure-as-code and automation (Terraform, Bicep/ARM, CloudFormation) and CI/CD for database changes is an advantage.
• Understanding of data governance, security, and compliance principles, including ISO 27001 and data residency / cross-border requirements.
• Experience working under SLAs and structured support frameworks (ITIL-based incident, problem, and change management).

Compliance:

Comply with the company’s Information Security Management System (ISMS) and ISO 27001 requirements and actively participate in regular training and audits to ensure continuous alignment with the standard.

Key Expectations :

• Embrace a dedication to excellence and actively seek opportunities for improvement and innovation.
• Contribute to a positive and collaborative environment that encourages effective teamwork and support.
• Collaborate with colleagues to adapt and achieve shared goals, driving success together.
• Serve as a role model for peers by embodying the Bespin Motto and upholding the behaviors and ethics that define our organization.
• Help create a vibrant and inspiring workplace for all employees.