Data Engineer III

Position Title:

Data Engineer III

Pay Rate :

$83,000 - $121,000

Multiple factors, including Individual experience, skills and abilities will determine where an employee is ultimately placed in the pay range.

Category/Shift :

Salaried Full-Time

Physical Location :

75 5th Street NW Atlanta, GA 30308 United States

The Job You Will Perform:

The Data Engineer is an important member of the Cost Innovation Team, which is a strategic, innovative, and cross-functional team that is focused on creating value for the company through the advanced analytics of manufacturing data. This position offers a flexible working arrangement provided that the home office is within a typical commuting distance to the A2C office.

Core Responsibilities

Analytics & Data Science Enablement

-

Partner with analysts, data scientists, engineers, and business stakeholders to translate functional requirements into scalable technical data solutions.

-

Deliver self-service, analytics-ready datasets that empower downstream teams to consume data without deep knowledge of underlying source systems.

-

Enable industrial analytics use cases by providing trusted, contextualized time-series, asset, event, and operational data for analytics platforms and applications.

-

Eliminate redundant data-preparation and contextualization efforts across disparate analytics and application teams.

-

Help establish standard data definitions, reusable data products, and consistent business context across enterprise and industrial analytics environments.

-

Support downstream analytics workflows, including advanced analytics, data science, visualization, and industrial process analytics.

Data Ingestion & Integration

-

Design, build, and maintain robust pipelines to ingest data from relational databases, operational systems, industrial historians, APIs, cloud platforms, and flat files.

-

Establish reliable and secure connections between source systems, cloud data platforms, and industrial data platforms.

-

Write optimized SQL queries and extraction logic for high-performance data retrieval.

-

Implement flexible ingestion patterns including batch, incremental, streaming, and replication-based approaches based on business latency requirements.

-

Integrate high-volume time-series data and industrial metadata while maintaining relationships between assets, measurements, events, and other operational entities.

-

Support ingestion and integration patterns used by platforms such as Cognite Data Fusion (CDF), Seeq, and Microsoft Fabric.

Data Storage & Lakehouse Engineering

-

Design and manage scalable storage structures across raw (bronze), intermediate (silver), and curated (gold) layers.

-

Leverage Spark-native formats and engines such as Delta Lake to support scalable analytics workloads.

-

Design curated data structures that support both enterprise analytics and industrial data consumption.

-

Enforce data retention, archiving, and purging policies to manage data lifecycles.

-

Monitor and optimize storage, compute costs, and cloud resource utilization.

Data Transformation

-

Develop Python and SQL-based transformations to clean, standardize, enrich, and unify data.

-

Develop, tune, and proactively optimize scalable PySpark applications to efficiently process high-volume datasets.

-

Build modular, reusable transformation components rather than siloed, application-specific logic.

-

Resolve complex data-quality anomalies including schema drift, missing values, duplicates, timestamp alignment, and inconsistent identifiers.

-

Engineer efficient incremental processing patterns to handle high-volume, frequently changing datasets.

-

Transform raw operational and historian data into analytics-ready time series, calculations, conditions, events, and contextualized datasets for downstream consumption.

-

Enable reusable data transformations that can support platforms such as Seeq, Cognite, Fabric, and custom analytics applications.

Semantic Modeling & Industrial Data Contextualization

-

Translate complex source schemas into intuitive, business-aligned semantic entities, asset hierarchies, and industrial data models.

-

Build and maintain relationships between assets, equipment, time series, events, files, and other operational data.

-

Standardize naming conventions, identifiers, metadata, and relationship mappings across source systems.

-

Perform data contextualization to connect operational data with relevant equipment, process, and business context.

-

Support asset-centric and knowledge-graph-style data models commonly used in industrial data platforms such as Cognite.

Pipeline Orchestration & Operations

-

Architect end-to-end workflows that seamlessly coordinate ingestion, transformation, contextualization, validation, and publication.

-

Orchestrate data movement between source systems, lakehouse environments, industrial data platforms, and analytics applications.

-

Implement resilient error handling including retries, automated recovery, checkpointing, and alerting.

-

Monitor pipeline health, SLA compliance, data freshness, and data drift using centralized observability tools.

-

Design pipelines capable of supporting both traditional enterprise data and high-frequency industrial time-series workloads.

Data Governance, Security & DevOps

-

Enforce data security and privacy standards, including row- and column-level security, RBAC (Role-Based Access Control), and masking of sensitive data.

-

Apply consistent access-control and governance patterns across enterprise data platforms and industrial analytics environments.

-

Embed automated validation checks as needed to enforce data quality.

-

Maintain Git-based development workflows using branching, code reviews, and pull requests.

-

Implement automated CI/CD pipelines for code testing, data validation, configuration management, and repeatable environment deployments.

-

Support deployment and lifecycle management of data pipelines, transformation logic, semantic models, and analytics configurations across development, test, and production environments.

-

Maintain comprehensive documentation of data architectures, pipeline schemas, source-to-target mappings, contextual relationships, and lineage maps.

Preferred Experience

-

Bachelor’s Degree in Computer Science, Software Engineering, Data Science, Engineering, or related field + 2 years of relevant experience; equivalent Master’s degree accepted; non-relevant bachelor’s degree requires 5+ years of equivalent hands-on experience.

-

Experience working with:

-

SQL and Python for data engineering and transformation.

-

PySpark and Microsoft Fabric for lakehouse transformations.