Data Engineer

Better Companies is a multi-brand financial services organization providing debt relief, tax support, and business funding solutions. We’re strengthening our operations, refining our systems, and building a more cohesive, efficient experience across our brands. Our approach is grounded in transparency, strong operational foundations, and well-structured processes that support reliable execution and scalable growth.

About the Role

We are seeking an experienced Data Engineer to design, build, support, and optimize our modern enterprise data platform . In this role, you will modernize our data architecture toward a Microsoft Fabric–based environment, developing reliable ingestion pipelines and modeling data into trusted lakehouse and warehouse layers . A key focus of the position involves accelerating data transformation performance and maximizing the efficiency of Fabric capacity utilization . You will take ownership of end-to-end data reliability, ensuring the freshness, completeness, and accuracy of data required for operational reporting and AI-driven insights . If you are solutions-oriented, thrive on ambiguity, and take pride in turning complex data problems into reliable systems, we’d love to have you on our team .

Location: Irvine, CA (Fully In-Office)

Compensation: $140,000 - $155,000 / year

Job Type: Full-Time (Exempt/Salary)

What You'll Be Doing

In this role, priorities can shift rapidly based on system optimization needs, business analytics demands, and strategic platform initiatives .

-

Data Platform Architecture & Lakehouse Engineering

-

Design and implement scalable data warehouse and Lakehouse architectures utilizing Microsoft Fabric and Azure .

-

Architect and organize data structures within OneLake, Fabric Warehouse, and Lakehouse using a medallion (bronze, silver, gold) approach .

-

Develop and maintain ingestion and transformation pipelines using Data Factory in Fabric (pipelines, Copy jobs) and Spark/Python notebooks .

-

Ingest data from REST APIs and relational databases (MySQL/MariaDB, PostgreSQL/Redshift, SQL Server), including sources operating behind on-premises data gateways .

-

Design dimensional models and star schemas with SCD Type 2 history and conformed dimensions to support cross-functional reporting .

-

Centralize business logic in curated gold-layer tables to ensure semantic models remain consistent and feed near-real-time operational dashboards .

-

Performance Tuning & Capacity Optimization

-

Profile and tune Spark and SQL workloads to accelerate transformation run times via partitioning, file sizing, join strategies, and caching .

-

Redesign legacy full-reload processes into incremental and change-data-capture (CDC) pipelines to minimize compute overhead .

-

Monitor and manage Fabric capacity (CU) consumption with the Fabric Capacity Metrics app to prevent compute throttling during peak workloads .

-

Maintain Delta tables using OPTIMIZE, VACUUM, and V-Order to ensure fast read performance in Direct Lake and warehouse queries .

-

Audit existing pipelines, notebooks, and dataflows to eliminate redundant processing and consolidate duplicate logic .

-

Data Reliability, Observability & Stakeholder Collaboration

-

Implement automated freshness, volume, and completeness monitoring alongside proactive alerting on all production pipelines .

-

Reconcile loaded data against source systems to resolve ingestion gaps and anomalies at the root cause .

-

Execute safe historical data corrections and backfills with dry runs and validation prior to deploying to production .

-

Collaborate closely with vendor partners, internal business users, BI, and analytics teams to deliver cost-efficient, scalable data solutions .

Who You Are

We value adaptability, a solutions-oriented mindset, and deep technical curiosity :

-

Bachelor's degree in Computer Science, Data Science, Information Systems, or a related field (or equivalent professional experience) .

-

7+ years of hands-on experience in Data Engineering or Data Platform Development, including direct production experience with Microsoft Fabric and Azure data technologies .

-

Proven ability to identify, decompose, and solve complex data challenges independently and as part of a team . You adapt smoothly to shifting priorities, take initiative in fast-paced environments, and maintain resilience when troubleshooting intricate system issues .

-

Strong proficiency in Python, PySpark, and advanced SQL . Proven track record of reducing run times and compute costs through workload optimization, incremental loading, and dimensional modeling (star schema, SCD Type 2) .

-

Practical experience with Git-based development and deployment promotion workflows using Azure DevOps or GitHub .

-

Excellent communicator with the ability to bridge conversations across executive leadership (CEO, COO), vendor partners, and non-technical stakeholders alike .

Bonus Points (Great to Have)

-

Microsoft Fabric Data Engineer Associate (DP-700) certification .

-

Familiarity with Azure AI Foundry or Azure Machine Learning .

-

Experience leveraging AI-assisted coding tools to accelerate pipeline development .

-

Familiarity with secondary cloud ecosystems such as Google Cloud Platform (GCP) .

-

Prior database administration (DBA) experience managing data security, redundancy, and high availability .

What We Offer

-

Comprehensive medical, dental, and vision insurance coverage options.

-

Paid vacation and paid company holidays throughout the year.

• Lunch is on us every Friday!

-

A vibrant team culture featuring regular quarterly social events, team outings, and annual holiday parties.

-

401(k) retirement savings plan options to support your long-term financial health.

-

Direct visibility and reporting to executive leadership (CEO & COO), giving you the autonomy to architect and scale our enterprise data footprint .