Head of Engineering

One of the world’s most successful commodity trading houses is building a single data platform for the entire firm, and they need an engineering leader to run the build.

The problem

Physical commodities are one of the hardest data environments there is. This business moves oil, metals and energy around the world on its own ships, through its own terminals and mines. A cargo can change hands several times mid-voyage. Prices respond to weather, freight, geopolitics and supply disruption in real time. Vessels are tracked by satellite. Trading, risk, shipping, finance and operations all depend on the same data, and today most of it arrives in overnight batches from a mix of trading systems, cloud and on-premises.

The aim is to change that. Traders and risk managers want flash P&L, current vessel positions and exposure in seconds, not the next morning. The firm wants one governed, real-time view of the business, including history you can move back and forth through, that its people and its AI tools can trust.

What you’ll be doing

• Leading the engineering of the platform across two teams: one running the Databricks and AWS foundations, and one building new capabilities, starting with streaming and change data capture
• Standing up the streaming backbone on Kafka (Amazon MSK) and the first change data capture feeds from the trading systems
• Making the platform reliable: ordering, replay, no lost events, and service levels that hold up across a global trading week
• Building out the lakehouse (Iceberg and Delta), the knowledge graph, search and the layer that lets AI agents use the firm’s data safely
• Connecting cloud and on-premises systems so they work as one platform
• Setting engineering standards (CI/CD, infrastructure as code, testing, code review) so domain teams can build their own data products
• Directing external partners and keeping them to one architecture and one delivery plan
• Hiring and developing the team

What they’re looking for

• A track record of leading engineering teams that delivered a large-scale data platform, with streaming in production
• Experience of data built for a whole enterprise, not a single desk or product
• Deep knowledge of AWS and Databricks or a comparable lakehouse platform
• A clear understanding of what makes real-time data reliable, and when batch is the better answer
• Enough hands-on ability to prototype ideas and review designs down to schema and partitioning detail
• Genuine use of AI agents in how you and your teams deliver
• Experience directing vendors and systems integrators

Commodities experience isn’t required. What matters is that you’ve worked with data that is high in volume, fast-moving and complex: trading, energy, mining, telemetry or similar.