Senior Data Engineer

Intercontinental Exchange Holdings, Inc.
Atlanta, GA
Overview:

Job Purpose

We're seeking a talented Senior Data Engineer to join our Enterprise Architecture team in a cross-cutting role that will help define and implement our next-generation data platform. In this pivotal position, you'll lead the design and implementation of scalable, self-service data pipelines with a strong emphasis on data quality and governance. This is an opportunity to shape our data engineering practice from the ground up, working directly with key stakeholders to build mission-critical ML and AI data workflows.

 

We emphasize building systems that are maintainable, scalable, and focused on enabling self-service data access while maintaining high standards for data quality and governance.

 

The ideal candidate is a problem-solver who enjoys working on complex data systems and is passionate about data quality. You thrive in collaborative environments but can also work independently to deliver solutions. You're comfortable working directly with technical and non-technical stakeholders and can communicate complex technical concepts clearly. Most importantly, you're excited about creating systems that empower others to work with data efficiently and confidently.

 

Responsibilities

  • Design, build, and maintain our on-premises data orchestration platform using the best-in-breed open source tools
  • Create self-service capabilities that empower teams across the organization to build and deploy data pipelines without extensive engineering support
  • Implement robust data quality testing frameworks that ensure data integrity throughout the entire data lifecycle
  • Establish data engineering best practices, including version control, CI/CD for data pipelines, and automated testing
  • Collaborate with ML/AI teams to build scalable feature engineering pipelines that support both batch and real-time data processing
  • Develop reusable patterns for common data integration scenarios that can be leveraged across the organization
  • Work closely with infrastructure teams to optimize our Kubernetes-based data platform for performance and reliability
  • Mentor junior engineers and advocate for engineering excellence in data practices

Knowledge and Experience

  • 5+ years of professional experience in data engineering, with at least 2 years working on enterprise-scale data platforms
  • Demonstrated experience with orchestrating workflows, performance optimization, and operational management
  • Strong understanding of data transformation techniques, including experience with testing frameworks and deployment strategies
  • Experience with stream processing frameworks and technologies
  • Proficiency with SQL and Python for data transformation and pipeline development
  • Familiarity with containerized application deployment
  • Experience implementing data quality frameworks and automated testing for data pipelines
  • Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders

Preferred Knowledge and Experience 

  • Experience with self-hosted data orchestration platforms (rather than managed services)
  • Background in implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience leveraging AI tools (e.g., GitHub Copilot, Cursor, Claude Code) to debug, develop unit tests and generate test cases from requirements documents
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior experience in a technical leadership role

#LI-HR1 #LI-ONSITE

----------: Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.
Posted 2025-11-26

Recommended Jobs

Cold Station Cook

The Capital Grille
Dunwoody, GA

We Wine. We Dine. We Build Careers!  For this position, pay will be variable by location - See additional job details and benefits below We cordially invite you to apply. Joining our …

View Details
Posted 2026-01-15

General Dermatology & Cosmetic Surgery Opportunity

National Health Partners
Atlanta, GA

General Dermatology & Mohs Surgery Opportunities Join this system of Dermatology practices looking due to expansion. Enjoy doing a mix of General Dermatology & Cosmetics Monday - Friday Sche…

View Details
Posted 2025-07-27

Logistics Associate

HonorBuilt
Mcdonough, GA

HonorBuilt is looking for a motivated individual to fill the position of Logistics Associate in our IT Provisioning Warehouse. The position is based out of McDonough, GA. Our associates receive, insp…

View Details
Posted 2026-01-01

Carside / Take Out Specialist

Applebee's
Moultrie, GA

Get paid DAILY!  We’ve partnered with Rain to offer you the ability to get 50% of your pay within hours of your shift every day! Neighborhood Restaurant Partners (NRP) is the fourth largest franchise…

View Details
Posted 2025-07-25

Sr. Project Manager - PMO Lead

MTech Systems
Dunwoody, GA

At MTech Systems, our company mission is to increase yield in protein production to help feed the growing world population without compromising animal welfare or damaging the planet. We aim to create…

View Details
Posted 2025-12-23

Sales Engineer

Dynamics ATS
Ideal, GA

Sales Engineer   JOB-10045635   Anticipated Start Date January 18, 2026   Location Rancho Cucamonga, CA   Type of Employment Direct Hire   Employer Info Our client is a …

View Details
Posted 2026-01-06

Hourly 1099 Drywall finisher

Superior Contracting & Maintenance
Atlanta, GA

Drywall Repair • Patching • Texture Matching 🏠 Home Ease Division | Superior Contracting and Maintenance 📍 Atlanta Metro Area | $20 to $30 per hour | About the Role Seeking hourly 1099 …

View Details
Posted 2026-02-18

LPN - Rockdale Pulm Sleep

Piedmont Healthcare Inc.
Conyers, GA

Overview: Experience the advantages of real career change Join Piedmont to move your career in the right direction. Stay for the diverse teams you’ll love, a shared purpose, and schedule flexibili…

View Details
Posted 2026-01-27

Golf - Outside Service Staff

West Lake Country Club
Augusta, GA

West Lake Country Club Job Description: Job Title: Cart Attendant | Effective Date: Current Department: Golf | FLSA Status: Non-Exempt Reports To: Golf Professional POSITION SUMM…

View Details
Posted 2025-11-01

Sprinter/Cargo Van Owner Operator

Victoria Logistics Carrier LLC
Valdosta, GA

Hello, guys!  We are hiring Cargo Van and Sprinter Van owner-operators for our company, VICTORIA LOGISTICS CARRIER.  We work within an independent contract agreement and offer very competitive ra…

View Details
Posted 2025-10-30