Senior Data Engineer
Job Purpose
We're seeking a talented Senior Data Engineer to join our Enterprise Architecture team in a cross-cutting role that will help define and implement our next-generation data platform. In this pivotal position, you'll lead the design and implementation of scalable, self-service data pipelines with a strong emphasis on data quality and governance. This is an opportunity to shape our data engineering practice from the ground up, working directly with key stakeholders to build mission-critical ML and AI data workflows.
We emphasize building systems that are maintainable, scalable, and focused on enabling self-service data access while maintaining high standards for data quality and governance.
The ideal candidate is a problem-solver who enjoys working on complex data systems and is passionate about data quality. You thrive in collaborative environments but can also work independently to deliver solutions. You're comfortable working directly with technical and non-technical stakeholders and can communicate complex technical concepts clearly. Most importantly, you're excited about creating systems that empower others to work with data efficiently and confidently.
Responsibilities
- Design, build, and maintain our on-premises data orchestration platform using the best-in-breed open source tools
- Create self-service capabilities that empower teams across the organization to build and deploy data pipelines without extensive engineering support
- Implement robust data quality testing frameworks that ensure data integrity throughout the entire data lifecycle
- Establish data engineering best practices, including version control, CI/CD for data pipelines, and automated testing
- Collaborate with ML/AI teams to build scalable feature engineering pipelines that support both batch and real-time data processing
- Develop reusable patterns for common data integration scenarios that can be leveraged across the organization
- Work closely with infrastructure teams to optimize our Kubernetes-based data platform for performance and reliability
- Mentor junior engineers and advocate for engineering excellence in data practices
Knowledge and Experience
- 5+ years of professional experience in data engineering, with at least 2 years working on enterprise-scale data platforms
- Demonstrated experience with orchestrating workflows, performance optimization, and operational management
- Strong understanding of data transformation techniques, including experience with testing frameworks and deployment strategies
- Experience with stream processing frameworks and technologies
- Proficiency with SQL and Python for data transformation and pipeline development
- Familiarity with containerized application deployment
- Experience implementing data quality frameworks and automated testing for data pipelines
- Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders
Preferred Knowledge and Experience
- Experience with self-hosted data orchestration platforms (rather than managed services)
- Background in implementing data contracts or schema governance
- Knowledge of ML/AI data pipeline requirements and feature engineering
- Experience leveraging AI tools (e.g., GitHub Copilot, Cursor, Claude Code) to debug, develop unit tests and generate test cases from requirements documents
- Experience with real-time data processing and streaming architectures
- Familiarity with data modeling and warehouse design principles
- Prior experience in a technical leadership role
#LI-HR1 #LI-ONSITE
----------: Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.Recommended Jobs
Director of Business Development
Director of Business Development This is a remote position | Travel Required Headquarters located in Alpharetta, GA Position Overview The Director of Business Development plays a crit…
Sales Associate
We are looking for an Automotive Sales Associate to join our new car sales team! The right person will have experience selling, and must love working with people. By working at a Ford dealership, y…
Controller - Beverage Operations
Controller – Beverage Operations To Apply: email your resume to [email protected] Who: Seeking a Controller to oversee accounting, financial reporting, budgeting, and o…
Chef de Cuisine
Description The goal of the Chef de Cuisine is to manage a specific restaurant. The Chef de Cuisine will take council from the Executive Chef to create, implement menu and concepts but the positio…
Travel Radiology CT Tech Job
Take the next step in radiology as a CT Tech in Georgia, anchored in Lawrenceville, where precision imaging meets compassionate patient care. This multi-week assignment, starting on 03/23/2026, invite…
Construction Manager 4 - Combine Cycle
Job Summary By joining our BV Construction group, you will have the ability to develop and enhance your skills in an on-site, hands-on environment. We are committed to providing our construction p…
Lecturer Communication Arts
Lecturer Communication Arts Job ID: 295278 Location: Statesboro, Georgia Full/Part Time: Full Time Regular/Temporary: About Us Georgia Southern University is the state's largest and most …
Material Handling Operator
Material Handling Operator Here at Siemens, we take pride in enabling sustainable progress through technology. We do this through empowering customers by combining the real and digital worlds. We …
Repacker
Immediate Openings – Now Hiring Repackers! SelecSource is currently hiring Repackers to join our team right away! If you're dependable, ready to work, and looking for steady hours, we want to hea…
Market Data Service, Associate
About Invesco As one of the world’s leading independent global investment firms, Invesco is dedicated to rethinking possibilities for our clients. By delivering the combined power of our distincti…