Lead Data Engineer

Job Position - Data Integration Lead (Streaming & Warehousing) - Retail Wealth

Location: Remote

Role

The role is a hands-on Data Engineering Lead to build streaming and batch data pipelines
powering our digital self service roadmap. Design robust ETL/ELT on AWS, Kafka,
Databricks, Oracle, and guide the transition to Snowflake. The role blends architecture and
delivery-owning data quality, observability, and governance in the Retail Wealth domain.
Domain Expertise: Deep experience in retail wealth, broker dealers, warehouses,
family offices, custodians, trade order management, and digital self-service platforms
(e.g., robo advisors).
Technical Stack: AWS, Kafka, Databricks, Oracle, Snowflake (future), ETL processes.
Hands-On Leadership: Integration lead must design/develop APIs and coordinate
with architects; data lead must be strong in ETL/data integration.

Responsibilities
Design and build streaming & batch pipelines for ingestion, curation, and
consumption (realtime + microbatch).
Engineer scalable ELT/ETL on Databricks (PySpark/Spark SQL), integrating sources
including APEX, custodians, broker dealers, and market/reference data.
Optimize workloads on AWS (S3, Glue, EMR/Databricks, Lakehouse patterns);
manage Oracle sources; drive Snowflake migration strategy and execution.
Enforce data quality (DQ), lineage, metadata, and governance with bestpractice
frameworks and tooling.
Partner with analytics, product, and integration teams to support dashboards,
operational reporting, advanced analytics, and ODS.
Establish DevSecOps for data (versioned transformations, CI/CD, IaC patterns, secrets
mgmt).
Define and track SLAs/SLOs, cost controls, and performance baselines.



Must-Have Qualifications
5+ years (7+ preferred) in data engineering with lead-level ownership delivering
production pipelines.
Retail Wealth expertise: custodians, broker dealers, warehouses/family offices;
order/trade and position/transaction data.
Hands-on with Kafka (topics, partitions, schema/registry), Databricks (PySpark/Spark
SQL), AWS data stack, and Oracle sources.
Strong SQL/performance tuning; ELT/ETL design patterns; batch orchestration (e.g.,
Airflow/Databricks Jobs).
Practical data governance: lineage, DQ, PII controls, encryption, RBAC, and
regulatory awareness (FINRA/SEC).
Experience planning/executing Snowflake migrations (data modeling, performance,
cost/pricing levers).



Nice to Have

Familiarity with Apex Fintech data domains and vendor ecosystems (Orion, Envestnet,
Pershing/Schwab).
Knowledge of DTCC/NSCC, Morningstar data, advisory/UMA/SMA billing &
commissions.
Observability for data (Great Expectations/Deequ, Delta Live Tables), cost optimization,
and dbt or equivalent.
Bachelor's in CS/Engineering/Math/IS (Master's a plus).