Data Engineer - ODS, Silver/Gold & Migration

Position Overview

Click Global Talent is seeking a hands-on Data Engineer for a large-scale enterprise data warehouse (EDW) migration and build programme. This is a builder's role, focused on developing the Operational Data Store (ODS) and the Silver (conformed/integration) and Gold (consumption) layers. Key responsibilities include converting legacy IBM DataStage ETL into in-platform ELT, migrating Golden Customer Information File (GCIF) and Single Customer View (SCV) pipelines, and converting data models and objects from a legacy MPP appliance to a modern target MPP platform, with functional equivalence as the baseline.

Position Objective

The Data Engineer will turn source-to-target mappings and models into working, tested, and performant ELT pipelines and layers. This role is accountable for migrated data that reconciles to source and passes SIT, UAT, and parallel run testing.

Key Responsibilities

1. Operational Data Store (ODS) & Layered Build

• ODS Build: Build Batch 1 and Batch 2 Operational Data Store structures across in-scope source systems, covering landing/ingestion, source-aligned (Bronze) structures, and ODS tables feeding the integration layer.
• Medallion Layers: Build the Silver conformed/integration layer (conformed dimensions, integrated entities, standardisation, deduplication, historisation) and the Gold consumption layer (data marts, reporting/analytical models).
• Load Patterns & Governance: Implement incremental/CDC and full-load patterns, control columns, error handling, restartability, and embedded data-quality checks/governance hooks (lineage, metadata).

2. Legacy ETL-to-ELT Re-Engineering & Model Migration

• IBM DataStage Conversion: Analyse legacy IBM DataStage jobs and re-engineer them into in-platform, push-down set-based ELT (orchestrated via enterprise data-integration toolsets), retiring row-by-row logic while preserving exact transformation logic (functional equivalence).
• Platform & Model Migration: Migrate and convert data models, schemas, DDL, data types, distribution/partitioning keys, stored logic, views, and sequences from legacy MPP platforms (e.g., IBM Netezza) to target MPP environments (e.g., GaussDB/DWS, Snowflake, Teradata, BigQuery), documenting any unavoidable deviations.
• GCIF / SCV Build: Build and migrate Golden Customer Information File (GCIF) and Single Customer View (SCV) pipelines, enforcing matching/identity-resolution rules, survivorship, and golden-record generation/publication.

3. Pipeline Orchestration, Reconciliation & Performance Tuning

• ELT Orchestration: Develop, parameterise, and schedule ELT workflows and batch jobs; integrate with enterprise schedulers; and implement dependency, SLA, and exception handling.
• Reconciliation & Testing: Build robust reconciliation routines (row counts, control totals, balances, checksums) between legacy/source and target; produce unit-test evidence; support SIT, UAT, performance, DR testing, and parallel runs; and resolve defects to root cause.
• Optimization & Collaboration: Tune ELT pipelines and SQL queries for MPP architecture (distribution, partitioning, workload management) and collaborate closely with data modellers, business analysts, DBAs, and test teams.

Requirements & Qualifications

• Experience: Minimum of 7 years in data engineering / ETL-ELT development on enterprise data warehouse (EDW) programmes, including hands-on migration experience.
• Education: Bachelor's degree in Computer Science, Information Systems, Engineering, or a related discipline.
• SQL & Development Expertise: Production-grade SQL (including PL/pgSQL or procedural equivalent), window functions, set-based transformations, and query tuning on large datasets.
• Warehouse Architecture: Proven track record building ODS/EDW across Medallion layers (Bronze, Silver, Gold), SCD handling, CDC/incremental loading, and Data Vault 2.0 (preferred).
• Legacy & Target Tooling: Hands-on experience reading and converting legacy IBM DataStage jobs; experience migrating schemas/objects from IBM Netezza (or comparable MPP) to target MPP environments (GaussDB/DWS, Snowflake, Teradata, BigQuery); familiarity with Sunline DataMind or equivalent ELT tools.
• Domain & Regulatory Knowledge: Banking/financial services background with familiarity in core banking, deposits/CASA, loans, treasury, general ledger, GCIF/SCV, and regulatory reporting feeds (e.g., BNM/PIDM).
• Engineering Standards: Proficiency with scripting (Python, Shell), enterprise schedulers (Control-M, Autosys, Zena), and Git/JIRA/CI-CD workflows.
• Language: Ability to communicate effectively in spoken Mandarin Chinese and English.

Key Deliverables

• Built and tested ODS (Batch 1 & 2) load pipelines.
• Functional Silver (conformed/integration) and Gold (consumption/marts) layers per approved models.
• Converted legacy IBM DataStage jobs re-engineered as in-platform ELT with functional equivalence mapping.
• Validated GCIF & SCV pipelines and golden records.
• Converted target data models/DDL/objects with full conversion notes and deviation logs.
• Scheduled ELT workflows/batch jobs with exception/dependency handling and performance tuning documentation.
• Comprehensive reconciliation routines, unit-test evidence, and SIT/UAT/parallel-run defect resolution.

What We Offer

• Large-Scale Enterprise Impact: Play a pivotal role in a major enterprise data warehouse modernisation and re-platforming programme.
• Hands-on Technical Leadership: Opportunity to engineer complex, high-throughput medallion architectures and customer-mastering pipelines.
• Collaborative Environment: Work alongside top-tier data architects, data modellers, and platform specialists.

Application Process

To apply for this full-time, remote position, please create your profile at www.clickmatch.ai or send your CV to hello@clickglobaltalent.com.