NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.
Duties & Responsibilities
- Develops automated data pipelines based on recommended solution by Data Engineering Lead using official ETL tools determined by Data Engineering Head
- Creates automated data pipeline design documentation, Source-to-Target spreadsheets, and technical design specifications.
- Provides technical expertise, troubleshoots, guides and supports during design, development, testing, production and post-production.
- Understands the source system, captures and documents business rules.
- Analyzes and proposes changes to maximize code reusability and maintain business value.
- Develops, implements and maintains highly efficient and scalable automated data pipeline processes.
- Provides expert technical knowledge and solutions pertaining to Data integration.
- Identify opportunities to optimize the automated data pipeline including environment setup, implement monitoring, quality and validation processes to ensure data accuracy and integrity.
- Provides inputs into data governance organization to drive the data governance charter.
- Ensures compliance to internal standards and guidelines.
- Conducts unit testing and component integration testing for solutions developed
- Reviews business and technical requirements and ensures the data integration platform meets requirements.
- Applies industry best practices for automated data pipeline design and development.
- Implementation of an end-to-end complex automated data pipeline system using common Big Data Tools such as (but not limited to) Apache Spark, Talend, Kafka, Airflow, NiFi i and Hadoop Tools.
- Implements monitoring and measurement of data quality as defined by business rules.
- Ensures adherence to architectural governance standards and practices.
- Develops best practices, standards of excellence, and guidelines for programming teams.
- Ensures compliance to the DevSecOps practice.
- Conducts System Testing - execute job flows, investigate system defects, resolve defects and document results
- Provides a level of effort estimates for new initiatives and change controls in order for projects to be evaluated, budgeted and tracked for success.
Minimum Qualifications
- Total Data Engineering experience - 7- 8yrs and above
- Skilled in Python (atleast 4/5 rating), SQL (atleast 3/5 rating)
- Working or has experience working on modern data cloud platforms (Databricks/Snowflake)
- With exposure to cloud services (preferably Azure or AWS)
- Wiling to work on a hybrid work set up in BGC Taguig
