Senior Data Engineer

Job Responsibilities:

● Design, develop and maintain data pipelines that extract, load and transform data from

various sources into the data warehouse

● Design and implement data models that align with business requirements and support

efficient querying and analysis

● Ensure the team's orchestration pipelines are reliable, scalable and efficient whilst

considering factors including data volume, velocity and variety

● Work across data engineering teams, vendors and internal stakeholders to contribute to the

continuous improvement of data products, processes and systems.

● Follow data engineering best practices including coding, DevOps, documentation and

metadata standardizations

● Experience with the management and nature of audience tracking and subscribers

datasets

Employee responsibilities:

● Take ownership and accountability for assigned work, including development and end-to-

end testing

● Ability to conduct comprehensive feasibility assessments of architectural and data modelling

options, systematically evaluating the trade-offs, advantages, and limitations of each

proposed solution

● Ensure high standard and ongoing quality of Engineering output whilst meeting project

timelines

● Document existing and future solution designs to enhance the team's collective knowledge

Qualifications:

Essential qualifications, experience & skills

● Fluent in Python programming language

● Proficient in SQL

● Understanding of OLTP concepts including normalised schema design

● Understanding of OLAP concepts and data model design

● Experience leading the designing and building data pipelines, ETL processes, and

managing data infrastructure

● Experience leading workflows automation, managing task dependencies and

transformations using Apache Airflow and DBT

● Experience in real time streaming pipelines like Kafka/pubsub and Pyspark/dataflows