Job Description

The Senior Data Engineer is responsible for designing, developing, and maintaining scalable data engineering solutions that support the migration from legacy big data platforms to modern, cloud-based data environments. The role ensures reliable data operations while enabling ongoing and new business initiatives.

Duties and Responsibilities

● Design, build, and optimize automated data pipelines, ETL/ELT processes, and data models to ingest, process, and store large volumes of data within cloud-based platforms.

● Support large-scale data migration initiatives, ensuring data accuracy, performance efficiency, and minimal business disruption.

● Develop and maintain ETL/ELT workflows to ingest, transform, and load data from multiple internal and external sources with a focus on scalability and reliability.

● Partner with business, analytics, and product teams to translate data requirements into effective technical solutions that support strategic initiatives.

● Design and deliver data marts and customized data extractions aligned with business and reporting needs.

● Ensure compliance with enterprise data governance, security, and regulatory standards.

● Monitor data pipeline health and performance, troubleshoot data incidents, and implement preventive and corrective measures.

● Document data workflows, schemas, technical specifications, and operational runbooks to support operational stability and knowledge transfer.

● Collaborate closely with product owners, data architects, and data scientists to maintain a reliable and efficient data infrastructure.

● Drive continuous improvement of data engineering practices, tools, and automation frameworks.

● Provide technical guidance and mentorship to junior engineers through code reviews, best-practice sharing, and troubleshooting support.

● Manage and deliver multiple data engineering initiatives concurrently while meeting quality, scope, and timeline expectations.

Required Experience, Technical Knowledge, and Skills

● Experience

○ At least 8 years of total Data Engineering experience, with strong exposure to large-scale data pipelines, ETL/ELT development, and enterprise or cloud-based data platforms.

○ Proven experience designing, building, and optimizing scalable data solutions in modern data environments.

● Knowledge

○ Python – At least 4 out of 5 proficiency level, with strong hands-on experience in data transformation, automation, and pipeline development.

○ SQL – At least 3 out of 5 proficiency level, with demonstrated capability in complex queries, data modeling, and performance tuning.

○ Experience working with modern data cloud platforms, such as Databricks and/or Snowflake.

○ Experience with cloud services, preferably Microsoft Azure (e.g., Azure Data Factory, Azure Synapse, Azure Storage, etc.).

○ Platform and Cloud Exposure - Currently working with or has recent hands-on experience in enterprise-grade cloud data ecosystems. With a strong understanding of cloud-native data architecture and best practices.

● Soft Skills

Strong verbal and written communication skills

○ Demonstrated leadership and technical influence

○ Strong analytical, critical thinking, and problem-solving abilities

○ Process orientation with the ability to enforce standards and best practices

○ Strong organizational, multitasking, and time-management skills

○ Stakeholder and cross-functional collaboration skills