We are seeking a Senior Data Engineer to design, build, and maintain scalable, modular, and cost-effective data pipelines and products in the Microsoft Azure ecosystem. The role involves collaborating with cross-functional teams to deliver high-quality, automated data solutions that support our data architecture and analytics initiatives.
Key Responsibilities
- Design, develop, and maintain ETL processes and data pipelines using Azure Data Factory, Azure Synapse, Azure Databricks, and Azure Fabric.
- Organize and manage data pipeline outputs using Azure Data Lake Storage Gen2 and Azure Blob Storage.
- Collaborate with data scientists, analysts, and architects to understand requirements and deliver robust data solutions.
- Optimize pipelines for performance, scalability, and reliability.
- Ensure data quality and integrity through validation and monitoring.
- Document data processes, configurations, and best practices.
- Manage CI/CD deployment processes for data solutions and troubleshoot pipeline issues.
- Stay updated on industry trends and emerging data technologies.
Qualifications
- Bachelors degree in Computer Science, Engineering, or related field (or equivalent experience).
- 7+ years of experience as a Data Engineer or in similar data/ETL roles.
- Strong expertise in Microsoft Azure services (Data Factory, Synapse, Databricks, Blob Storage, Data Lake Gen2).
- Proficient in SQL and working with modern RDBMS (e.g., SQL Server, PostgreSQL).
- Solid understanding of software engineering principles applied to data engineering (CI/CD, version control, testing).
- Experience with big data technologies (e.g., Spark).
- Excellent problem-solving, communication, and collaboration skills.
