We are looking for an experienced Senior Azure Data Engineer with strong expertise in Microsoft Azure Data Services and Databricks to join our growing team. In this role, you will design, build, and maintain scalable data solutions that support data integration, transformation, and analytics.
You will work closely with cross-functional teams to develop high-performance data pipelines, ensure data quality, and implement modern ETL/ELT solutions that drive business insights and decision-making.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure and Databricks.
- Build and optimize ETL/ELT processes for data ingestion, transformation, and integration.
- Develop data solutions using Python, PySpark, and SQL.
- Implement and maintain Databricks features such as Unity Catalog, Delta Live Tables, and Auto Loader.
- Develop and manage data workflows using Azure Data Factory.
- Monitor, troubleshoot, and resolve data quality and pipeline issues.
- Collaborate with architects, developers, analysts, and business stakeholders to deliver reliable data solutions.
- Optimize data processing performance and improve pipeline efficiency.
- Document technical solutions, workflows, and best practices.
- Provide technical leadership and contribute to key architectural and team decisions.
Required Qualifications
- Minimum of 5 years of experience in Data Engineering.
- Strong hands-on experience with Microsoft Azure Data Services and Databricks.
- Expert knowledge of:
- Databricks
- Unity Catalog
- Delta Live Tables (DLT)
- Auto Loader
- Proficiency in:
- Python
- PySpark
- SQL
- Azure Data Factory (ADF)
- Strong understanding of ETL/ELT development and data integration.
- Experience with data modeling, data warehousing, and database design.
- Familiarity with Azure cloud services and modern data architectures.
- Excellent analytical, troubleshooting, and problem-solving skills.
Preferred Qualifications
- Experience working in Agile environments.
- Knowledge of CI/CD practices for data engineering.
- Experience implementing data governance and security best practices.
- Strong communication and stakeholder management skills.
