Data Engineer

Job- Role Details:

Job-Role-Data Engineer

Job-Location:Manila, Philippines location

Experience required: 5+ years

Salary to be offered: decent hike on current CTC as per the company/market standards

Job-Requirements:

Job Description:

5+ years of hands-on experience in development and production implementation with Azure Databricks and Spark

We are on the lookout for a skilled Data Engineer to join our dynamic team. The ideal candidate will have hands-on experience with Azure Databricks for data engineering and analytics, Microsoft Purview for data governance, and Microsoft Fabric for modern data integration and analytics. Proficiency in Python (PySpark) and SQL, along with expertise in data lake architecture, ETL processes, and data quality management, is essential. Experience with Azure data services and excellent technical documentation skills are also required.

*Responsibilities:*

- Design, develop, and implement data processing pipelines using Azure Databricks and Spark.

- Write efficient and optimized code in Python (PySpark) and SQL for large-scale data processing.

- Architect and maintain data lakehouse solutions leveraging Microsoft Fabric for analytics and reporting.

- Implement data governance and cataloging using Microsoft Purview, ensuring compliance and lineage tracking.

- Build and optimize ETL workflows for ingestion, transformation, and integration across Azure services.

- Ensure data quality, security, and integrity through robust validation and governance practices.

- Collaborate with cross-functional teams to understand business requirements and deliver scalable solutions.

- Maintain version control using Git and follow best practices for CI/CD in data engineering.

- Create comprehensive technical documentation, including requirements, design, and testing artifacts.

Requirements:*

- 5+ years of hands-on experience in development and production implementation with Azure Databricks and Spark.

- Proven experience with Microsoft Purview for data governance and compliance.

- Working knowledge of Microsoft Fabric for analytics and integration.

- Proficiency in Python (PySpark) and SQL coding.

- Strong understanding of data lake architecture, ETL processes, and data quality management.

- Experience with Azure data analytics services (e.g. Data Lake, Synapse, Data Factory).

- Excellent technical writing and communication skills.

- Ability to optimize performance for large-scale data processing and ensure scalability