Job Description:

Infosys is seeking a highly skilled Azure Databricks Engineer with 6 to 8 years of experience in designing developing and optimizing cloud based data engineering solutions
The ideal candidate will possess strong expertise in Azure Databricks PySpark Azure Data Factory ADF Azure Data Lake Storage ADLS SQL and Data Engineering practices

Key Responsibilities:

Design develop and maintain scalable Azure Databricks solutions for enterprise data platforms
Build and optimize end to end ETL ELT pipelines using PySpark Spark SQL and Azure Data Factory ADF
Develop robust data ingestion frameworks from multiple structured and unstructured data sources
Implement and manage Delta Lake and Lakehouse Architecture for high volume data processing
Create and optimize data models transformations and workflows to support analytical and business requirements
Perform performance tuning and troubleshooting of Databricks workloads and Apache Spark jobs
Work with Azure services including ADLS Gen2 Azure Synapse Analytics Azure Key Vault and Azure Data Factory
Collaborate with business stakeholders architects and cross functional teams to deliver data solutions
Ensure adherence to data governance security and best practices
Participate in code reviews CI CD implementation testing deployment and production support activities

Technical Requirements:

6 to 8 years of experience in Data Engineering and Big Data technologies
Strong hands on experience with Azure Databricks and PySpark
Expertise in Apache Spark Spark SQL and distributed data processing
Experience with Azure Data Factory ADF and building enterprise scale data pipelines
Strong knowledge of Azure Data Lake Storage ADLS Gen2

Preferred Skills:

Technology->Cloud Integration->Azure Data Factory (ADF),Technology->Cloud Platform->Azure Core services->Azure Virtual Machines,Technology->Cloud Platform->Azure Networking Services->Azure Application Gateway,Technology->Data Engineering->Databricks

Azure Databricks Consultant