Data Engineer (Databricks)
Data Engineer (Databricks)
xurpas, inc.4-6 Years
- Posted 2 months ago
- Be among the first 10 applicants
Job Description
Summary
As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to enhance data accessibility and usability, contributing to the overall data strategy of the organization.
Roles & Responsibilities
Expected to be an SME.
Collaborate and manage the team to perform.
Responsible for team decisions.
Engage with multiple teams and contribute on key decisions.
Provide solutions to problems for their immediate team and across multiple teams.
Develop and optimize data pipelines to ensure efficient data flow and processing.
Monitor and troubleshoot data quality issues, implementing corrective actions as necessary.
Document data processes and workflows to ensure clarity and compliance with best practices
Professional & Technical Skills
Must have 4+ years of experience in data engineering
Expert proficiency in Databricks Unified Data Analytics Platform
Experience with Python (Programming Language)
Strong understanding of data modeling and database design principles.
Experience with ETL tools and data integration techniques.
Familiarity with cloud platforms and services related to data engineering.
Proficient in data warehousing concepts and practices.
Must have skills: Databricks (Unity Catalog, Delta Live Tables, Auto Loader) -Python, Pyspark -SQL -Azure Data Factory
As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to enhance data accessibility and usability, contributing to the overall data strategy of the organization.
Roles & Responsibilities
Expected to be an SME.
Collaborate and manage the team to perform.
Responsible for team decisions.
Engage with multiple teams and contribute on key decisions.
Provide solutions to problems for their immediate team and across multiple teams.
Develop and optimize data pipelines to ensure efficient data flow and processing.
Monitor and troubleshoot data quality issues, implementing corrective actions as necessary.
Document data processes and workflows to ensure clarity and compliance with best practices
Professional & Technical Skills
Must have 4+ years of experience in data engineering
Expert proficiency in Databricks Unified Data Analytics Platform
Experience with Python (Programming Language)
Strong understanding of data modeling and database design principles.
Experience with ETL tools and data integration techniques.
Familiarity with cloud platforms and services related to data engineering.
Proficient in data warehousing concepts and practices.
Must have skills: Databricks (Unity Catalog, Delta Live Tables, Auto Loader) -Python, Pyspark -SQL -Azure Data Factory
More Info
Key Skills
Databricks Unified Data Analytics Platform
data integration techniques
Python Programming Language
Auto Loader
Delta Live Tables
cloud platforms and services related to data engineering
data warehousing concepts and practices
Databricks Unity Catalog
About Company