Data Engineer

Description

Position Overview:We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for data migration, data replication, and system implementation, ensuring high-quality data management and integration processes.

Key Responsibilities:

• Data Migration:
• Perform data migration from one system to another, involving data matching, mapping, cleaning, and writing queries.
• Execute data loading and conduct post-load validation to ensure data integrity and accuracy.
• Data Replication:
• Enable data replication from the target system to PostgreSQL on Azure.
• Convert source system data to CSV format and place it in S3
• Utilize Azure Data Factory (ADF) to pull data into PostgreSQL on Azure SQL.
• Run queries in PostgreSQL to generate reports and extracts as needed.
• System Implementation:
• Participate in the implementation of new systems, utilizing a base Entity-Relationship Diagram (ERD) while customizing it based on specific business requirements.

Primary Technologies:

• Have at least 3 year's experience as Data Engineer.
• Have strong experience under this following:

MSSQL, PostgreSQL on Azure; SSIS, Azure Data Factory, Python, Azure Data Bricks