Senior Data Engineer

About the Role

Operate, monitor, and continuously improve production data workloads. The role combines DataOps/ETL operations and Data Engineering, requiring hands-on experience in Python, PySpark, SQL, and AWS to support data pipelines, production workloads, data quality, and operational reliability.

The successful candidate will manage schedules and SLAs, execute controlled changes, troubleshoot production data pipelines, lead first-line incident response, and contribute to the development and optimization of scalable data processing solutions..

Key Responsibilities

• Data Engineering & Pipeline Development. Develop, maintain, and enhance data pipelines using Python and PySpark.
• Scheduling & Monitoring. Maintain job calendars, dependencies, workflows, and scheduling configurations. Implement health checks, monitoring, and alerts to proactively identify pipeline and job failures.
• Incident & Problem Management. Triage production incidents involving ETL/data pipelines, Python/PySpark jobs, scheduling, and cloud data workloads.
• Change & Release Execution. Execute application, data pipeline, configuration, and deployment changes in accordance with established change controls.
• Data Quality Operations Support

Key Qualifications

• At least 8+ years of overall IT experience, with strong experience in Data Engineering, DataOps, ETL, or production data support.
• 3–6 years of experience in DataOps/ETL operations, including hands-on experience with enterprise schedulers such as Control-Mand production support.
• Hands-on experience in Data Engineering using Python and PySpark, including developing, maintaining, troubleshooting, and optimizing data pipelines.
• Strong working knowledge of SQL and experience working with large datasets and data processing workloads.
• Hands-on or strong working experience with AWS cloud services used for data engineering and data processing.
• Experience with data pipeline scheduling, monitoring, troubleshooting, and performance optimization.
• Familiarity with Data Quality, Data Lineage, Metadata Management, and Data Governance processes is a plus.
• Experience working in a production environment with defined SLAs, incident management, change management, and release processes.
• Strong analytical and problem-solving skills with the ability to troubleshoot complex data and pipeline issues.