Job Summary
As a Senior Data Engineer (IC), you will design, build, and maintain scalable, high-performance data systems. You will collaborate with cross-functional teams to architect data pipelines, enforce data quality, and elevate our data platform. Although this is not a managerial role, you will operate as a technical expert, leading initiatives, reviewing code, mentoring junior engineers, and demonstrate a mindset of continuous improvement.
Key Responsibilities
Design and Architecture
- Lead the design and implementation of scalable data pipelines and data warehouse solutions using SQL, Python, AWS, and Redshift.
- Architect and optimize databases for performance, scalability, and security.
- Develop and maintain robust ETL processes to ingest and transform large datasets.
- Collaborate with stakeholders to translate business needs into data models and schemas.
- Ask the right questions to clarify requirements, challenge assumptions, and ensure thoughtful, effective data solutions.
Data Pipeline & Infrastructure Development
- Build, maintain, and optimize data pipelines ensuring consistent, high-quality data flow across systems.
- Develop automation tools and scripts for monitoring data quality, performance, and system reliability.
- Take proactive ownership of issues, identifying risks, proposing improvements, and preventing problems before they occur.
- Review code from team members, providing feedback to ensure best practices, maintainability, and high-quality engineering standards.
Data Quality & Governance
- Implement and maintain data quality frameworks including validation rules, automated checks, and error-handling procedures.
- Promote and enforce data governance best practices for integrity, security, and compliance.
- Lead initiatives to improve data quality, proactively identifying opportunities to enhance reliability and accuracy of data in our data warehouse.
Collaboration & Mentorship
- Work closely with Data Analysts, Data Engineers, DevOps, and other stakeholders to ensure usability and accessibility of data.
- Act as a mentor and approachable technical resource for junior team members, guiding them through projects, code reviews, and best practices.
- Demonstrate a continuous improvement mindset, helping to instil a culture of How can we do this better across the team.
- Take initiative while also accepting direction effectively, balancing autonomy with alignment to team and company goals.
Qualifications
Education
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
Technical Skills
- SQL: Expert-level proficiency in writing complex queries, performance tuning, and advanced database optimization.
- Python: Strong programming skills for ETL development, data manipulation, and automation.
- AWS: Deep experience with AWS services including Redshift, S3, Lambda, RDS, and related tools.
- Redshift: Proven track record managing and optimizing large-scale data warehousing solutions.
Data Quality
- Experience developing data quality frameworks, validation checks, and automated quality monitoring.
- Familiarity with data governance, security best practices, and compliance standards.
General Skills
- Strong problem-solving abilities with excellent attention to detail.
- Excellent written and verbal communication skills.
- Comfortable reviewing code and providing constructive feedback.
- Proactive approach, anticipating challenges, proposing solutions, and driving improvements.
- Approachable and supportive mentor for junior engineers.
- Continuous improvement mindset with the ability to instil best practices across the team.
Interview Process:
- The interview process will include a technical assessment where candidates will be required to demonstrate their skillset in SQL, Python, AWS, and Redshift. This may involve solving real-world data problems, optimizing database queries, and showcasing their ability to design data pipelines and data architectures.
- Candidates will also participate in discussions around data quality frameworks, database design, and architectural best practices.
