Senior Data Engineer

About the Role:

We support school districts, municipalities, and education non-profits by creating custom visualization dashboards that empower leaders to make data-informed decisions.

Because our partner organizations provide diverse data formats, we do not build static, one-off scripts. We treat data pipelines as production-grade software. We are looking for a Mid to Senior Data Engineer who approaches pipeline engineering with an Object-Oriented Programming (OOP) mindset—designing reusable classes, modular architectures, and configurable ETL engines that can process tens of millions of records cleanly and reliably.

What Makes This Role Different

• Software-Driven Engineering: You will design reusable Python classes and modular frameworks rather than writing single-use procedural scripts.
• Configurable Multi-Tenant Systems: You will build dynamic, configuration-driven pipelines (JSON/YAML) so new client schemas integrate smoothly without rewriting underlying code.
• High-Performance In-Memory Processing: You will leverage both Polars and Pandas to transform large datasets efficiently.

Responsibilities:

• Architect and maintain configurable, extendable ETL/ELT pipelines using object-oriented Python design patterns.
• Ingest, transform, and validate large-scale datasets (10M+ records) across multiple educational and public sector entities.
• Model and tune columnar data warehouses in Google BigQuery to support downstream analytics and dashboards.
• Implement structured logging, proactive alerting, and graceful error handling to maximize pipeline uptime.
• Write unit and integration tests using pytest and mocked data interfaces to ensure pipeline reliability prior to deployment.
• Maintain clean documentation, docstrings, and strict Git version control across codebases.

Qualifications, Skills and Experience:

• Must have minimum of 3 years of professional Data Engineering experience with Python as your core toolset.
• Demonstrated Object-Oriented Programming (OOP) background: Real-world experience structuring Python pipelines using classes, objects, inheritance, and design patterns.
• Hands-on proficiency in both Pandas and Polars: Practical experience transforming data in production using both libraries.
• Experience with Google BigQuery or another major columnar data warehouse (e.g., Snowflake, Redshift).
• High-volume data experience: Proven ability to transform and load datasets exceeding tens of millions of records.
• Testing & Reliability: Practical experience authoring automated tests (e.g., pytest), setting up logging frameworks, and configuring pipeline alerts.

Key Tech Stack: Python (OOP), Polars, Pandas, Google BigQuery, SQL, and Pytest

Employment Type: Full-Time

Work Set-up: 100% Work from home

Work Schedule: Night Shift - 9:00 PM to 6:00 AM PH time (aligned with US Business Hours/US Eastern Time)