Data Platform Engineer

SUMMARY

We are looking for a hands-on Senior Platform Engineer to build and extend an internal developer platform focused on self-service environment provisioning, platform automation, and reusable engineering capabilities.

This is not a traditional DevOps-only role. The engineer will work across software engineering and platform engineering, building both the internal vending platform and the reusable components it delivers to engineering teams.

JOB RESPONSIBILITIES

• Build and enhance internal platform services using Python, FastAPI/Flask, REST APIs, and related frameworks.
• Develop reusable Python libraries, wheel packages, plugins, SDKs, and automation components.
• Build custom plugins for data governance, data contracts, schema/DDL generation, event-driven architecture, validation, and platform guardrails.
• Automate data-contract-driven workflows, including generating DDLs, schemas, configuration, governance rules, or deployment assets.
• Create reusable golden paths and project templates using Jinja, Copier, or similar frameworks.
• Build and maintain Terraform modules for AWS, IAM, EKS, Airflow, Databricks, secrets, and supporting infrastructure.
• Support multi-account and multi-environment AWS deployments.
• Build and maintain Harness CI/CD pipelines for infrastructure, platform services, Airflow, Databricks, and application workloads.
• Integrate Harness with GitHub/GitLab, Terraform, AWS, Kubernetes, and secrets-management systems.
• Support Kubernetes / Amazon EKS, Airflow / Astronomer, Databricks / Unity Catalog, Kafka / Confluent, and observability platforms.
• Implement secure IAM, cross-account access, policy controls, and least-privilege design.
• Troubleshoot end-to-end platform workflows across APIs, pipelines, Terraform, AWS, Kubernetes, Airflow, and Databricks.

QUALIFICATIONS

• 5+ years in Platform Engineering, Cloud Engineering, DevOps, SRE, Infrastructure Engineering, or Software Engineering.
• Strong Python development experience beyond scripting.
• Experience with FastAPI, Flask, REST APIs, and production Python services.
• Experience building reusable Python packages, plugins, or internal developer tooling.
• Strong Terraform experience, including reusable modules and state management.
• Strong AWS experience, including multi-account environments and IAM.
• Strong CI/CD experience using Harness.
• Experience with GitHub and/or GitLab.
• Docker and Kubernetes / EKS knowledge.
• Strong troubleshooting, API integration, and software-engineering practices.

Preferred Skills

• Apache Airflow / Astronomer
• Databricks / Unity Catalog
• Databricks Asset Bundles
• Data contracts / ODCS
• Data governance automation
• Event-driven architectures
• Kafka / Confluent
• Backstage
• Jinja / Copier
• OPA / Rego
• Datadog
• Terraform Cloud / HCP Terraform
• Harness

JOB REQUIREMENTS

• Should be willing to accept a long-term work-from-home arrangement.
• Should be amenable to a permanent night shift schedule.