
Search by job, company or skills

Data Engineer
Role Summary
We are seeking a Data Engineer to design, build, and maintain scalable data platforms that
power analytics, reporting, and operational decision-making. The role is responsible for
data pipelines, warehouse architecture, Change Data Capture (CDC), and real-time data
integration, ensuring data is reliable, accessible, and optimized for business use.
Key Responsibilities
● Design and maintain end-to-end data architecture from source systems to analytics
platforms.
● Build and manage ETL/ELT pipelines using Apache Airflow.
● Develop and optimize data warehouses in Google BigQuery and ClickHouse.
● Implement and support CDC solutions using Debezium, Kafka, and Altinity Sink
Connector.
● Manage and monitor Kafka infrastructure through Strimzi Operator on
Kubernetes/EKS.
● Design and maintain scalable data models, fact tables, and dimension tables.
● Optimize ClickHouse performance through proper partitioning, indexing, storage
design, and query tuning.
● Implement data quality, monitoring, alerting, and observability frameworks.
● Ensure platform reliability, scalability, and cost efficiency.
● Collaborate with Analytics, Engineering, Product, and Business teams to deliver
trusted data solutions.
Required Qualifications
● Data Engineering experience.
● Advanced SQL and data modeling skills.
● Hands-on experience with:
○ Google BigQuery
○ ClickHouse
○ Apache Airflow
○ Apache Kafka
○ Debezium CDC
● Experience building large-scale ETL/ELT pipelines.
● Understanding of distributed systems, CDC, and event-driven architectures.
● Experience with cloud platforms and containerized environments.
● Strong troubleshooting and performance optimization skills.
Preferred Qualifications
● Experience with Strimzi Operator.
● Experience with Altinity Sink Connector.
● Python, dbt, Terraform, and GCP experience.
● Experience supporting Metabase, or other BI platforms.
Technology Stack
ClickHouse • BigQuery • Airflow • Kafka • Debezium • Altinity Sink Connector • Strimzi •
Kubernetes (EKS) • SQL • Python • AWS
Success Measures
● Reliable and scalable data pipelines.
● High-performance ClickHouse and BigQuery environments.
● Low-latency CDC data ingestion.
● High data quality and platform uptime.
● Optimized warehouse cost and performance.
● Timely delivery of data solutions for analytics and reporting.
Key Positioning
The Data Engineer owns the organization's data platform, including BigQuery, ClickHouse,
Airflow, Kafka, and CDC architecture, providing the foundation for analytics, reporting, and
data-driven decision-making.
Job ID: 150978685
Skills:
snowflake , Java, Hive, Pandas, Spark, MongoDB, Redshift, Python, Sql, ETL processes, Dplyr
Skills:
Data Transformation, Power Bi, Tableau, Data Modeling, Sql, ELT, Data Integration, Python, Etl, data visualization platforms, data pipelines, data quality management, Looker, business intelligence, Looker Studio, Data Validation
Skills:
Java, PostgreSQL, Sql, Apache Airflow, MySQL, Elasticsearch, Spark, Apache Kafka, Databricks, MongoDB, Python, AWS
Skills:
data wrangling , Information Retrieval, Python Scripting, automation, Ocr, LLM AI APIs, multi-format handling, incremental processing, structured extraction, SQL query optimization, document parsing, Data Validation, prompt design
Skills:
Java, BigQuery, Csv, Json, Avro, Sql, ELT, Apache Airflow, Apache Beam, Python, Etl, Parquet, Cloud Spanner, GCP Dataflow, GCP Cloud Monitoring, Cloud Composer
We don’t charge any money for job offers