Search by job, company or skills

Quantrics Enterprises Inc.

Senior Data Engineer (Work from Home) - Pooling

new job description bg glownew job description bg glownew job description bg svg
  • Posted 17 days ago
  • Be among the first 10 applicants
Early Applicant

Job Description

Specific Duties And Responsibilities

  • Actively participate in requirements gathering and analysis of business processes to determine technical feasibility.
  • Design and maintain ETL workflows using Airflow or similar tools.
  • Develop Spark jobs for data transformation and optimization.
  • Validate input and output data for quality and consistency.
  • Configure and monitor data quality alarms and thresholds.
  • Collaborate with QA and other engineers for testing and troubleshooting.
  • Maintain documentation and ensure reproducibility of data processes.

COMPETENCIES

Core Competencies (Must-have Competencies)

  • At least 5 years of Apache Spark - Expertise in distributed data processing, Spark SQL, and performance tuning.
  • At least 5 years of experience in Airflow / Workflow Orchestration - Ability to design and manage complex ETL pipelines with dependency handling.
  • At least 5 years of experience with BigQuery, SQL & PostGres - Strong querying skills and experience with large-scale data analytics.
  • At least 5 years of experience Python - Proficient in writing ETL scripts, data validation, and automation.
  • At least 5 years of Linux & Git- Comfortable with command-line tools and version control for data workflows.

Complementary Competencies (Good-to-have Competencies)

  • Experience with cloud services like Dataproc, Composer, Pub/Sub.
  • Knowledge of Parquet and columnar data formats.
  • Familiarity with data quality monitoring and alarm configuration.
  • Analytical mindset and attention to detail.
  • Ability to document processes and communicate findings clearly.
  • Experience in schema evolution and inventory management.
  • Debugging skills for complex data pipelines.
  • Strong bilingual communication and documentation habits.

Qualifications

Educational Qualifications

  • Bachelor's Degree in Computer Science, Information Technology, Engineering, or related fields

Work Conditions

  • Work from Home Setup
  • Amenable to work between 6PM 2 AM PHT

More Info

Job Type:
Industry:
Employment Type:

Job ID: 141449513