Search by job, company or skills

Data Engineer

Data Engineer

Asia Select Inc.
Fresher
Not Disclosed
  • Posted 10 hours ago
  • Be among the first 10 applicants

Job Description

Job Description

JOB RESPONSIBILITIES:

  • Design and develop data pipelines that will ingest the data from the respective source systems into the data warehouse
  • Support and optimize existing data pipelines to ensure that the downstream SLAs are met
  • Responsible in attending agile ceremonies as part of the Agile Tribe set-up and delivering the different demands of the Tribe related to data ingestions to the data warehouse platform
  • Ensures data quality within the data warehouse tables
  • Ensures the data engineering standards and practices are followed like code version history, readability, functionality, extendibility, test runs, deployments, and post deployment support
  • Supports L2 Data Operations team and act as L3 support
  • Manage data storage, computing unit requirements, and data security within the data warehouse
  • Gather data requirements coming from Tribe demands
  • Supports other data engineers on other Tribes or domains if necessary
  • Demonstrate technical excellence when it comes to the data warehouse platform
  • Supports stakeholders of the data warehouse platform by solving data related issues and conducting data investigations
  • Data Delivery Documentations - Data Dictionary, Data Mapping Documents, Data Models
  • Optimize Data Pipelines
  • Project delivery and operations support

JOB QUALIFICATION:

  • At least has 3 to 6 years of experience in IT project delivery or data development delivery particularly in Data Warehousing or Business Intelligence.
  • Graduate of any IT related courses like BS Computer Science, Information Technology or the likes
  • Strong SQL and ETL development experience, preferably related to Data Warehousing, Business Intelligence, or other Data Delivery projects.
  • Proficiency in optimizing data pipelines in order to meet SLA requirements and save cost
  • Experience in Big Data, Hadoop, EMR, Hive, and cloud environments is a plus
  • Proficiency in Python, R, Java, Scala is a plus
  • Proficiency in data modeling, data integrations, data quality, and data architecture is a plus
  • AWS data integrations (Glue, Athena)
  • Spark, Databricks
  • Delta lake, Data Lake
  • Kafka, Confluent, Real-time processing, Data Streaming
  • Azure Fabric (experience in ETL, Azure Data Factory, Azure blob storage)
  • Python (numpy, pandas, sklearn) works with PySpark and Scala technology
  • MustApache Airflow for Orchestration, can work with Azure Fabric pipelines and Azure DevOp
  • Must know these DE frameworks: Data Ingestion and working with API calls (see Azure Key vault), Data Transformation (table joins, aggregations, CTEs)

Work Setup: Hybrid: 1-2x a week onsite

More Info

Job Type:
Employment Type:

About Company

Similar Jobs

Hyderabad, Orion, India
Skills:
Power Bi, MongoDB, Sql, Python, Etl, ELT
Philippines
Skills:
Java, Scala, Apache Spark, Data Governance, Data mining, ELT, Sql Database Design, Azure Data Factory, Azure Synapse Analytics, Databricks, Python, Azure DevOps, Etl, Databricks SQL, Azure Cloud Databases Technologies, Segmentation techniques, MLflow, Unity Catalog, Azure Data Lake Storage, Delta Lake, data models, Databricks Workflows
Philippines, Central Visayas
Skills:
snowflake , Power Bi, PostgreSQL, AWS Glue, Tableau, Python, dbt, Athena
Taguig, Philippines
Skills:
snowflake , Advanced Analytics, Machine Learning, Data Modelling, Sql, Python, DataOps engineering practices, CI/CD, AI applications
Philippines, Manila
Skills:
Kibana, Numpy, Scipy, Pandas, Bash, Excel, Python, Sql, Apm, AWS