Search by job, company or skills

Data Senior Engineer

Data Senior Engineer

NTT Data
Fresher
Not Disclosed
Early Applicant
  • Posted a day ago
  • Be among the first 10 applicants

Job Description

Key Responsibilities
. Assess complex Informatica workflows including sessions, mappings, and parameterizations and translate them into equivalent Python EL scripts and dbt transformation models.
. Develop and maintain Python EL pipelines to land high-volume data, including billion-row tables, into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools such as Oracle Data Pump and SQL Server BCP.. Design and develop dbt transformation models based on Informatica mapping logic, incorporating dbt best practices including model layering, macros, incremental strategies, and snapshot patterns.
. Develop dbt-native tests as well as custom Python unit tests to validate transformation correctness and data quality.
. Develop Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, producing output that is functionally equivalent to the source Informatica workflows.
. Contribute to GitLab CI/CD pipeline for dbt and Airflow code, including lint gates, automated testing, and deployment to shared NAS.
. Perform peer code reviews and provide constructive technical feedback to fellow engineers.
. Troubleshoot performance issues and data discrepancies during SIT and UAT, including row-count reconciliation between source Oracle/SQL Server systems and the target warehouse.
. Contribute to technical documentation, runbooks, and handover materials.

More Info

Job Type:
Employment Type:

Key Skills

SQL Server BCP

GitLab CI/CD

pyodbc

cx_Oracle

About Company

Similar Jobs

Bengaluru, India
Skills:
oracle data pump , Sqlalchemy, Informatica, Python, SQL Server BCP, Airflow, cx Oracle, dbt, GitLab CI/CD, pyodbc
Bengaluru, India
Skills:
Automated Testing, Cloud Storage, Terraform, Python, Java, BigQuery, Scala, Sql, Git, Apache Beam, DataFlow, Dataform, CI CD, Pub Sub, Vertex AI Agent Builder, Cloud Composer, Gemini models, BigQuery ML, Vertex AI, IAM secrets, LookML, Google Cloud GCP, Looker, Dataplex
Remote, India
Skills:
Git, Data Modelling, Power Bi, Databricks, Data Warehouse Architecture, Tableau, Azure, Sql, Python, ETL pipelines
Remote, India
Skills:
Agile Development, Alteryx, Power Bi, Databricks, Data Warehouse Architecture, Tableau, Azure, Sql, Python, data modelling techniques, ETL pipelines
Bengaluru, India
Skills:
Apis, Pyspark, Sql, Git, Databricks, Python, embeddings, vector search, RAG architecture, text extraction, vector databases, metadata engineering, semantic retrieval, CI CD, document ingestion, Azure cloud storage