Job Description

Responsible for designing and developing scalable data engineering solutions using PySpark, Spark, Hadoop, Hive, Kafka and cloud platforms. The role involves building complex batch and streaming ETL/ELT pipelines, data migration and optimisation, integrating AWS data services, and implementing data processing frameworks.

The role also requires hands-on experience in Generative AI, LLM, RAG, vector databases, NLP, API development and containerised deployment using Docker/Kubernetes/OpenShift. Candidates should have strong programming skills in Python, Java or Scala and experience with enterprise-scale data platforms, CI/CD and production support.