AWS Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS services such as Glue, Lambda, Step Functions, EMR, Athena, and Redshift.
• Build and manage data lakes, ensuring organized, secure, and optimized storage on Amazon S3.
• Develop and maintain data warehouse solutions on AWS Redshift and implement performance tuning strategies.
• Implement and automate data ingestion frameworks from structured and unstructured data sources.
• Collaborate with data scientists, analysts, and business teams to transform business requirements into reliable data models and pipelines.
• Optimize data workflows for high availability, cost efficiency, and performance.
• Implement and enforce data governance, security, compliance, and IAM policies across the data ecosystem.
• Monitor and troubleshoot data pipelines using CloudWatch, Glue logs, and other observability tools.
• Work with CI/CD tools (CodePipeline, CodeBuild, GitHub, Jenkins) to automate deployments.
• Develop and maintain documentation for architecture, data flows, and operational processes.

Required Skills & Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.
• Strong hands-on experience with AWS cloud services (S3, Glue, Lambda, Redshift, EMR, Athena, DynamoDB, Kinesis).
• Proficiency in Python, PySpark, or Scala for data processing and automation.
• Strong experience with SQL, data modeling, and database optimization.
• Experience with big data frameworks (Spark, Hadoop) on AWS.
• Solid understanding of data warehousing, star/snowflake schemas, and best ETL design practices.
• Familiarity with CI/CD pipelines, Git, and container technologies like Docker.
• Knowledge of security best practices, including encryption, IAM, and VPC networking.
• Strong analytical, problem-solving, and communication skills.

Preferred Qualifications

• Experience with AWS Glue Studio, Glue Data Catalog, and schema management.
• Knowledge of machine learning data preparation and integration with services like SageMaker.
• Experience with Terraform or CloudFormation for infrastructure-as-code (IaC).
• AWS Certification (e.g., AWS Data Analytics Specialty, AWS Solutions Architect) is a plus.