I
Aws Developer
I
Aws Developer
Infosys- Posted 2 hours ago
- Be among the first 10 applicants
Job Description
- Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
- Develop AWS Glue jobs using PySpark, and SQL for large-scale data processing.
- Build data ingestion and transformation pipelines across structured and semi-structured data sources.
- Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.
- Work with AWS services such as Amazon S3, Redshift, and IAM.
- Implement data transformations, validations, data-quality checks, and reconciliation processes.
- Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.
- Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.
- Troubleshoot ETL pipeline failures and production performance issues.
- Implement appropriate security controls including IAM roles, encryption, and access management.
- Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.
- Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.
- Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.
- Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.
- AWS Glue
- Python/PySpark / Apache Spark
- SQL
- Amazon S3
- Data Lake / Cloud Data Warehouse concepts
- ETL/ELT design and development
- Performance optimization and troubleshooting Experience
- 7+ years of overall Data Engineering / ETL development experience.
- Strong hands-on experience developing production-grade AWS Glue,Python and PySpark solutions.
- Experience handling high-volume datasets and complex transformations in AWS environments. Preferred Certification
- AWS Certified Data Engineer – Associate or other relevant AWS certification.
More Info
Key Skills
ELT design and development
Cloud Data Warehouse concepts
About Company
Skills:
Amazon Redshift, Pyspark, AWS Glue, Python, Sql, ELT, Etl, Distribution Keys, RA3 nodes, Redshift Spectrum, Sort Keys
Skills:
S3, Typescript, Javascript, Terraform, Sns, Python, Api Gateway, Java, Aws Lambda, Dynamodb, Jenkins, Git, Cloudwatch, Amazon Connect, Sqs, Iam, AWS CloudFormation, Rest Apis, Service Cloud Voice, Infrastructure as Code, AWS CDK, Amazon Connect Contact Lens, EventBridge, Salesforce Service Cloud, GitHub Actions, Amazon Lex, Serverless architectures, Salesforce CTI, Open CTI
Skills:
amazon dynamodb , Java, Aws Lambda, CSS, PostgreSQL, Npm, Nosql, Jenkins, React, Terraform, AWS CloudFormation, Python, Api Gateway, Third-party API integrations, Event-driven patterns, AWS Cognito, boto3, Serverless Software Architecture, AWS Secrets Manager
Skills:
S3, Lambda, ECS, Pyspark, Iam, Python, Sql, AWS, Lake Formation, Step Functions, Glue
Skills:
snowflake , Java, RDS, PostgreSQL, Spring Boot, J2EE, Sql, Angular, Microservices, Quicksight, React, Mockito, JUnit, Python, AWS