COMPANY PROFILE:
CEF Solutions is a Consulting Services, and Business Process Outsourcing (BPO) company specializing in providing services such as BPO Operations, Operations Management, Staffing, Process Engineering and Innovation, and Cost Optimization Solutions to clients in the Customer Service industry.
We service some of the largest multinational companies in the world and are expanding quickly by delivering an unmatched end-to-end client experience.
Company Website: www.cem-lab.com
Job Description:
Our ideal candidate must have exceptional data management and data engineering background.
In today's fast evolving technology world, one aspect remains common reliance on data to drive the next wave of innovation. Strategic data analytics drives the best possible decision making and generate insights for new product developments. The team's core focus is developing best-in-class solutions that provide marketing and service organizations with a 360-degree view of the customers.
We are looking for a Big Data Engineer who will serve as the backbone of the Data Analytics organization and ensure both the reliability and applicability of the team's data products to the entire organization. The Big Data Engineer will work with a group of professionals from various departments at various level supporting the global top tier consumer electronics company.
Roles and Responsibilities:
- Architect highly scalable end-to-end pipeline using different open source tools, including building and operationalizing high-performance algorithms
- Understand how to apply technologies to solve big data problems with expert knowledge in programming languages like Java, Python, Linux, PHP, Hive, Impala, and Spark
- Implement complex big data projects with a focus on collecting, parsing, managing, analyzing, and visualizing large sets of data to turn information into actionable deliverables across customer-facing platforms
- Translate complex functional and technical requirements into detailed design
- Design for now and future success
- Hadoop technical development and implementation.
- Load from disparate data sets by leveraging various big data technology e.g. Kafka
- Pre-processing using Hive, Impala, Spark, and Pig
- Maintain security and data privacy in an environment secured using Kerberos and LDAP
- High-speed querying using in-memory technologies, such as Spark
- Follow and contribute best engineering practice for source control, release management, deployment, etc.
- Production support, job scheduling/monitoring, ETL data quality, data freshness reporting
- Other duties as assigned
Education:
- Bachelor's degree and/or Master's degree in Computer Science or Engineering required
Experience:
- Extensive experience working with both 1) big data platforms and 2) real-time / streaming deliver of data is essential.
- Minimum 5 years of related work experience
- 5-8 years of experience writing complex SQL queries.
- Minimum 3 years of Python development experience
- Minimum 3 years of demonstrated technical proficiency with Hadoop and big data projects
- Minimum 3 years of demonstrated experience and success in data modeling.
Knowledge, Skills, and Abilities:
- Ability to automate and implement ETLs to reduce manual work.
- Fluent in writing shell scripts [bash, korn]
- Writing high-performance, reliable, and maintainable code.
- Familiarity with and implementation knowledge of loading data using Sqoop.
- Knowledge and ability to implement workflow/schedulers within Oozie
- Experience working with AWS components [EC2, S3, SNS, SQS]
- Analytical and problem-solving skills, applied to Big Data domain
- Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala, and Spark
- Good aptitude in multi-threading and concurrency concepts.
- Proficiency in Microsoft Office
Other Qualifications:
- Ability to work in Philippines without any restrictions
- No restrictions for overseas business trip
- Background check and drug test
Work Hours / Location / Employment Type:
- Monday to Friday, 9:00 AM to 6:00 PM (eastern time zone)
- Location: Work from home, with potential for on-site
