Salary Market Aligned
Location Kuala LumpurFULL_TIME
Consultant Lynnette Lee
Date posted 22 November 20192019-11-22 2020-01-21 it Kuala Lumpur MY MYR 100000 120000 120000 YEAR Robert Walters https://www.robertwalters.com.my https://www.robertwalters.com.my/content/dam/robert-walters/global/images/logos/web-logos/square-logo.png
An exciting Data Engineer position has just become available at a reputable insurance company based in Malaysia. In this critical role, you are responsible to review and assemble large and complex data sets that meet functional and non-functional requirements within the company.
About the Data Engineer Role:
Reporting to the Unit Head - you will be responsible for the design and code batch as well as real time ETL for the group, as well as collaborate with other team members for any big data implementation projects.
- Designing schemas, data models and data architecture for Hadoop and HBase environments
- Building and maintaining code for real time data ingestion using Java, MapR-Streams (Kafka) and STORM
- Implementing data flow scripts using Unix/Sqoop/Hive QL/Pig scripting
- Designing, building and support data processing pipelines to transform data using Hadoop technologies
- Designing, building data assets in HBASE, and HIVE
- Developing and executing quality assurance and test scripts
- Working with business analysts to understand business requirements and use cases
- BAU support for any data issues and change requests, document all investigations, findings, recommendations and resolutions
To succeed in this Data Engineer role, you must have minimum of three years' professional experience in data warehouse, operational data store, and large scale data architecture implementations in Unix or/and Windows environment.
- Minimum of two years' hands-on experience with Hadoop, MapReduce, Hive, Pig, Sqoop, Flume, STORM, SPARK, Kafka and HBASE
- At least three years' of solid hands-on development experience with Talend ETL or similar tools to transform complex data structure in multiple data sources environment
- Understanding of best practices for building Data Lake and analytical architecture on Hadoop
- Strong scripting/programming with UNIX, Java, Python, Scala, etc,
- Strong SQL experience with the ability to develop, tune and debug complex SQL applications
- Expertise in schema design, developing data models and proven ability to work with complex data
- Experience in real time data ingestion into Hadoop
This company is much more than an insurance company and is anything but traditional. They have the people and ethos to act with speed, innovation, enterprise and creativity. The scope of offer, the size of the business, the freedom and autonomy to drive your career forward all adds up to a great place to work.
If you are looking for an opportunity to be a Data Engineer specialist, this is the role for you. Great career progression opportunities await the right person in this exciting Data Engineer role.
Apply today or email me at email@example.com to discuss this new opportunity.
Agensi Pekerjaan Robert Walters Sdn Bhd
(formerly known as Robert Walters Sdn Bhd)
Business Registration Number : 729828-T
Licence Number : JTKSM 423C