Data Engineer / Big Data Engineer

Responsibilities:

Job Highlights:

• Experience in ETL tools or Data Pipeline

• Strong SQL skills

• Data integration, Data Warehouse, Big Data, Data Lake

Responsibilities for Data Engineer / Big Data Engineer:

• Create and maintain optimal data pipeline architecture on data platforms either data warehouse and data lake.

• Design optimal data models that suits with either structured, semi-structured or unstructured dataset to gain performance and efficiency of data pipeline process.

• Build infrastructure and framework required for data pipeline process using Oracle and Cloudera technologies in the most efficient way.

• Utilize data platforms to provide insights into business metrics: customer acquisition, operational efficiency, operational risk, regulation, etc.

• Prepare data & tools for data analyst and data scientist team.

• Design and build BI reports that let business users are able to analyze data by themselves.

• Design, implement, and maintenance internal process improvements: automating manual process, optimizing data delivery, capability expansion, etc.

• Work with stakeholders including the Executive, Product and Business unit to assist with data-related technical issues and support their data infrastructure needs.

• Work closely with data experts, IT engineers and partners in various functionalities of data projects: identify, planning, procurement, contract, implementation, maintenance, etc.

Qualifications:

Job requirements:

• Degree in Computer Science/IT or equivalent.

• Experience in data warehouse design.

• Relevant working with SQL as well as data integration & ETL tools.

• Experience with a variety of databases: Oracle, SQL Server, Spark, HBase, Hive etc.

• Experience with cloud platforms: GCP, AWS, Azure, etc.

• Working knowledge of software life cycles methodology either development and operation: DevOps, Agile, CI/CD, CMMI, ITIL, etc.

Additional Qualifications for Big Data Engineer:

• Working knowledge of message queuing, stream processing, distributed data processing.

• Experience with big data tools, especially Cloudera: NiFi, Spark, Kafka, etc.

• Experience building and maintaining RESTful APIs.

• Experience coding with various computer languages: Python3, Java, Scala etc.

It’s Great If You Have:

• Experience in business of banking and life assurance industries.

• Strong project management and organizational skills.

• Strong in systematic thinking, problem solving, good leadership, interoperability and orchestration skills.