Senior Data Engineer

Work you will do

• Design data warehouse using standard data models such as star schema, snowflake, and data vault.
• Assemble large, complex data sets that meet functional/non-functional requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, infrastructure, etc.
• Build the infrastructure with optimal extraction, transformation, and loading pipeline from diverse data sources.
• Implement data pipeline utilizing processing tools used in Big Data platforms.
• Work closely with stakeholders, and team members such as data scientists, developers, and business domain personnel.
• Identify and troubleshoot any issues that occurred in data pipelines.
• Analyze and translate business needs into long-term-solution data models.
• Evaluate existing data systems.
• Work with the development team to create conceptual data models and data flows.
• Provide coding standards to ensure the consistency within the system.
• Review modifications of existing systems for cross-compatibility.
• Implement data strategies and develop physical data models.

Qualifications

• Bachelor’s degree or higher in Computer Engineering, Computer Science, or IT-related fields.
• Knowledge of Hadoop-related tools. (Sqoop, Nifi, Hive, Spark, etc.)
• Knowledge of cloud platforms such as Azure, AWS, or Google Cloud.
• Experience in working with non-relational/relational databases. (SQL, MySQL, NoSQL, Hadoop, HDFS, etc.)
• Experience in building and improving Big Data pipelines, architectures, and data sets.
• Able to build processes supporting data transformation, data structures, metadata, dependency, and workload management.
• Able to simplify complicated terms to audiences.
• Eager to catch up on new technologies along with sharing them with others.