Data Engineer

Responsibilities:

• Design, create and maintain the new and existing optimal data pipelines to efficiently ingest, clean, transform, and aggregate data from various sources

• Design and implement dimensional data models in data warehouse and data mart

• Optimize data pipeline performance to maintain the speed,reliability and scalability

• Contribute to the development of in-house ETL framework

• Develop tools for data data quality, data observability and data management

• Collaborate with cross-functional teams including data analysts, data scientists, product managers and software engineers to identify data opportunities, assist with data-related technical issues and support their data needs.

Qualifications:

• At least 1 years of experience in data engineer or data-related roles

• Strong programming skills including advanced SQL knowledge and Python

• Experience with relational databases and NoSQL : MySQL, Postgres, MongoDB, Elasticsearch

• Experience with one of data orchestration tools : Airflow, Dagster, Mage AI

• Experience with distributed computing system : Spark

• Knowledge of Docker and Kubernetes is a plus

• Experience with AWS services is a plus : EC2, EMR, Lambda, S3

• Knowledge of data lakehouse and open table format is a plus : Iceberg, Hudi, Delta

• Knowledge of software development is a plus : OOP, SOLID, TDD, CI/CD