In this Role, you will get to
- Lead the team technically in improving scalability, stability, accuracy, speed and efficiency of our existing Data Platform
- Build, administer and scale Data Platform
- Be comfortable navigating the following technology stack: S3, Kubernetes, KubeFlow, Spark, SQL, Impala, Scala, java, Python, scripting (Bash/Python) etc.
- Work with experienced engineers to identify and build tools to automate many large-scale data platform management
What You’ll Need To Succeed
- Bachelor’s degree in Computer Science /Information Systems/Engineering/related field
- 6+ years of experience in Data Platform Administration eg. Hadoop, S3, AWS, Google Cloud etc.
- 6+ years of experience in Kubernetes Administration
- Good experience in Apache Spark performance tuning and debugging
- Good level understanding of JVM and either Java or Scala
- Experience in WorkFlow Scheduler eg. KubeFlow, AirFlow, Oozie etc.
- SQL experience eg. SparkSql, Impala, BigQuery, Presto/Trino, StarRocks etc.
- Experience debugging and reasoning about production issues is desirable
- Analytical problem-solving capabilities & experience.
- Systems administration skills in Linux
It’s great if you have
- Experience working with Open-source products
- Python/Shell scripting skills
- Working in an agile environment using test driven methodologies
