Lead Data Engineer
Thanks for your interest in the Lead Data Engineer position.
Unfortunately this position has been closed but you can search our 940 open jobs by clicking here.
Job Requirements
-
BS in Computer Science or related field
-
6+ years of experience in the data and analytics space
-
Certification –preferably AWS Certified Big Data or any other cloud data platforms, big data platforms
-
4+ years experience developing and implementing enterprise-level data solutions utilizing Python (Scikit-lean, Scipy, Pandas, Numpy, Tensorflow) , Java, Spark, and Scala, Airflow , Hive and Python.
-
3+ years in key aspects of software engineering such as parallel data processing, data flows, REST APIs, JSON, XML, and micro service architectures.
-
2+ year of experience working on Big Data Processing Frameworks and Tools – Map Reduce, YARN, Hive, Pig, Oozie, Sqoop, and good knowledge of common big data file formats (e.g., Parquet, ORC, etc.)
-
6+ years of RDBMS concepts with Strong Data analysis and SQL experience
-
3+ years of Linux OS command line tools and bash scripting proficiency
Nice to have:
-
Kubernetes and Docker experience a plus
-
Prior working experience on data science work bench
-
Cloud data warehouse experience - Snowflake is a plus
-
Data Modeling experience a plus
Knowledge, Skills and Abilities:
-
A passion for technology and data analytics with a strong desire to constantly be learning and honing skills
-
Ability to deliver independently without oversight
-
Be productive even with ambiguity and highly fluid requirements during initial stages of projects
-
Flexibility to work in matrix reporting structure
-
Experienced in implementing large scale event based streaming architectures
-
Strong communication and documentation skills
-
Working knowledge of NoSQL, in-memory databases
-
Background in all aspects of software engineering with strong skills in parallel data processing, data flows, REST APIs, JSON, XML, and micro service architecture
-
Experienced in collaborating with cross-functional IT teams and global delivery teams
-
Solid Programing experience in Python - needs to be an expert in this 4/5 level
-
Working knowledge of data engineering aspects within machine learning pipelines (e.g., train/test splitting, scoring process, etc.)
-
Experience working in a scrum/agile environment and associated tools (Jira)