Overview

We are looking for a Lead Data Engineer for the Machine Learning (ML) engineering development team. The primary focus will be to gather requirements from ML/DS teams and identify the optimal solution. Then design, implement, monitor and maintain these scalable distributed big data pipelines for different big data ML use-cases. You will be working with Data Scientists to train, refresh and serve models using big data ML pipelines.

Responsibilities

• Collaborate with ML engineers and Data Scientists to gather requirements.
• Design and Implement ETL big data pipelines to train ML models.
• Selecting and integrating a variety of big data tools and frameworks required for processing
• Lead and conduct project activities based on an agile approach
• Develop, test and deploy customizations and new functionality based on changing business needs

Skills and Qualifications

• Minimum of 7+ years relevant experience.
• Strong desire to take ownership of a set of business needs and drive delivery of operational and technical processes to meet these product needs
• Experience with business Intelligence full stack development with technology e.g. Python, Snowflake and PowerBI in AWS environment
• Software development background is a plus
• Strong understanding and experience working in an agile development methodology as a product owner