Lead Big Data Developer

Summit Human Capital is seeking a highly motivated

Lead Big Data Developer

to support our client in the

financial services industry.

This position has the exciting opportunity to support a big data project with a Top 5 AWS partner across the US. The ideal candidate will meet the following criteria:
Requirements:
6+ years of development experience in Java AND Scala working on ETL projects in an Agile environment
4+ years of experience in big data projects
3+ years of experience with API development (REST APIs) and processing with JSON and csv files
2+ Years of experience working in a Lead role supporting design of applications and ETL projects
Strong knowledge of big data technologies including: AWS EMR, AWS EKS, Apache Spark, SQL & serverless technologies like AWS Dynamo DB, AWS Lambda
Unit testing using JUnit or ScalaTest
Desired:
Business Process Management experience using AWS functions
Python & Shell Scripting
Experience with performance testing tools such as Gatling or Jmeter
Responsibilities:
Comprehend Complex Business Requirements
Design and develop ETL pipelines to collect, validate, and transform data according to the requirement
Develop automated unit tests, functional tests and performance tests.
Support optimal data pipeline architecture
Design ETL jobs for seamless execution in AWS cloud environment
Reduce processing time and cost of ETL workloads
Lead cross-functional review meetings with peers and stakeholders
Continuous data quality checks.
Identify areas where machine learning can be used to identify data anomalies

#J-18808-Ljbffr