Job Description
Cloud Data Engineer ROCGJP00027944
A leading biotechnology company is seeking a highly skilled and motivated Cloud Data Engineer to join our growing data team.
The ideal candidate will have a strong background in developing data pipelines, implementing data models, and executing best practices for developing data products.
This role requires expertise in AWS, GitLab CI/CD, dbt, Snowflake, SQL, Python, Git, and DevOps, as well as hands-on experience with orchestrators such as Airflow and AutoMateNow, and data governance technologies like Monte Carlo and Collibra.
Cloud Data Engineer Pay and Benefits: Hourly pay: $70-$80/hr (pay varies based on candidate's experience)Worksite: Leading biotechnology company (South San Francisco, CA 94080)W2 Employment, Group Medical, Dental, Vision, Life, Retirement Savings Program, PSL40 hours/week, 7 Month Assignment Cloud Data Engineer Responsibilities: Develop Data Pipelines: Design, develop, and maintain robust, scalable, and efficient ETL/ELT pipelines to support diverse data sources and large-scale data processing.Data Modeling: Create and maintain data models and data architecture to ensure data integrity and optimum performance.Best Practices Implementation: Apply industry best practices in data warehousing, data governance, and data lifecycle management.Collaboration: Work closely with data scientists, analysts, and other stakeholders to gather requirements and deliver high-quality data products.Automation and CI/CD: Implement CI/CD pipelines using GitLab for automated testing, deployment, and integration of data workflows.Database Management: Manage and optimize Snowflake databases for performance, scalability, and cost-effectiveness.Coding and Scripting: Write efficient SQL queries and Python scripts to extract, load, and transform data.Version Control: Utilize Git for version control, ensuring traceable and manageable code changes.Orchestration: Utilize orchestration tools like Airflow and AutoMateNow to manage and automate data workflows.Data Governance: Implement and manage data governance and data observability solutions using technologies like Monte Carlo and Collibra.Monitoring and Optimization: Monitor data pipeline performance and troubleshoot issues to ensure reliability and efficiency.Documentation: Maintain comprehensive documentation for data pipelines, models, and processes. Cloud Data Engineer Qualifications: 5+ years of experience in data engineering or a similar role.Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.AWS Certified Data Analytics - Specialty or AWS Certified Solutions Architect.Proven experience with AWS cloud services related to data processing, such as S3, Redshift, Lambda, Glue, and Data Pipeline.Proficiency in designing and implementing CI/CD pipelines with GitLab.Experience with dbt (data build tool) for transforming data within the warehouse.Experience with orchestrators like Airflow and AutoMateNow.Experience with real-time data processing and streaming technologies (e.g., Kafka, Kinesis).Experience in a regulated industryExperience in a manufacturing environmentStrong knowledge of Snowflake with hands-on experience in data warehousing solutions.Expertise in SQL for data querying, manipulation, and optimization.Proficiency in Python for scripting and automation tasks.Familiarity with DevOps practices and tools.Knowledge of data governance and observability tools such as Monte Carlo and Collibra.Strong working knowledge of Git for version control.Strong problem-solving and analytical skills.Excellent communication skills, both written and verbal.Ability to work collaboratively in a fast-paced, dynamic environment.High attention to detail and commitment to producing high-quality work.