Duties and Responsibilities Create data pipelines with Python, SQL, and SparkIdentify opportunities for improving existing systems to ensure scalabilityImprove visibility into existing data, making it easy to query and visualizeDesign and build schema and access plans for backend features that require an internal data storePerform QA with all code, including unit, integration, and / or regression testsCreate data quality checks for ingested and post-processed dataEnsure alerting and monitoring of automated pipelinesMaintain and enhance existing data productsBring new perspectives to data engineering problems and take initiativeMaintain a clean and secure python environment Skills Fluent in SQLProficient with python; understands the importance of maintainingComfortable with python dependency management tools (conda, venv, etc.
)Capable of building and running cost-efficient Spark jobsExperience with Cloud Technologies (AWS, Azure, or GCP), AWS preferred (Kinesis, SQS, Redshift, Athena, S3, and Lambda)Understands database systems, both relational (postgres, mysql, etc.)
and non-relational (mongodb, cassandra, etc.
)Familiarity with event systems (kafka, SQS, etc.
)Capable of auditing and securing dataExperience with data analytics platforms like Databricks or equivalent platformProficient with git, GitHub, pull requests, continuous integration and continuous deployment. What we offer: A collaborative and supportive work environment, recognized as a Great Place to Work.Cell phone allowance of $100Monthly parking pass reimbursedMedical, dental, and visions benefitsLife insuranceUnlimited PTOBring your pet to workYour pet insurance is covered (up to $100)401k with Company matchAnnual performance-based bonus Spot Pet Insurance is the fastest growing pet insurance company in North America.
Our commitment to an exceptional end-to-end customer experience and our data-driven approach have quickly established us as a leading pet insurance provider.
We're dedicated to providing pet parents with peace of mind by offering accessible and comprehensive coverage so their furry companions can lead happier, healthier lives.
To demonstrate this, we recently joined forces with MrBeast to find homes for 100 homeless pets and committed to giving each of them pet insurance for life!
Along the way, we've created a company culture that allows our employees to thrive, with perks like daily free meals, a pet-friendly office, and ridiculously fun company events every quarter.
Our dedication to fostering a positive and rewarding work environment for our team has even earned us a Great Place to Work certification.
The ideal candidate is passionate about designing cost efficient and robust data pipelines.
The data engineer will work on SPOT's backend team and work directly with the backend engineers on building and improving data warehouses, access, and processes.
They will also be expected to provide guidance and direction for improving or replacing existing data workflows for the data team.