We are currently seeking a Databricks Engineer to join our team remotely in the United States.
Key Responsibilities Conceptualize and create an end-to-end vision for the seamless flow of data through successive stages.Design and implement data solutions tailored to customer needs, including data ingestion, pipeline building, and analytics within a dynamic technical stack.Provide thought leadership by recommending the most suitable technologies and solutions for various use cases, covering the entire spectrum from application layer to infrastructure.Collaborate seamlessly across technical stacks, with a strong focus on Databricks, SQL Server, and Azure.Formulate and maintain data models, establishing policies and procedures for functional design.Document the data architecture and environment to ensure an accurate understanding of the overall data landscape.Design, build, and deploy databases and data stores to support organizational requirements.Institute processes for governing the identification, collection, and utilization of corporate metadata, ensuring accuracy and validity.Implement methods and procedures for tracking data quality, completeness, redundancy, compliance, and continuous improvement.Evaluate and determine governance, stewardship, and frameworks for effective data management across the enterprise.Develop strategies for data capacity planning, security, life cycle management, scalability, backup, disaster recovery, business continuity, and archiving.Develop and deliver detailed presentations to effectively communicate complex technical concepts.Generate comprehensive solution documentation, including sequence diagrams, class hierarchies, and logical system views.Offer technical recommendations to senior managers and technical staff in the development and implementation of databases and documentation.Collaborate with project managers and business leaders on all projects involving enterprise data.Stay informed about upgrades and emerging database technologies through continuous research.Identify potential areas for policy and procedure enhancements, initiating changes where required for optimal data management.Adhere to Agile practices throughout the solution development process.Basic Qualifications 8+ years of experience with coding languages such as Python, SQL, and PySpark, with a focus on performance, security, scalability, and robust data integrations.6+ years of experience with Cloud-based systems and Enterprise Data Architecture, driving end-to-end technology solutions.5+ years of experience working with Databricks in Azure Cloud.3+ years of experience building data pipelines using Azure Data Factory, SSIS, Python, and Spark.Preferred Skills Production experience with core data platforms such as SQL Server, Databricks, Azure, Spark, and Hadoop.Hands-on knowledge of Cloud and Distributed Data Storage, including expertise in HDFS and ADLS.Strong understanding of data integration technologies, including Spark, Kafka, eventing/streaming, and Azure DataFactory.Professional written and verbal communication skills to effectively convey complex technical concepts.Valuable "Big 4" Management Consulting experience or exposure to multiple industries.Ability to travel at least 25%.
#J-18808-Ljbffr
