Lead Data Engineer (Databricks)

At Godel Technologies, we are passionate about building innovative data platforms and software solutions that empower businesses around the world. We are growing and looking for a Lead Data Engineer with strong Databricks expertise to join our team. If you enjoy working with modern data technologies, solving complex data challenges, and designing scalable cloud data platforms — we would love to hear from you.As a Lead Data Engineer, you will play a key role in designing and implementing large-scale data platforms based on Databricks. You will lead the development of distributed data processing pipelines, ensure high performance and reliability of data systems, and guide engineering teams in implementing best practices for modern data architecture.You will work closely with Solution Architects, Data Engineers, Data Scientists, and business stakeholders to build secure, scalable, and high-performing data solutions that support advanced analytics, machine learning, and data-driven decision making.This is a hybrid role, which means we'd like you to work in the office occasionally, especially during client visits or other important company meetings. We'd also like you to be willing to take an occasional short business trips to Warsaw (approximately four times a year).ResponsibilitiesDesign and implement scalable data platforms and pipelines using Apache Spark on DatabricksLead the development of distributed data processing pipelines using PySpark and SparkSQLBuild and manage Databricks Workflows for orchestration, scheduling, monitoring, and error handlingOptimize Spark workloads by applying join strategies, shuffle optimization, caching, and partitioning techniquesDesign and maintain Delta Lake architectures, including schema evolution, ACID transactions, and performance tuningImplement data governance and access control using Unity Catalog, including permissions, lineage, and secure data sharingCollaborate with architects and engineering teams to design cloud-native data platformsEnsure data quality, observability, and reliability across pipelines and data productsLead performance optimization of large-scale data processing workloadsMentor and support other Data Engineers, contributing to engineering standards and best practicesParticipate in architecture discussions and contribute to the evolution of the company’s data engineering practicesRequirementsMust have:7+ years of experience in Data Engineering (including min. 5 years with Databricks)Strong experience in Databricks Data Platform for distributed data processingExcellent programming skills in Python and SQLStrong understanding of data modeling, data lakehouse architecture, and ELT/ETL patternsExperience designing scalable cloud-based data platforms (AWS / Azure)Knowledge of data governance, security, and access control best practices (Unity Catalog, dbt)Experience leading or mentoring engineers is a strong advantageStrong analytical thinking and problem-solving skillsExcellent communication and collaboration skillsFluency in English (at least B2)Nice to have:Data Streaming: KafkaDatabases: MS SQL (SSIS, SSAS), PostgreSQL, MySQLBI tools: PowerBI