Senior Data Engineer

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:Senior Data EngineerResponsibilities:Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-timeLeveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analyticsWorking closely with analysts and business stakeholders to develop analytics modelsContinuous delivering on Hadoop and other Big Data PlatformsAutomating processes where possible and are repeatable and reliableWorking closely with QA team Requirements:Minimum 5 years of hands-on work with Spark & ScalaAt least 7 years of professional Python developmentMinimum 5 years of working with Linux environmentsSolid background in distributed data processing engines (e.g., Spark)Practical knowledge of Hadoop ecosystem components (Hive, Oozie, MapReduce, etc.)Strong to advanced SQL proficiencyProven track record of designing and implementing data flowsExpertise in Bitbucket and GitAWS certification or practical experience with AWS servicesExperience with unit testing frameworks and tools (JUnit 5, Mockito, Spark testing frameworks)Understanding of code versioning and branching strategiesAdvanced level of English (written and spoken) Offer:Private medical careCo-financing for the sports cardConstant support of dedicated consultantEmployee referral program