Job Details

Data Platform Engineer (NRT/Spark) at Yandex Fintech. Developing a platform for NRT data processing, improving fault tolerance and performance of streaming processes. Commercial development experience of 4+ years.

Yandex Fintech is one of the company's key and most dynamically developing areas. It is an ecosystem of financial services integrated into the daily lives of millions of users and businesses. Key Fintech products include: Yandex Pay, Split, Save, Plus Card, credit products, Yandex ID, Yandex Pro. We are looking for an Apache Spark specialist with a deep understanding of the framework's internal architecture to join the platform data team. The main tasks involve developing a platform for NRT (near real-time) data processing, improving the fault tolerance and performance of streaming processes. What tasks await you: • Designing and developing a cloud data platform Optimize infrastructure and design new platform components using Yandex Cloud technologies. • Building a unified observability platform for DWH tables Create a single entry point for monitoring build statuses, data quality, and dependency analysis, including building datalineage and logging processes. • Accelerating current data delivery processes Optimize existing and build new reliable data delivery processes, including snapshotting, direct increment loading from audit tables and CDC, as well as implementing Spark Streaming or similar solutions to ensure data updates with minimal latency. • Building test environments Create isolated development, testing, and pre-production environments with automated verification processes. • Developing data processing frameworks Develop internal frameworks for data loading, transformation, and quality control.

We expect you to: • Have at least four years of commercial development experience • Understand database structures • Be proficient with various types of DBMS: PostgreSQL, Oracle, MongoDB, Greenplum • Have experience with CDC • Be confident working with S3-compatible storage

We care about your well-being and cover your lunches. And at the coffee points, you can always snack on fresh fruit. These are not all the bonuses – the full list is here.