Senior Data Engineer (ETL Migration)

Senior Data Engineer for telecom ETL migration to Apache stack. Remote, full-time, RUB 250,000-270,000 net. IP contract.

Project: Telecom. Migration of an information system to Apache stack. Tasks: • Migrate ETL system of a telecom operator from Oracle/Informatica to Apache stack (S3, Airflow, NiFi, Kafka, Greenplum) • Transfer business logic from Informatica to high-performance SQL code • Write and optimize SQL for data marts and ETL processes • Develop DAGs in Airflow • Configure pipelines in NiFi (sources: Oracle, PostgreSQL; sinks: S3, file archives) • Develop Spark jobs (PySpark) for large volumes • Integrate with Kafka for streaming • Set up monitoring (Grafana/Prometheus) and track metrics • Investigate incidents on data quality and load errors • Code review

• Experience migrating ETL systems (preferably from Oracle/Informatica to Apache stack) • Deep SQL: window functions, complex joins, hierarchical queries • SQL optimization for highload (partitioning, indexes, explain plans) • Oracle (PL/SQL) and PostgreSQL • Greenplum and/or Clickhouse • Designing architectures on Apache Spark (PySpark), NiFi, Airflow • Hadoop (HDFS) and Hive • ETL monitoring: Grafana/Prometheus or Victoria Metrics • Experience with billing systems (preferably telecom domain) Nice to have: • Streaming: Kafka Streams / Spark Streaming • Informatica PowerCenter (understanding of migrating system) • Binary storage formats in Hadoop • Differences: Hive vs Impala, Greenplum vs PostgreSQL, Clickhouse vs PostgreSQL • Patroni for PostgreSQL/Greenplum clustering • S3 storage • CryptoPro or secure data transmission channels

Salary: 250,000 - 270,000 RUB net Employment: Full-time Work format: Remote Location: Russia, timezone MSK ± 3 hours Contract: Individual entrepreneur (IP) only Company: Top Selection