Data Engineer (Hadoop, Scala)

Data Engineer (Hadoop, Scala) at Медиалогия. Full-time, remote or hybrid. 3-6 years experience. Develop and optimize ETL/ELT pipelines for big data.

Development and optimization of ETL/ELT pipelines for big data processing. Setup and administration of Hadoop/Spark infrastructure (HDFS, YARN, Hive, HBase). Optimization of distributed computing and queries. Working with streaming data (Kafka, Spark Streaming, RabbitMQ). Participation in designing and scaling data architecture. Automation of processes using DevOps practices (Docker, CI/CD).

Higher (or incomplete higher) education. Experience with Java/Scala and Big Data frameworks (Apache Spark, Hadoop). Deep knowledge of Hadoop ecosystem (YARN, HDFS, Hive, HBase, Parquet). Experience in performance optimization of distributed systems. Working with stream processing (Kafka, Spark Streaming). Skills in SQL/NoSQL (PostgreSQL, MS SQL, MongoDB). Nice to have: experience with PySpark, Flink, Airflow; knowledge of Elasticsearch, ClickHouse, Vertica; cluster and infrastructure setup skills.

Full-time employment. Schedule: 5/2. Work format: remote or hybrid.