Job Details

Python Developer with 5+ years of experience in data tools development, focusing on Apache Airflow and dbt. Company: Arenadata. Remote/Office position.

Development of product functionality based on Apache Airflow: development of custom operators, hooks, and providers; Development of dbt modules: creation of adapters for new data sources, development of custom plugins; Designing the architecture of modules and connectors; Interaction with architects and the product team to define the data platform development strategy; Development and support of test infrastructure: unit tests, integration tests; Preparation of architectural documents and technical notes on adopted solutions; Defense and justification of architectural and technical solutions; Participation in code review; Review of user and technical documentation for products.

5+ years of experience in Python development, with a significant portion in product development of data tools; Practical experience with dbt Core: understanding of architecture, experience in developing or modifying adapters; Knowledge of the Big Data stack: Apache Hadoop ecosystem (HDFS, YARN, MapReduce), Apache Spark / PySpark, Apache Ozone, Trino, Apache Hive; Understanding of distributed storage and computation principles; 3+ years of experience with various DBMS; Knowledge of the internal structure of Apache Airflow: architecture, scheduler, executors (Celery, Kubernetes), plugin system, providers API; Understanding of dbt Core architecture: execution lifecycle, adapter system, data source interaction protocol, Jinja compilation, manifest and artifacts; Proficient in Python 3; Experience developing connectors/drivers for DBMS: knowledge of DB-API 2.0, SQLAlchemy, ODBC/JDBC; Knowledge of the principles of operation and functioning of Linux operating systems; Experience with Git; Understanding of Continuous Integration and Continuous Delivery (CI/CD) principles; Containerization experience: Docker, Docker Compose; understanding of Kubernetes; Decomposition of development tasks and ability to estimate effort.