Data Engineer

Middle Data Engineer at Action tech. Remote-friendly, full-time. Build and scale data infrastructure on Hadoop and GreenPlum.

We have prepared infrastructure based on Hadoop and GreenPlum for loading, processing, and building data marts. We have described and loaded data from several products of the holding and are starting to describe and load the next ones. We plan to develop and scale the infrastructure, launch streaming data ingestion, increase the number of connected sources, develop ETL and Reverse ETL, work with data quality, and develop Data Catalog and Data Lineage. - Develop and refine ETL processes in Airflow - Connect new data sources - Create and refine data marts - Optimize SQL queries in GreenPlum and PostgreSQL - Work with large volumes of data - Develop data quality control processes

Self-sufficient, convincing, ambitious, with leadership qualities (or the ability to make management unnecessary), striving to build quality processes and, as a result, high-quality data collection, storage, and delivery. - At least 1 year of experience working with data - Excellent knowledge of SQL: JOIN, window functions, procedures - Ability to profile and optimize queries, understand execution plans - Understanding of relational database principles and DWH architecture: ODS, DDS, DM - Practical experience with Airflow and ETL processes - Python at a basic level - Knowledge of Linux and ability to work with the command line - Experience with Git - Understanding of Docker principles - Experience with big data (>100 GB) - Experience with DataHub

Full-time employment, remote work option.