Senior Data Engineer (m/f)
Consultor Ines Buekenhout
Data de publicação 27 de Agosto de 2018 2018-08-272018-10-19 it Avenida da Liberdade, 110 1269-046, Lisbon Robert Walters Portugal
A data driven start-up, focused on providing insights by introducing more structure with A.I. techniques, is looking for a Senior Data Engineer to lead the team.
As their data engineering tech lead you will:
- Extend and maintain the data pipeline, by ensuring deadlines are met as well as quality standards, using big data technologies such as Spark;
- Be responsible for the DataLake expansion and maintenance (Spark, Scala, AWS, Apache Airflow), infrastructure setup for data processing pipelines (EMR, Spark, Airflow, Automation, Docker);
- Design and build big data processing architectures (AWS solutions/architecture design, Python, ...);
- Data Mart/Warehouse design (SQL database/data warehouse design, Python, data modeling);
- You will also make sure that the company stays competitive by using the latest and most suitable data processing technologies.
You have a master in computer engineering (or similar), 5 years of relevant experience with at least 3 years of Python development experience, a deep understanding of databases/data warehouses/data lakes as well as experience with big data technologies (such as Hadoop, Spark, Airflow, Cassandra, Elasticsearch). You will also have a good level of spoken and written English.
Ideally, you have experience building on top of Amazon Web Services, knowledge of Scala and are able to lead data engineering projects as a tech lead.
If you are looking for a dynamic start-up that offers remote work as well as relevant training, this might be the opportunity for you!