As a Senior Data Engineer, you will design, develop, and document data pipelines for various data types within a cloud infrastructure, focusing on performance and event-driven architecture. You will collaborate with cross-border teams and contribute to technology choices and architecture while implementing robust testing frameworks to ensure high-quality software delivery.
Tech stack/ knowledge:
- Mandatory:
- Proficient in relational databases and SQL.
- Proven big data experience, either from an implementation or a data science perspective.
- Strong coding experience in the likes of Java, Scala or Python.
- Experience with GCP tech stack (BigQuery, Dataflow, Cloud Composer, Google Cloud Storage) or other public cloud providers equivalents (AWS, Azure)
- Experience with Git, Maven, CI/CD (eg. Azure DevOps) and documentation tools
- Experience with Infrastructure as Code (eg. Terraform)
- Experience or affinity with IT Security concepts
- Nice to have:
- DBT
- Looker
- Data governance, Data lineage, Data Catalogue (Metadata)
- Apache Beam
- Spark
- Flink
- Kafka
- Automated testing framework
Foreign languages: English (advanced)
Here are your daily responsibilities:
- As a Senior Data Engineer, you contribute and are responsible for the design, development, test automation, hardening (security, stability, deployment) and documentation of components and data pipelines for the extraction and provision of structured and unstructured data within a cloud infrastructure or between cloud and on-premises systems.
- You contribute to technology choices and architecture of newly built components and services
- You have a strong focus on performance, large data sets and towards event-driven architecture
- You assess the efficiency of end-to-end processes by identifying and mitigating risks
- You continuously improve and optimize the ingestion, post-processing and reporting layer of the data platform
- You standardize and reutilize automation patterns as part of the data ingestion and processing stages
- You design and develop automated unit, integration and regression tests
- You prove strong analytical skills, proactivity and ability to work in cross-border international teams.
- You work in an Agile environment, collaborate closely with the Product Owner and Customer Journey Expert to interpret business requirements and estimate, plan, implement, test and deliver as part of short delivery cycles.
- You focus on delivering working software of high quality that satisfies customer requirements.
Top Skills
Java
Python
Scala
Similar Jobs
Fintech • Payments • Financial Services
As a Senior Data Engineer, you will design, develop, test automation, and document components and data pipelines for structured and unstructured data within cloud infrastructure. You'll focus on performance, identify risks, optimize processes, and work closely with Agile teams to deliver high-quality software that meets customer needs.
Top Skills:
JavaPythonScala
Fintech • Payments • Financial Services
The Data Engineer will design and implement data warehouse solutions, manage data ingestion and transformation processes, and create BI dashboards. The role requires strong analytical skills and collaboration with cross-border teams. Familiarity with IT Security and Google Cloud technologies is essential for managing financial risk metrics and ensuring high-quality software delivery.
Top Skills:
JavaPythonSQL
Fintech • Payments • Financial Services
The Data Engineer will design and implement data warehouse solutions, manage data ingestion and transformation, and create BI dashboards. They will work with international teams, optimize SQL queries, and support Continuous Integration and Delivery practices within financial markets technology.
Top Skills:
JavaPython
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
