Hands-on implementation and optimization of large-scale GCP data pipelines and analytical solutions. Build production data pipelines using BigQuery, Cloud Composer, Dataflow, Dataproc, Cloud Storage, and Pub/Sub. Develop complex transformations following medallion architecture using Python, SQL, and Java/Scala.
JD NCS Customer Role : Data Engineer TSR : Big Data Engineer (G4) Location : Pune Job Description for Data Engineer 1. Resource Details ● Job Title: Data Engineer - 4-7 yrs exp ● Reports To: Data Management Lead ● Department: Technology ● Focus: Hands-on GCP Data Pipeline Implementation and Optimization ● Duration: 3-6 months 5. Resource Requirements Must-Have ● Extensive hands-on experience in building large-scale data pipelines and analytical solutions in a production environment. ● Deep technical expertise with GCP data services: BigQuery, CloudSQL(Federated Query), Cloud Composer (Airflow), Dataflow (Apache Beam), Dataproc (Spark), Cloud Storage, and Pub/Sub. ● Strong proficiency in Python, SQL, and Java/Scala for data processing and pipeline development. ● Proven experience implementing medallion architecture and complex data transformations at scale.
Similar Jobs
Information Technology • Consulting
Hands-on Big Data Engineer to develop ETL and analytics pipelines on Azure Databricks using PySpark and advanced SQL. Own end-to-end development, data ingestion, preparation, integration, testing, debugging and deployment. Must understand architectures involving Data Lakes, Data Warehouses and Data Marts. Strong communication required.
Top Skills:
AzureAzure DatabricksData LakeData MartData WarehousePysparkPythonSQL
Information Technology • Consulting
Hands-on Big Data Engineer responsible for PySpark development on Azure Databricks, writing advanced SQL, and working with DWH, data lakes, and marts. Own end-to-end development including ingestion, integration, testing, debugging, deployment, and operationalization. Strong communication required.
Top Skills:
Azure DatabricksCloud (Azure)Data LakeData MartData WarehousePysparkPythonSQL
Information Technology • Consulting
Develop and deliver end-to-end big data solutions on Azure Databricks using PySpark. Design and implement data ingestion, preparation, integration, and DWH/Datalake architectures. Write complex SQL, code, test, debug, and deploy pipelines. Collaborate and communicate effectively with stakeholders.
Top Skills:
Azure DatabricksData LakeData MartData WarehousePysparkPythonSQL
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
