Develop and deliver end-to-end big data solutions on Azure Databricks using PySpark. Design and implement data ingestion, preparation, integration, and DWH/Datalake architectures. Write complex SQL, code, test, debug, and deploy pipelines. Collaborate and communicate effectively with stakeholders.
Overall 6.5 - 10 yrs of IT experience preferably in cloud· Min 5 years in Azure Databricks on development projects· Should be 100% hands on in Pyspark coding· Should have strong SQL expertise in writing advanced/complex SQL queries· DWH experience is a must for this role· Experience in programming using Python is an advantage · Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements· Should be able to understand system architecture which involves Data Lakes, Data Warehouses and Data Marts· Experience to own end-to-end development, including coding, testing, debugging and deploymentExcellent communication is required for this role
Similar Jobs
Information Technology • Consulting
Hands-on implementation and optimization of large-scale GCP data pipelines and analytical solutions. Build production data pipelines using BigQuery, Cloud Composer, Dataflow, Dataproc, Cloud Storage, and Pub/Sub. Develop complex transformations following medallion architecture using Python, SQL, and Java/Scala.
Top Skills:
AirflowApache BeamBigQueryCloud ComposerCloud StorageCloudsqlDataflowDataprocGCPJavaPub/SubPythonScalaSparkSQL
Information Technology • Consulting
Hands-on Big Data Engineer to develop ETL and analytics pipelines on Azure Databricks using PySpark and advanced SQL. Own end-to-end development, data ingestion, preparation, integration, testing, debugging and deployment. Must understand architectures involving Data Lakes, Data Warehouses and Data Marts. Strong communication required.
Top Skills:
AzureAzure DatabricksData LakeData MartData WarehousePysparkPythonSQL
Information Technology • Consulting
Hands-on Big Data Engineer responsible for PySpark development on Azure Databricks, writing advanced SQL, and working with DWH, data lakes, and marts. Own end-to-end development including ingestion, integration, testing, debugging, deployment, and operationalization. Strong communication required.
Top Skills:
Azure DatabricksCloud (Azure)Data LakeData MartData WarehousePysparkPythonSQL
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
