Lead the design and implementation of real-time data pipelines and scalable data streaming solutions for healthcare data engineering projects.
Role Overview:
We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects. The ideal candidate will have deep experience in real-time data streaming, cloud-based data platforms, and large-scale data processing. This role requires strong technical leadership, problem-solving abilities, and the ability to collaborate with cross-functional teams.
Key Responsibilities:
- Lead the design, development, and implementation of real-time data pipelines using Kafka, Python, and Azure Databricks.
- Architect scalable data streaming and processing solutions to support healthcare data workflows.
- Develop, optimize, and maintain ETL/ELT pipelines for structured and unstructured healthcare data.
- Ensure data integrity, security, and compliance with healthcare regulations (HIPAA, HITRUST, etc.).
- Collaborate with data engineers, analysts, and business stakeholders to understand requirements and translate them into technical solutions.
- Troubleshoot and optimize Kafka streaming applications, Python scripts, and Databricks workflows.
- Mentor junior engineers, conduct code reviews, and ensure best practices in data engineering.
- Stay updated with the latest cloud technologies, big data frameworks, and industry trends.
Required Skills & Qualifications:
- 4+ years of experience in data engineering, with strong proficiency in Kafka and Python.
- Expertise in Kafka Streams, Kafka Connect, and Schema Registry for real-time data processing.
- Experience with Azure Databricks (or willingness to learn and adopt it quickly).
- Hands-on experience with cloud platforms (Azure preferred, AWS or GCP is a plus).
- Proficiency in SQL, NoSQL databases, and data modeling for big data processing.
- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines for data applications.
- Experience working with healthcare data (EHR, claims, HL7, FHIR, etc.) is a plus.
- Strong analytical skills, problem-solving mindset, and ability to lead complex data projects.
- Excellent communication and stakeholder management skills.
Top Skills
Azure Databricks
Docker
Kafka
Kubernetes
NoSQL
Python
SQL
Similar Jobs
Healthtech • Information Technology • Other • Software • Analytics
Seeking a Senior Backend Engineer with expertise in Scala, Clojure, Java, and Oracle databases for a migration initiative to AlloyDB on GCP.
Top Skills:
AlloydbClojureGCPJavaOracleScala
Healthtech • Information Technology • Other • Software • Analytics
The Senior Consultant will utilize Edifecs tools for healthcare integration, manage EDI transactions, and leverage SQL database skills for data transformation and mapping in a DevOps environment.
Top Skills:
Edifecs Healthcare Transaction RepositoryEdifecs Integration ConsoleEdifecs SpecbuilderEdifecs Transaction ManagerEdifecs Xengine ServerGitGroovyJavaJavaScriptJSONMySQLOracleRdbmsRestSoapSpecbuilderSQLSQL ServerXMLXslt
Healthtech • Information Technology • Other • Software • Analytics
The role involves designing and developing scalable web applications, utilizing both backend and frontend technologies, and providing mentorship.
Top Skills:
AngularJavaPostgres
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
