Design, implement, and optimize Apache DolphinScheduler workflows and ETL pipelines. Integrate with Hadoop, Spark, Flink, Hive, and Kafka. Develop custom plugins, troubleshoot performance and scalability issues, monitor job execution and resource allocation, and maintain CI/CD, version control, and infrastructure automation while collaborating with data engineers and DevOps teams.
Key Responsibilities:
- Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
- Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
- Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
- Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
- Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
- Develop custom plugins and extensions for DolphinScheduler as needed.
- Monitor and optimize job execution, resource allocation, and workflow efficiency.
- Maintain best practices for CI/CD, version control, and infrastructure automation.
Required Skills & Qualifications:
- Strong experience with Apache DolphinScheduler in a production environment.
- Proficiency in Java, Python, or Scala for workflow scripting and automation.
- Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
- Understanding of workflow orchestration, DAG execution, and scheduling strategies.
- Strong problem-solving skills and ability to work in a fast-paced environment.
Similar Jobs
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Design and implement a unified orchestration layer coordinating vendor and internal LLMs and human agents. Build production-grade Python services, drive ML projects from design to delivery, ensure secure, scalable AI-enabled customer experiences, and collaborate across teams to meet security, compliance, and performance standards.
Top Skills:
Aws BedrockDeep LearningGoogle AdkLanggraphLangsmithLlmsPythonVertex Ai
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
The role involves managing enterprise customer relationships, implementing territory strategies, and ensuring customer success with Atlassian's solutions in India.
Top Skills:
AICloud-Based SolutionsCRMDevOpsEnterprise Service ManagementSaaSScaled AgileWork Management
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Lead end-to-end delivery of contact center automation and digital transformation programs for banking clients. Manage project financials, RAIDs, escalations, stakeholder relationships, and delivery teams to ensure on-time, on-budget outcomes.
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
