Design, build, optimize, and maintain high-performance Spark-based data pipelines using Scala/Java and Hive on Hadoop/CDP. Own full project lifecycle, enforce coding best practices, troubleshoot Spark/Hive/YARN performance, and collaborate with stakeholders to deliver scalable data solutions.
We need a Senior Data Engineer with 10+ years exp proficient in Spark, Scala/Java, and Hive, with extensive hands-on development experience in the Big Data Ecosystem.
Key Responsibilities:
- Design, implement, and optimize highly performant data pipelines using Spark, Scala/Java, and Hive on platforms like Cloudera Data Platform (CDP) or other Hadoop echo systems.
- Take complete ownership of complex data engineering projects within the big data ecosystem, covering the entire lifecycle from initial design and development to deployment and ongoing maintenance.
- Develop robust and efficient Hive queries for extensive data analysis and reporting.
- Champion and enforce best practices and coding standards for new and existing data flows to ensure they are robust, scalable, secure, and maintainable using Spark, Scala/Java, and Hive within the big data ecosystem.
- Diagnose, troubleshoot, and resolve complex issues related to Spark, Scala/Java, and Hive applications and YARN resource management, implementing performance optimization solutions.
- Proactively collaborate with stakeholders, working closely to develop solutions with full commitment and accountability.
Technical Skills & Experience:
- Proven hands-on development expertise with Apache Spark
- Strong programming proficiency in Scala and/or Java
- In-depth knowledge and practical experience with Hive, including query optimization and data analysis.
- Experience with data platforms such as Cloudera Data Platform (CDP) is highly desirable.
Education:
- Bachelor’s / Master's degree/University degree or equivalent experience
Similar Jobs
Agency • Information Technology
Design, build, and optimize end-to-end data pipelines for large structured and unstructured data. Implement near-real-time ETL, data validation, monitoring, and performance optimization. Collaborate with stakeholders, document designs and workflows, and provide technical guidance to the team.
Top Skills:
Amazon RedshiftSparkAzure Data FactoryAzure DatabricksAzure DevopsAzure Storage AccountsCi/CdDatabricksDelta LakeAzurePythonScalaSnowflakeSQL
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Senior Salesforce Engineer responsible for designing and building Salesforce solutions: develop Apex, triggers, Lightning Web Components, Aura, and Visualforce; build and modernize integrations and platform experiences; lead Quote-to-Cash implementations; troubleshoot production (L3) issues; follow CI/CD and deployment best practices; participate in Agile ceremonies; monitor org performance and advocate Salesforce best practices.
Top Skills:
AgentforceAgenticApexAuraCopadoData CloudLightning Web ComponentsLwcQuote-To-CashSalesforceSlackVisualforce
Cloud • Information Technology • Productivity • Software • Automation
Provide technical support to customers via meetings, chat, and email; troubleshoot product, configuration, networking and cloud issues; empathize and analyze reported problems; use technical knowledge to resolve or escalate; collaborate with Product and Engineering; participate in on-call rotations and specified shift hours.
Top Skills:
APIsAWSBoomiGoogle Cloud PlatformGroovyHadoopJavaJavaScriptLinuxAzureNetSuiteOauth 2.0PostmanReactSalesforceWindows
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
