Design, build, optimize, and deploy large-scale Spark/Scala data processing and ETL pipelines on Cloudera (CDH). Collaborate with data teams, ensure data integrity and security, troubleshoot performance, and implement version control and CI/CD for Spark applications.
Key Responsibilities:
- Develop, test, and deploy data processing applications using Apache Spark and Scala.
- Optimize and tune Spark applications for better performance on large-scale data sets.
- Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.
- Collaborate with data scientists, business analysts, and other developers to understand data requirements and deliver solutions.
- Design and implement high-performance data processing and analytics solutions.
- Ensure data integrity, accuracy, and security across all processing tasks.
- Troubleshoot and resolve performance issues in Spark, Cloudera, and related technologies.
- Implement version control and CI/CD pipelines for Spark applications.
Required Skills & Experience:
- Minimum 8 years of experience in application development.
- Strong hands on experience in Apache Spark, Scala, and Spark SQL for distributed data processing.
- Hands-on experience with Cloudera Hadoop (CDH) components such as HDFS, Hive, Impala, HBase, Kafka, and Sqoop.
- Familiarity with other Big Data technologies, including Apache Kafka, Flume, Oozie, and Nifi.
- Experience building and optimizing ETL pipelines using Spark and working with structured and unstructured data.
- Experience with SQL and NoSQL databases such as HBase, Hive, and PostgreSQL.
- Knowledge of data warehousing concepts, dimensional modeling, and data lakes.
- Ability to troubleshoot and optimize Spark and Cloudera platform performance.
- Familiarity with version control tools like Git and CI/CD tools (e.g., Jenkins, GitLab).
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Senior Salesforce Engineer responsible for designing and building Salesforce solutions: develop Apex, triggers, Lightning Web Components, Aura, and Visualforce; build and modernize integrations and platform experiences; lead Quote-to-Cash implementations; troubleshoot production (L3) issues; follow CI/CD and deployment best practices; participate in Agile ceremonies; monitor org performance and advocate Salesforce best practices.
Top Skills:
AgentforceAgenticApexAuraCopadoData CloudLightning Web ComponentsLwcQuote-To-CashSalesforceSlackVisualforce
Cloud • Information Technology • Productivity • Software • Automation
Provide technical support to customers via meetings, chat, and email; troubleshoot product, configuration, networking and cloud issues; empathize and analyze reported problems; use technical knowledge to resolve or escalate; collaborate with Product and Engineering; participate in on-call rotations and specified shift hours.
Top Skills:
APIsAWSBoomiGoogle Cloud PlatformGroovyHadoopJavaJavaScriptLinuxAzureNetSuiteOauth 2.0PostmanReactSalesforceWindows
Cloud • Information Technology • Productivity • Software • Automation
As a Product Support Engineer, you'll troubleshoot customer issues, develop solutions, and work with various technologies to ensure operational success while providing global support during remote and future in-office hours.
Top Skills:
Boomi AtomsphereCharles ProxyEltETLGroovyHadoopHttp/SJavaJavaScriptKubernetesLinuxNetSuiteOauth 2.0PostmanRancher DesktopReactRestSalesforceSftpSoapSsl/TlsTcp/IpWindowsWiresharkWsdl
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
