The Bigdata Engineer will design and develop ETL/Hadoop and analytics components, ensuring performance optimization and adherence to architectural standards, while managing distributed computing tasks and data ingestion workflows.
This is a remote position.
Responsibilities:
- As a developer, possess excellent Knowledge of distributed computing architecture, core hadoop component (HDFS, Spark, Yarn, Map-Reduce, H base, HIVE, Impala) and related technologies.
- Technical Design and development of ETL/Hadoop and Analytics services /components
- Contribute in end to end architecture and process flow
- Understand Business requirement and publish reusable designs
- Result oriented approach with ability to provide apt solutions.
- Proficient in performance improvement & fine-tuning ETL and Hadoop implementations
- Conduct code reviews across projects. Takes responsibility for ensuring that build and code adhere to architectural and quality standards and policies.
- Can work independently with minimum supervision.
- Strong analytical and problem solving skills
- Experience/Exposure to SQL, advanced SQL skills
Requirements
Skills Set:
- Strong understanding of distributed computing architecture, core hadoop component (HDFS, Spark, Yarn, Map-Reuduce, H base, HIVE, Impala) and related technologies.
- Hands on experience with batch data ingestion (Sqoop)
- Expert level understanding of relational data structure and RDBMS as well as NoSQL databases (Cassandra, MongoDB, Elasticsearch)
- Experience with automation/Scheduling of workflows/jobs (via shell-scripting, Tivoli)
- Solid Grasp of data storage formats (Parquet, Avro, HBase, Cassandra)
- Understanding of Agile methodologies as well as SDLC life-cycles and processes.
- Strong Understanding of Data warehousing and lakes
Similar Jobs
Digital Media • Information Technology • News + Entertainment
Design, develop, and maintain microservices-based Python applications and REST APIs. Implement asynchronous processing, ensure test coverage and code quality, deploy and operate services in Docker/Kubernetes, contribute to CI/CD pipelines, mentor junior engineers, collaborate with cross-functional teams, and optimize performance, scalability, and reliability.
Top Skills:
CeleryDjangoDjango Rest FrameworkDockerFastapiGit/GitlabGitlab CiJenkinsKubernetesMongoDBNoSQLPostgresPythonRabbitMQRedisSQL
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Lead business analysis for Risk Weighted Assets (RWA) and Counterparty Credit Risk (CCR) initiatives. Translate Basel III/IV and SA-CCR/IMM/EAD regulatory requirements into BRDs, FRDs, user stories, process flows and data mappings. Liaise with Market Risk, CCR, Regulatory Reporting, Finance, Quant and Technology teams to support capital calculations, RWA optimization and regulatory reporting for transformation programs.
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and develop web applications and UI/UX solutions, collaborate with stakeholders to define technical solutions, implement Python and JavaScript components, evaluate visualization tools and recommend approaches, deliver quality work in Agile teams under tight deadlines.
Top Skills:
JavaScriptPython
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
