Design, build, and optimize large-scale ETL pipelines and distributed data applications (batch and streaming) using Spark and Scala. Ensure data quality and performance, deploy and manage workloads in cloud/container environments, troubleshoot production issues, and collaborate with product, engineering, and analytics teams to deliver scalable AdTech data solutions.
We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.
You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
- Build and optimize distributed data applications using Spark and Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Work with large datasets to ensure data quality, consistency, and performance.
- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, and cost efficiency.
- Deploy and manage data workloads in cloud and containerized environments.
- Troubleshoot production issues and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering or Big Data Engineering.
- Strong hands-on experience with Apache Spark and Scala.
- Experience building and maintaining ETL pipelines.
- Familiarity with Google Cloud Storage (GCS).
- Experience with Kubernetes (K8s).
- Strong SQL skills and understanding of distributed data processing.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with AWS and cloud-native data services.
- Familiarity with streaming technologies such as Kafka.
- Experience working on large-scale data platforms or AdTech systems.
- Exposure to orchestration tools such as Airflow.
Benefits
- Best-in-class salary: We hire strong talent and compensate accordingly.
- Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
- Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
- High-impact work: Build AI-first systems and products used at scale by global clients.
About Us
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.
Similar Jobs
Cloud • Information Technology • Security • Software
Develop, test, and deploy scalable authorization and IAM services using Go, Node.js, or Python. Design automated tests, improve CI/CD quality gates, define monitoring alerts and runbooks, and support reliable zero-downtime deployments. Partner with Product and UX across time zones to refine requirements and align technical roadmaps with business value. Contribute technical leadership, documentation, process improvements, and AI-assisted development practices in a distributed, remote-first environment.
Top Skills:
Ci/CdDockerElk StackGitopsGoIamJwtNode.jsOauth 2.0OidcPythonRedisTerraformVue
Cloud • Information Technology • Security • Software
Lead architectural strategy and hands-on development for secure, scalable microservices and privileged access management systems. Architect solutions using C#, .NET, Go, TypeScript, gRPC, PAM platforms, cloud infrastructure, Kubernetes, and multiple databases. Drive cross-team technical alignment, RFCs, incident resolution, CI/CD automation, testing rigor, and AI-assisted development governance. Mentor senior engineers and participate in on-call rotations.
Top Skills:
.NetAbp FrameworkAsp.Net CoreAWSBeyondtrustBmadC#Claude CodeCopilotCursorCyberarkDocumentdbGCPGoGrpcHashicorp VaultKargoKubernetesOrigin CiPlaywrightPostgresScimSQL ServerTypescriptWok
Cloud • Information Technology • Security • Software
Lead the architecture and development of secure, scalable microservices and cross-platform applications using C#, .NET, Go, and TypeScript. Design enterprise PAM and IDAM integrations, implement Zero Trust and least-privilege controls, own CI/CD and automated testing, and establish observability and reliability practices. Champion AI-assisted engineering workflows, mentor engineers, review RFCs, manage technical debt, and participate in incident response and on-call operations.
Top Skills:
.Net CoreAbp FrameworkAsp.Net CoreAWSBeyondtrustC#Claude CodeCursorCyberarkDockerDocumentdbGithub CopilotGoGrpcHashicorp VaultKargoKubernetesLinuxmacOSOrigin CiPlaywrightPostgresRestScimSQL ServerTypescriptWindowsWok
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
