Designs, deploys, and operates production-scale data platforms in fully air-gapped environments. Builds Airflow and Spark pipelines for high-volume sensor, satellite, and third-party data; develops PostgreSQL, TimescaleDB, PostGIS, and DuckDB solutions; manages Docker-based deployments and MinIO storage; and handles data quality, observability, documentation, troubleshooting, and SLA compliance without cloud services.
We're hiring a Data Engineer (7+ years) to design deploy independently, and operate production-scale data platforms in fully air-gapped, secure customer environments - no cloud dependency, with manual deployment and troubleshooting expected. The ideal candidate has hands-on experience with Apache Airflow, Spark, PostgreSQL, Docker, Python, SQL, and S3-compatible storage (MinIO), along with strong knowledge of geospatial and time-series data, and is comfortable owning infrastructure end-to-end without managed cloud services - from data pipelines to deployment to troubleshooting in fully isolated environments.
Requirements
- 7+ years of experience in Data Engineering and production-scale data platforms
- Design and manage Apache Airflow pipelines for high-volume sensor, satellite, and third-party data ingestion within isolated environment.
- Build and optimize Apache Spark workloads for batch processing, geospatial analytics, and large-scale aggregations
- Develop and maintain PostgreSQL, TimescaleDB, PostGIS, and DuckDB-based data solutions
- Implement efficient data ingestion, transformation, and bulk-loading pipelines without external connectivity
- Containerize and deploy applications using Docker and Docker Compose in air-gapped environments
- Work with S3-compatible object storage (e.g., MinIO) for on-prem data storage
- Proven experience deploying via Docker/Docker Compose in air-gapped or restricted-network environment.
- Comfortable working in Ubuntu-based environments with manual/local infrastructure management
- Strong proficiency in Python, SQL, Airflow, Spark, and PostgreSQL
- Experience with geospatial and time-series data
- Collaborate with customer teams to translate business requirements into data products
- Ensure data quality, lineage, observability, performance, and SLA compliance
- Troubleshoot production issues on-site/remotely without cloud tooling support
- Create and maintain technical documentation, data models, and operational run-books
- Strong communication, stakeholder management, and problem-solving skills
Benefits
- Comprehensive insurance coverage that gives you peace of mind, so you can focus on doing your best work
- Flexible work arrangements designed to support sustained productivity, personal well-being, and work-life balance
- Continuous learning and accelerated skill development through hands-on projects and mentorship from experienced industry leaders
- Global client exposure across 20+ countries, offering real-world experience with diverse markets and business environments.
- Opportunity to work on high-impact, large-scale projects that have collectively generated over $1B in measurable business value
- Competitive, market-aligned compensation packages that recognize performance, expertise, and long-term contribution
- Monthly demo days that celebrate innovation, showcase your work, and give you a real voice in what we build
- Annual recognition programs and performance-driven awards in a truly meritocratic environment
- Referral bonuses that reward you for helping grow a strong, like-minded team
- A strong problem-solving culture with opportunities to tackle meaningful, real-world challenges
- A positive, people-first workplace that supports happiness, balance, and long-term growth
Similar Jobs
Blockchain • Fintech • Software • Cryptocurrency • Metaverse
Build and operate scalable financial market data infrastructure, including source evaluation, ingestion, cleansing, modeling, real-time and batch processing, quality governance, reconciliation, monitoring, and data services. Develop Flink-based pipelines supporting trading, research, and AI applications. Partner with product, trading, compliance, procurement, and engineering teams to define data contracts, source strategies, and consistent financial data semantics across markets.
Top Skills:
Anomaly DetectionApache FlinkApache KafkaSparkCi/CdClickhouseData GovernanceData LineageData ModelingDistributed StorageDorisElasticsearchHbaseJavaKnowledge GraphsMetadata ManagementPythonRagScalaSQLTask Orchestration
Blockchain • Fintech • Software • Cryptocurrency • Metaverse
Build and operate production-grade AI data pipelines and financial knowledge services for equities content. Responsibilities include processing text, tables, documents, and multimedia; creating knowledge bases and search indexes; engineering RAG retrieval and citation systems; integrating LLMs and NLP models; managing orchestration, APIs, quality, cost, observability, reliability, and scalable production operations.
Top Skills:
APIsAsynchronous Task ProcessingCachingDistributed SystemsFull-Text SearchJavaKnowledge EngineeringLlmsMachine LearningMessage QueuesNlpPythonRagVector Search
Social Impact • Agriculture
Designs, develops, and maintains scalable Azure data infrastructure, pipelines, storage layers, and integrations. Provides technical leadership, supports data literacy, collaborates with analysts and stakeholders on data solutions, and oversees external consultants. The role promotes platform adoption, documentation, governance, security, privacy, data quality, and access controls while driving continuous improvement. It also develops optimized data models and automated workflows using SQL, Python or PySpark, and infrastructure-as-code tools.
Top Skills:
Azure Data FactoryAzure Data LakeAzure SqlAzure Synapse AnalyticsBicepC#C++Github Ci/CdJavaAzureNosql DatabasesPower BIPysparkPythonRelational DatabasesSalesforceSQLTerraform
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
