Teams at emids

The Senior Consultant will develop automated testing frameworks for data pipelines, validate outputs, and integrate tests into CI/CD processes, leveraging AI for coding tasks.

Design and build scalable data pipelines and a modern data platform on AWS using Databricks and Lakehouse architecture. Implement batch and streaming processing, cluster management, vacuuming, and pipeline cost modeling. Establish and drive engineering standards including CI/CD, unit testing, security scanning, and practical AI integration. Operate at an architect level to provide technical guidance and ownership.

Design and build scalable data pipelines and modern data platform components on AWS Lakehouse using Databricks. Implement batch and streaming solutions, manage clusters and Delta Lake operations (vacuuming), optimize costs, and establish CI/CD, unit testing, security scanning, and AI-assisted development practices. Provide architect-level guidance and drive engineering standards.