Big Data Engineer (Databricks)

Required qualifications

• Experience with Databricks Asset Bundles (DAB).
• Experience using Azure DevOps.
• Knowledge of incremental processing and schema evolution.
• Healthcare or Insurance industry experience.
• Experience with cloud-based data platforms.

Key responsibilities

• Develop and maintain data pipelines using Databricks.
• Support the design and optimization of Delta Lake tables.
• Work with Unity Catalog to support data governance and access management.
• Participate in CI/CD processes using Azure DevOps and Databricks Asset Bundles (DAB).
• Implement incremental data processing and support schema evolution requirements.
• Collaborate with team members to troubleshoot data quality and performance issues.
• Assist with data modeling activities and platform optimization efforts.
• Document solutions and follow engineering best practices.
• Ideally 5 YOE
• Strong expertise in Apache Spark, including delivery on Databricks

Conditions

• Contrato a largo plazo.
• 100% Remoto.
• Vacaciones y PTOs
• Posibilidad de recibir 2 bonos al año.
• 2 revisiones salariales al año.
• Clases de inglés.
• Equipamiento Apple.
• Plataforma de cursos en linea
• Budget para compra de libros.
• Budget para compra de materiales de trabajo
• mucho mas..

Nice to have

• Azure Databricks certifications.
• Exposure to Medallion Architecture.
• Data quality and monitoring framework

We are looking for a Data Engineer with hands-on experience in Databricks, Python, and SQL to support the development and optimizationof modern data solutions. The ideal candidate will contribute to data pipeline development, Delta Lake implementation, data governanceinitiatives, and CI/CD processes while working closely with senior engineers and cross-functional teams.