Job Description:
• Design, develop, and maintain scalable data pipelines using Databricks (PySpark) and Python.
• Build and optimize ETL/ELT processes within Azure cloud environments.
• Implement data models following modern Data Lakehouse principles (e.g., Medallion architecture).
• Ensure data quality, consistency, and performance across ingestion, staging, and curated layers.
• Collaborate with data architects, analysts, and business stakeholders to translate healthcare data requirements into technical solutions.
• Develop reusable data transformation logic and modular processing components.
• Support deployment processes following CI/CD and DevOps best practices.
• Monitor and optimize data workflows for performance, scalability, and reliability.
• Contribute to data governance, security, and compliance practices relevant to healthcare environments.
Requirements:
• Current knowledge of an using modern data tools like (Databricks,FiveTran, Data Fabric and others)
• Core experience with data architecture, data integrations, data warehousing, and ETL/ELT processes
• Applied experience with developing and deploying custom whl and or in session notebook scripts for custom execution across parallel executor and worker nodes
• Applied experience in SQL, Stored Procedures, and Pyspark based on area of data platform specialization.
• Strong knowledge of cloud and hybrid relational database systems, such as MS SQL Server, PostgresSQL, Oracle, Azure SQL, AWS RDS, Aurora or a comparable engine.
• Strong experience with batch and streaming data processing techniques and file compactization strategies.
Benefits:
• Allata is an equal opportunity employer.
• We celebrate diversity and are committed to creating an inclusive environment for all employees.
• This policy applies to all terms and conditions of employment, including but not limited to, recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.