Senior Data Engineer

A leading healthcare group is seeking a Senior Data Engineer to play a pivotal role in their strategic Data Commercialisation Initiative. This role empowers you to build robust data infrastructure that powers analytics, AI/ML pipelines and regulatory-compliant data products at scale. If you are passionate about healthcare data and want to be part of a dependable network of professionals, this is your chance to make a lasting difference.

What you'll do:

As a Senior Data Engineer you will:

• Design, build, and maintain robust ELT data pipelines using Databricks and PySpark, ensuring alignment with the team's advanced medallion architecture including Landing, Bronze, Silver, Gold, and Platinum layers.
• Support the migration of data infrastructure to AWS cloud by contributing to VPC injection processes, environment configuration tasks, and ADBC endpoint management for seamless transition.
• Implement comprehensive data governance frameworks utilising Unity Catalog, focusing on access control mechanisms, detailed data lineage tracking, and effective classification strategies.
• Build and optimise modular data transformation workflows using dbt to ensure testable and maintainable logic that supports scalable analytics solutions.
• Develop interactive Power BI reports and dashboards by applying advanced data modelling techniques and DAX formula development to empower organisational decision-making.
• Collaborate closely with business stakeholders to translate complex business questions into reliable and actionable data products.
• Engage in open communication with team members by raising concerns early and constructively pushing back on design decisions when necessary.

What you bring:

The ideal Senior Data Engineer brings:

• 5+ years of hands-on experience in data engineering or 3+ years with demonstrated technical maturity; proven ability to build scalable pipelines using ELT design patterns.
• Beyond pure engineering, you are expected to act as a consultant, providing design feedback and foresight for the platform’s growth over the next 3 to 5 years and should be able to explain the "why" behind your engineering choices.
• You are not a "yes man" and are willing to challenge opinions, ask questions, and suggest better ways forward
• Proficiency in Python and PySpark for complex data transformation tasks; adept at writing optimised code across large datasets with strong debugging skills.
• Solid SQL capabilities including query optimisation techniques for handling extensive analytical datasets efficiently.
• Deep familiarity with modern Lakehouse architecture such as Medallion architecture, Delta Lake, or Apache Iceberg; able to architect solutions beyond standard models.
• Hands-on experience with Databricks including environment setup procedures, cluster configuration management, and VPC injection processes.
• Expertise in Unity Catalog for implementing comprehensive governance frameworks covering access control policies and detailed lineage tracking.
• Advanced proficiency with dbt for transformation modelling; skilled at managing modular pipeline logic that supports maintainability at scale.
• Extensive experience with AWS or another major cloud provider; comfortable navigating cloud networking concepts like IAM roles and credential management systems.
• Proficient use of Git version control systems combined with CI/CD workflows for deploying robust data infrastructure solutions.
• Good Power BI skills including advanced data modelling techniques, DAX formula development expertise, and dashboard design capabilities.

What sets this company apart:

This healthcare group stands out as an industry leader dedicated to commercialising proprietary clinical data through innovative technology platforms. Their commitment to flexible working arrangements—transitioning from on-site onboarding to hybrid models—reflects their understanding of work-life balance needs among employees. The organisation fosters an inclusive culture where collaboration is valued above all else; every team member’s voice is heard during architectural discussions or project planning sessions.

What's next:

If you are ready to shape the future of healthcare data commercialisation while enjoying flexible working opportunities within an inclusive environment—apply now!

Apply today by clicking on the link provided.

Robert Walters SEA is acting as an Employment Business in relation to this vacancy.