We are seeking a hands-on and highly capable Data Engineer to support the delivery of a major multi-year data transformation initiative within a large, government-focused organisation. This program is establishing a unified enterprise data platform designed to uplift data quality, governance and analytics capability across the organisation. You will play a key role in designing, building and operating secure, reliable and scalable data pipelines and products using Microsoft Fabric and Azure services, working closely with governance, analytics and platform teams.
The role requires someone who can operate confidently across ingestion, transformation, modelling, governance integration and CI/CD, with a strong focus on privacy-by-design and data quality.
Build and maintain data ingestion and transformation pipelines using Microsoft Fabric (Pipelines, Dataflows Gen2, Notebooks/Spark, Lakehouse/OneLake, SQL Endpoints) and relevant Azure services.
Implement medallion architecture across Bronze/Silver/Gold layers and develop high-quality semantic and dimensional models for analytics and reporting.
Run and schedule Microsoft Purview scans, apply classifications and sensitivity labels, and ensure clear lineage from source through to consumption.
Implement robust data quality checks, automated tests, documentation, runbooks and data dictionaries to support operational excellence.
Apply security and privacy-by-design principles including RBAC, managed identities, masking/obfuscation and purpose-based access controls.
Use Git-based workflows and Azure DevOps to version, test, deploy and automate data pipelines, models and supporting artefacts.
Configure monitoring, logging and alerting, and support performance tuning and cost optimisation activities.
Collaborate with governance teams, data stewards, custodians and product owners to deliver prioritised use cases and reusable data assets.
5+ years’ experience in data engineering within enterprise or large-scale environments.
Strong hands-on experience with Microsoft Fabric (Pipelines, Dataflows Gen2, Notebooks/Spark, Lakehouse/OneLake, SQL Endpoints).
Excellent SQL and PySpark skills, with practical experience across ingestion, transformation and modelling.
Exposure to Azure data services such as Data Factory, Synapse or Databricks (Fabric-first experience highly regarded).
Demonstrated experience with Microsoft Purview, including scans, collections, glossary, lineage and classifications.
Strong understanding of medallion architecture, semantic modelling and Power BI data serving patterns.
Experience with CI/CD practices using Azure DevOps; Infrastructure-as-Code (Bicep/Terraform) advantageous.
Solid grasp of security, privacy, data quality and governance fundamentals in a regulated environment.
Experience working in government, regulated or compliance-focused environments.
Background contributing to engineering standards, reusable templates or platform accelerators.
Familiarity with metadata management, enterprise data governance or data stewardship practices.
Exposure to cost optimisation and operational support for cloud-based data platforms.
For a confidential chat, please contact George via email at gwright@exclaim-it.com.au. We would be more than happy to provide further information and support you through the application process.
Rely on our experience
We’ll help you find your next team member or opportunity.
