Data Engineer

Data Engineer – Databricks Cleanroom Location: London, UK (Hybrid, 2 days to office per week) Type: Contract (Outside IR35) Duration: 6 Months Initially : Key responsibilities - - Architect and configure Databricks Clean Rooms (Delta Sharing-based) across clouds and workspaces, including cross-cloud and cross-region collaboration

• Set up Unity Catalog metastores, catalogs, and sharing so only approved tables, views, and volumes enter each clean room
• Build and approve clean room notebooks and queries (SQL, Python) for joint analytics, audience overlap, and ML scoring
• Apply privacy controls: row filters, column masks, aggregation thresholds, and pseudonymisation or hashing of join keys
• Define collaborator onboarding, output approval workflows, and revocation processes
• Configure audit logging, lineage, and monitoring for every clean room run
• Work with legal, privacy, and security teams to translate data-sharing agreements into technical controls
• Produce architecture diagrams, runbooks, and reference patterns for reuse Required skills - - 6+years in data engineering or platform engineering, with 2+ years on Databricks.
• Strong Unity Catalog, Delta Lake, and Delta Sharing experience.
• Solid SQL and Python, plus Terraform or other infrastructure-as-code.
• Understanding of privacy-enhancing techniques and regulations (GDPR, UK GDPR, CCPA, sector rules).
• Experience with IAM, network security, and audit on AWS or Azure. Desirable - - Prior clean room experience (Databricks, LiveRamp, or similar)
• Familiarity with differential privacy, tokenisation, or confidential computing
• Databricks certifications (Data Engineer Professional, Platform Administrator)