A Unity Catalog-native AI Governance & Assurance Platform built on Databricks that assesses real-world AI systems against leading governance, risk, assurance, and privacy frameworks.
The platform evaluates 25 publicly cited AI systems across 8 sectors and demonstrates how governance requirements can be translated into measurable controls, assurance testing, accountability structures, and risk reporting.
Framework coverage includes:
- ISO/IEC 42001
- NIST AI Risk Management Framework (AI RMF)
- Australian Privacy Principles (APPs)
- OWASP Top 10 for LLM Applications (2025)
The project combines AI Governance, Responsible AI, AI Risk Management, AI Assurance, and Data Governance into a single governance-oriented analytics platform.
| Metric | Value |
|---|---|
| AI Systems Assessed | 25 |
| Industry Sectors | 8 |
| Risk Scores Generated | 150 |
| Governance Controls Evaluated | 20 |
| Control-to-Use Case Mappings | 500 |
| Risk Register Entries | 25 |
| ISO 42001 Maturity Assessments | 20 |
| Stewardship Assignments | 6 |
| Assurance Test Prompts | 30 |
| OWASP Coverage | 10 / 10 Categories |
As organizations deploy AI across critical business functions, governance teams face increasing challenges in:
- Identifying AI-related risks
- Demonstrating regulatory compliance
- Validating AI controls
- Tracking accountability
- Producing audit-ready evidence
- Measuring the effectiveness of governance controls
Many governance programs focus on documentation and policy creation but lack measurable assurance outcomes.
This project demonstrates how governance frameworks can be operationalized using structured datasets, governance controls, assurance testing, and measurable performance indicators.
One of the platform's core capabilities is governance traceability.
The platform creates evidence-based mappings between:
OWASP Top 10 Risks
↓
NIST AI RMF Controls
↓
ISO/IEC 42001 Controls
↓
Privacy Framework Controls
↓
Assurance Results
This enables technical control validation to be linked directly to governance and compliance obligations.
The platform includes a practical AI Assurance Framework aligned with the OWASP Top 10 for LLM Applications (2025).
Governance controls are validated through adversarial testing and manual assurance review.
- PII detection
- Sensitive data identification
- Presidio-based privacy controls
- Data redaction
- Prompt injection detection
- Adversarial prompt identification
- Heuristic pattern matching
- Rule-based validation
- Token limits
- Character limits
- Resource consumption controls
- Abuse prevention safeguards
- Content moderation
- Toxicity detection
- Harmful content classification
- Safety validation
- Output PII detection
- Generated response review
- Data leakage prevention
- Privacy redaction
The assurance framework evaluates the effectiveness of governance controls through adversarial testing.
| Metric | Result |
|---|---|
| Total Tests Executed | 30 |
| Adversarial Tests | 24 |
| Control Tests | 6 |
| Controls Effective | 21 / 24 |
| Control Effectiveness | 87.5% |
| Attack Success Rate | 12.5% |
| Attacks Resisted | 21 |
| Attack Successes | 3 |
| OWASP Coverage | 10 / 10 Categories |
All adversarial outcomes were manually reviewed and documented to provide defensible assurance evidence.
The primary governance metric is Control Effectiveness, which measures the proportion of adversarial scenarios successfully resisted by the AI Governance Assurance Framework.
The platform evaluates evidence against key Annex A control areas including:
- Policy documentation
- Governance oversight
- AI impact assessments
- Lifecycle management
- Data lineage and governance
- Model transparency and documentation
- Intended-use restrictions
- Third-party dependency management
Coverage spans all four functions.
- Governance structures
- Accountability models
- Risk management policies
- Organizational oversight
- Context establishment
- Stakeholder analysis
- Risk identification
- Impact assessment
- Performance monitoring
- Fairness evaluation
- Transparency assessment
- Explainability analysis
- Risk treatment
- Incident response
- Human oversight
- Escalation mechanisms
The platform maps governance controls to:
- APP 3 – Collection of Personal Information
- APP 6 – Use and Disclosure of Personal Information
- APP 10 – Quality of Personal Information
- APP 11 – Security of Personal Information
Each AI use case is assessed across six independent governance dimensions:
- Privacy Risk
- Security Risk
- Bias Risk
- Regulatory Risk
- Reputational Risk
- Operational Risk
Rather than relying on a single aggregated risk score, the platform uses multidimensional assessment to improve governance visibility and support targeted risk treatment decisions.
The platform includes assessments of publicly cited AI systems across government, financial services, and other regulated sectors.
Examples include:
Digital Transformation Agency (DTA)
AI-assisted procurement decision support.
Digital Service Standard (DSS)
AI disclosure and transparency case study.
Australian Taxation Office (ATO)
Tax and superannuation compliance risk differentiation.
Commonwealth Bank of Australia (CBA)
Real-time fraud and scam detection.
BioCatch Cross-Bank Pilot
Behavioural biometrics-based fraud detection with publicly disclosed governance limitations.
The platform includes:
- AI Use Case Inventory
- AI Risk Register
- Governance Control Library
- ISO/IEC 42001 Maturity Assessments
- NIST AI RMF Mappings
- Privacy Framework Mappings
- RACI Accountability Assignments
- Governance Evidence Tracking
- Assurance Testing Results
- Control Effectiveness Reporting
- Databricks
- Unity Catalog
- Delta Tables
- Python
- SQL
- GitHub
- GitHub Actions
- Dependabot
- ISO/IEC 42001
- NIST AI RMF
- OWASP Top 10 for LLM Applications (2025)
- Australian Privacy Principles
ai_governance/
├── data/
├── docs/
├── images/
├── notebooks/
├── scripts/
├── sql/
└── tests/
Public AI governance disclosures vary significantly in maturity and detail.
Each assessment artefact is tagged as:
disclosed
or
inferred_by_assessor
This provides transparency regarding what was explicitly evidenced versus what required assessor judgement.
Automated determination of attack success can produce unreliable outcomes.
A manual review process was adopted to provide:
- Higher assurance confidence
- Improved explainability
- Defensible governance evidence
- Stronger auditability
- Expand from 25 to 50+ assessed AI systems
- Additional ISO/IEC 42001 maturity assessments
- Versioned assurance runs
- Longitudinal governance tracking
- Executive dashboards
- Governance scorecards
- Control effectiveness trends
- OWASP coverage analytics
- Framework traceability analytics
- Assurance reporting
This project demonstrates the integration of:
- AI Governance
- Responsible AI
- AI Risk Management
- AI Assurance
- Regulatory Alignment
- Governance Framework Mapping
- Technical AI Control Validation
into a single governance-oriented analytics platform.
The objective is not simply compliance documentation, but the creation of measurable governance evidence, assurance outcomes, and audit-ready decision-support artefacts that support trustworthy AI deployment.
MIT License