AI Audit

AI is already making decisions inside your business.

But what proves the control worked?

Scope an audit How an audit runs

A request arrivesRecorded
AI recommendsRecorded
A person reviewsNo record
The decision is madeRecorded

Why

AI can keep working while assurance breaks.

Did a person really review it?
Is this still the version that was approved?
Where did the information come from?
What changed at the vendor?
Could you explain the decision afterwards?

That is why you audit AI.

What we audit

Audit of AI is bigger than the model.

Does it perform as specified?

An AI system that produces inaccurate outputs with confidence is more dangerous than one that produces no outputs at all — because the inaccuracy is invisible.

What we evaluate

Whether a deployed AI system performs as specified:

Whether its outputs are accurate
Whether performance is consistent across the full range of inputs it processes
Whether it degrades over time as data and conditions change

How

Technical evaluation of model performance metrics
Review of the system's monitoring infrastructure
Testing against edge cases and adversarial inputs
Assessment of the human oversight mechanisms designed to catch failures when they occur

What you get

The audit report documents findings against defined performance criteria and identifies the specific interventions required to bring the system to an auditable standard.

Are its decisions fair, explainable and defensible?

The decisions AI algorithms make at scale — on credit, on hiring, on pricing — can discriminate systematically along lines the organization never intended and may not be able to see without independent evaluation.

What we examine

The logic of AI decision-making, for evidence of bias across protected and proxy characteristics
Whether the AI's decisions can be explained in terms that affected parties can understand and challenge
Whether the level of human oversight applied to AI decisions is proportionate to their consequences

What you get

The audit uses established fairness metrics, interpretability tools, and documentation review to produce findings that identify both the presence of bias and its source — enabling remediation at the right level, whether that is the training data, the model architecture, or the operational context.

Can we trust what feeds it?

Every AI system is as reliable as the data it was trained on and continues to receive. Data that is incomplete, outdated, biased in its collection, or corrupted in its processing produces AI outputs that carry those flaws silently and consistently.

What we examine

The data assets that AI systems depend on, across four dimensions:

Quality
Integrity
Completeness
Lineage

What you get

A finding report with:

A data quality score
Identified integrity risks
Gaps in data coverage
A lineage map that documents where the data came from and how it was processed before reaching the AI system

Does it meet the obligations that apply to it?

AI systems operating in the UAE are subject to a growing body of compliance obligations:

The Dubai AI Seal standards
The UAE Personal Data Protection Law
DIFC Data Protection regulations
Sector-specific requirements in financial services and healthcare
The principles of the UAE National AI Strategy 2031

For organizations with global operations, the EU AI Act adds further complexity.

What you get

Our AI compliance audit maps each AI system against the applicable compliance obligations and produces a compliance gap analysis with documented evidence of conformance and non-conformance, structured to satisfy both internal governance requirements and the documentation standards required for regulatory submissions or certification applications.

Can it withstand AI-specific attack?

AI systems introduce security vulnerabilities that traditional cybersecurity frameworks were not designed to detect.

The threats

Adversarial attacks — inputs engineered to manipulate AI outputs — can be invisible to standard monitoring tools.
Data poisoning during training can compromise an AI system long before it is deployed.
Model theft exposes proprietary intellectual property through the outputs of a deployed system.

What we assess

The attack surface exposed by each system
Controls in place to detect and prevent adversarial inputs
The security of the training pipeline
Access controls around model weights and training data
Monitoring infrastructure for anomalous AI behavior in production

What risk are we inheriting?

Most organizations use more AI than they have built. The AI embedded in their CRM, HR platform, financial reporting system, customer service tools, and productivity suite represents a portfolio of third-party AI dependencies accepted without independent evaluation.

When those tools produce decisions that affect customers, employees, or financial outcomes, accountability sits with the organization that deployed them — not the vendor that built them.

What we evaluate

Accuracy and reliability
Bias and fairness
Data handling and privacy
Security posture
Vendor governance
Contractual provisions for accountability

What you get

An evaluation report for each tool reviewed
An overall risk rating for the third-party AI portfolio
Specific guidance on tools requiring remediation or replacement

How it runs

How does an AI audit actually happen?

First, why does this need auditing at all?

Client enquiryQualificationProposal & scopeKickoff

Audit opened

Archive shelving holding rows of labelled document boxes.

What you get

The evidence becomes a decision.

One report: what your AI can be relied on for, what it cannot, and what has to change.

Executive conclusionScope & methodologyMaterial findingsRemediation & re-test

Then we re-test. Change can invalidate yesterday's assurance.

Nakoda

You bring the AI. We bring the audit.

Nakoda AI's Audit division covers both tracks with equal depth. Our practice is grounded in ISACA frameworks — COBIT, CRISC, and CISA — and brings the professional standards of chartered accountancy to every engagement.

Audit findings and reports are produced to professional standards that withstand board-level scrutiny and, where relevant, regulatory examination.

For organizations building internal AI audit capability, we build the function from scratch — methodology, tooling, training, and the governance structures that give internal audit the independence it requires.

For organizations that want a permanent internal AI audit capability rather than relying on external engagements, Nakoda AI designs and establishes the function from scratch.

What it covers

The methodology the internal team will apply
The tooling and data access required
The training required to build the team's AI audit competence
The governance structure that gives the audit function appropriate independence
The reporting lines that connect AI audit findings to the board and senior leadership

What you get

An operational function — including the first cohort of internal AI audits completed under Nakoda AI's supervision to validate that the methodology is being applied correctly before the function operates independently.

ISACA's frameworks provide the globally recognized methodology basis for AI audit engagements that need to meet professional and regulatory standards.

COBIT for governance and management of enterprise IT
CRISC for risk and information systems control
The CISA standard for information systems auditing

When we align to them

Organizations preparing for external certification
Responding to regulatory requests
Building internal audit functions that need to demonstrate professional standards

This alignment is not a labeling exercise — it means the audit methodology, evidence standards, documentation requirements, and reporting format follow the ISACA standards that inform globally recognized professional practice.

Questions

Audit of AI means independently evaluating AI systems, algorithms and data to determine whether they are trustworthy, accurate, fair, compliant and secure - what an organization does to the AI it uses. AI in auditing means deploying AI tools within the audit function to make audit processes more effective and comprehensive - what the audit function does with AI.

Both are needed, and they are not interchangeable.

Where does the greatest risk sit?

Start an audit conversation

The right entry point for most organizations is a scoping conversation that separates which AI systems most need independent audit attention from those that can be addressed through internal review.

For organizations uncertain about which track is most relevant — audit OF their AI systems or AI IN their audit function — the answer is typically both, and the sequencing depends on where the greatest risk currently sits.

Nakoda AI's audit team can walk through this assessment in a single working session.

A shelf of old bound files standing side by side, each spine carrying a handwritten paper label.

Two disciplines. Both critical. Neither optional.

Can we trust how AI is being used in the audit process?

Ask about AI in auditing

There are two fundamentally different audit disciplines in the AI era, and organizations need both.

The audit of AI — examining AI systems, algorithms, and data from the outside to determine whether they can be trusted, whether they comply with applicable standards, and whether the decisions they produce are accurate, fair, and defensible.
The use of AI in auditing — deploying AI tools within the audit function to make audit processes faster, more comprehensive, and more effective at identifying what matters.

These disciplines are not interchangeable and they are not in competition. They are complementary.

A finance team that uses AI to accelerate its audit sampling still needs an independent auditor to evaluate whether the AI it used can be trusted.
An internal audit function that deploys AI for continuous monitoring still needs a framework for auditing the AI tools it has introduced.

The relationship between the two disciplines is circular, and the expertise required for each is distinct.

AI IN Auditing means deploying AI tools within the audit function itself to make audit processes more effective, more comprehensive, and more efficient.

This is what the audit function does with AI — not what is done to AI.

Traditional audit sampling operates on statistical principles designed for an era when complete population testing was impractical.

AI makes complete population testing feasible in many contexts — and where it is not, it enables risk-stratified sampling that concentrates audit coverage on the transactions, accounts, and processes that statistical approaches would identify only by chance.

What we do

Nakoda AI works with internal and external audit teams to introduce AI-enhanced sampling methodologies:

Configuring the tools
Validating their outputs against manual benchmarks
Training the audit team to interpret AI-surfaced results appropriately
Building the documentation trail that ensures AI-assisted sampling meets professional audit standards

The result

Audit coverage that is simultaneously broader and more precisely targeted than traditional approaches — without increasing the time or resource required to achieve it.

Point-in-time audits find what was wrong at the moment they looked. Continuous monitoring finds what is wrong as it happens.

AI makes continuous monitoring of financial transactions, access logs, operational data, and compliance indicators operationally practical for organizations that previously relied on periodic review.

What we design

Continuous monitoring frameworks using AI anomaly detection:

The data sources to be monitored
The alert thresholds that distinguish genuine anomalies from operational noise
The workflow for investigating and escalating alerts
The reporting structure that keeps audit leadership informed without creating alert fatigue

Document review is one of the highest-volume and lowest-leverage activities in audit work.

Reviewing contracts for clause variations, extracting data from invoices and purchase orders, searching correspondence for evidence of material communications — these tasks consume audit hours that could be spent on the analysis and judgment that AI cannot replace.

What we do

Nakoda AI implements AI document analysis tools in audit workflows:

Selecting the right tools for the document types and review tasks involved
Configuring them for the organization's document formats and audit objectives
Validating their extraction accuracy against manual benchmarks
Training the audit team to use the output as evidence rather than as a substitute for professional judgment

Internal audit planning is only as good as the risk assessment that drives it. When risk assessments rely primarily on management interviews and last year's audit plan, they miss the emerging risks most important to identify.

What AI adds

It processes a broader range of signals to surface risk patterns that structured interviews would not reveal:

Transaction data
Control test results
External risk indicators
Operational metrics
Exception reports

What you get

Nakoda AI works with internal audit functions to integrate AI-enhanced risk assessment into their annual planning cycle, producing an AI-informed audit universe and risk ranking that the team can defend to the audit committee.

The intersection of AI and financial auditing requires specialized knowledge across both domains.

Financial AI systems — those used in credit decisioning, fraud detection, financial forecasting, expense analysis, and regulatory reporting — introduce audit complexities that general-purpose AI audit methodologies do not fully address.

What we apply

The principles of financial audit — materiality, evidence, professional skepticism — to AI systems operating in financial contexts
AI audit methodology — model evaluation, bias assessment, data lineage — to the specific characteristics of financial data and financial decisions

Audit of AI

What is an audit of AI?

An independent examination of an AI system and everything around it — the people accountable for it, the data feeding it, the instructions governing it, the tools it can reach, the vendors behind it and the context it operates in — to establish, on evidence, whether it can be relied on for its intended use.

Why does AI need independent audit?

Because an AI system can keep operating normally while the assurance around it breaks. A model version changes, a data source is reshaped, a vendor updates something invisible, a human review step stops being recorded — the outputs still look right, and nothing fails loudly. Audit exists to establish whether the control still works, on evidence rather than on the policy that describes it.

What are the different types of AI audit?

Six disciplines, each answering a different question about the same system. An AI System Audit asks whether it performs as specified. An Algorithm Audit asks whether its decisions are fair, explainable and defensible. A Data Audit asks whether the data feeding it is reliable, complete and traceable. An AI Compliance Audit asks whether the system meets the obligations that apply to it. An AI Security Audit asks whether it withstands AI-specific threats such as adversarial input, data poisoning and model theft. A Third-Party AI Tool Audit asks what risk the organisation is inheriting from AI it did not build.

How does a Nakoda AI audit engagement work?

Twenty-four steps in six phases. Enter and define covers client enquiry, qualification, proposal and scope, and kickoff. Understand the environment covers discovery, process mapping, current AI assessment, AI maturity assessment and data and technology assessment. Find what matters covers AI ideation, the use-case catalogue, value and feasibility scoring, risk assessment, the business case and prioritization. Turn findings into a plan covers the AI roadmap, pilot planning, execution and validation, and KPI measurement. Assure and decide covers the final audit report, the executive presentation and decision or sign-off. Continue covers the 30, 60 and 90-day follow-up and implementation or scale.

What does Nakoda do at each stage of an AI audit?

In steps 01 to 04 we qualify the need, define the scope, establish objectives and set the engagement. In 05 to 09 we discover the environment, map the process, assess current AI, evaluate maturity and examine data and technology. In 10 to 15 we identify relevant use cases, catalogue them, score value and feasibility, assess risk, build the business case and prioritize the portfolio. In 16 to 19 we build the roadmap, plan pilots, support validation and define measurement. In 20 to 22 we produce the final audit report, present material findings and support executive decision-making. In 23 to 24 we follow up, re-test where required, and support implementation and scale.

What does an AI audit produce?

An AI System Assurance Report: an executive conclusion on whether the system can be relied on for its intended use, the scope and methodology, the material findings with the evidence behind each, and a remediation plan with owners, target dates and a scheduled re-test. The sign-off records one of five outcomes — accept, remediate, restrict, do not use, or re-test.

Can Nakoda build an internal AI audit function?

Yes. Internal AI Audit Function Design establishes permanent capability rather than repeat external engagements: the methodology adapted to AI evidence, the tooling and data access, training to the level required to audit rather than build, the governance structure that gives the function independence, the reporting line from a finding to the board, and a first cohort of audits run under supervision to validate the method before the team operates on its own.

Which frameworks does the audit practice follow?

ISACA's frameworks — COBIT for governance and management of enterprise IT, CRISC for risk and information systems control, and the CISA standard for information systems auditing — alongside the professional standards of chartered accountancy. Alignment means the methodology, evidence standards, documentation requirements and reporting format follow those standards.

What is the difference between audit of AI and AI in auditing?

Audit of AI examines the AI systems an organisation relies on — the system, its data, controls, people, vendors, risk and accountability — to answer whether it can be trusted for its intended use. AI in auditing examines how an audit function uses AI in its own work: enhanced sampling and coverage, continuous monitoring and anomaly detection, document analysis, AI-enhanced risk assessment for audit planning, and finance audit specialisation — and how those outputs are validated, how evidence is handled and how audit judgement is protected. They are different disciplines and they work better together.

When does an AI system need to be re-audited?

When something material changes. A new model version, a changed data source, a vendor altering its service, a change in volume, scope or regulation. Assurance is a statement about a system at a point in time, so a material change is the trigger to establish whether the earlier conclusion still holds. The engagement schedules 30, 60 and 90-day follow-up for exactly this reason.

What an audit of AI examines

Seven areas surround the AI system, and each carries one question the audit has to answer on evidence.

People — Who is accountable?: Decision rights: Who is allowed to decide, and who answers for it. Named owner, delegated authority, escalation path, the reviewer who sees it first.
Data — Can we trust the information?: Data lineage: Where did the information come from? Source systems, transformations, retention, the point where provenance stops.
AI system — Does it behave as intended?: Control effectiveness: Did the control actually work? Version in production, performance across the input range, monitoring, degradation.
Instructions — What are we asking it to do?: Control boundary: What the system has been told it may and may not do. System instructions, guardrails, refusal conditions, who may change them.
Tools — What can it access?: Authority: What the system can reach, and what it can do there. Connected systems, write permissions, transaction limits, the stop.
Vendors — What risk are we inheriting?: Third-party risk: What we accepted when we bought it. Model provider, contractual accountability, change notice, what we can see.
Context — When does yesterday's assurance stop applying?: Change management: What changed since we last looked. Model versions, data sources, volumes, regulation, the trigger to reassess.

The six audit disciplines

AI System Audit — Does it perform as specified?: Accuracy across the full input range, consistency, degradation as conditions change, monitoring, edge cases and the human oversight designed to catch failures. An inaccurate system that is confident is more dangerous than one that fails loudly. Control effectiveness: Did the control actually work?
Algorithm Audit — Are its decisions fair, explainable and defensible?: Decision logic tested for bias across protected and proxy characteristics, whether a decision can be explained to the person it affects, and whether oversight is proportionate to consequence. Bias scales before anyone notices it. Disparate impact: Does the outcome differ by group without a reason that holds?
Data Audit — Can we trust what feeds it?: Quality, integrity, completeness and lineage across every source the system depends on — and the point at which provenance stops being recoverable. Bad data does not always look bad. Data lineage: Can we trace where the information came from?
AI Compliance Audit — Does it meet the obligations that apply to it?: The system mapped against its applicable obligations — Dubai AI Seal, UAE PDPL, DIFC data protection, sector rules — with documented evidence of conformance and of every gap. An obligation you have not mapped is one you cannot evidence. Conformance: Which rules apply here, and can we show we meet them?
AI Security Audit — Can it withstand AI-specific attack?: Attack surface, detection of adversarial input, training pipeline security, access control around model weights and training data, and monitoring for anomalous production behaviour. Traditional security controls do not see every AI attack. Adversarial input: Input crafted to make the system answer wrongly.
Third-Party AI Tool Audit — What risk are we inheriting?: Externally sourced AI evaluated on accuracy, bias, data handling, security posture, vendor governance and contractual accountability, rated as a portfolio. Most organisations use more AI than they built. Third-party risk: What we accepted when we bought it.

The engagement — 24 steps in six phases

01 Enter & define — First, why does this need auditing at all?

We establish what prompted the question and what should actually be examined — before anything is looked at. Audit scope: What is being examined, and what for. Result: Audit opened.

01 Client enquiry — Something prompted the question. We start there.
02 Qualification — Is an independent audit the right instrument for this?
03 Proposal & scope — What will be examined, against what, and what is excluded.
04 Kickoff — Owners named, access agreed, the file is opened.

02 Understand the environment — First, we understand what we're actually auditing.

AI rarely exists in isolation. We map the business, the processes, the systems and the environment around it. AI inventory: Everything in scope, and what each part decides. Result: Baseline established.

05 Discovery — What the organisation actually does, before any AI is discussed.
06 Process mapping — The workflow the AI sits inside, end to end.
07 Current AI assessment — Every AI system in scope, and what each one decides.
08 AI maturity assessment — How well the organisation governs what it already runs.
09 Data + technology assessment — The infrastructure underneath: sources, pipelines, platforms, controls.

03 Find what matters — Now we determine what deserves attention.

An audit that reports two hundred things reports nothing. The question is which findings, risks and opportunities actually matter. Materiality: Would this change what leadership does? Result: Priorities established.

10 AI ideation — Where AI could matter, and where it demonstrably could not.
11 Use-case catalogue — Every candidate written down before any is judged.
12 Value + feasibility scoring — What it is worth, and whether it can actually be done here.
13 Risk assessment — What could go wrong, how badly, and who carries it.
14 Business case — The cost, the return, and the assumptions both rest on.
15 Prioritization — The short list leadership can actually act on.

04 Turn findings into a plan — A finding is useful only when it changes what happens next.

The priorities move into lanes with owners and dates, and the first of them is proven on a pilot before anything scales. Remediation: What has to change before the control can be relied on. Result: Plan validated.

16 AI roadmap — Sequenced by dependency and risk, not by enthusiasm.
17 Pilot planning — The smallest test that would change our mind.
18 Execution / validation — The pilot runs and produces evidence, not opinion.
19 KPI measurement — The measure agreed before the pilot, applied after it.

05 Assure & decide — The evidence becomes a decision.

Everything gathered assembles into one report, the material part of it is put to leadership, and leadership decides. Executive sign-off: Someone with the authority accepts the conclusion. Result: Decision recorded.

20 Final audit report — Evidence, findings, conclusion and remediation in one file.
21 Executive presentation — Only what is material, to the people who decide.
22 Decision / sign-off — Accept, remediate, restrict, do not use, or re-test.

06 Continue — The audit doesn't end when the report is signed.

The system is re-examined at 30, 60 and 90 days, and what holds becomes part of how the organisation operates. Continuous assurance: Does the assurance still hold after the system changes? Result: Continuous assurance.

23 30/60/90-day follow-up — What changed, and does the assurance still hold?
24 Implementation / scale — What worked becomes part of the operating model.

What Nakoda does at each stage

Steps 01–04 — Frame the engagement: We establish whether an independent audit is the right instrument, and what it has to cover. Qualify the need. Define the scope. Establish the objectives. Set the engagement.
Steps 05–09 — Establish the baseline: We map what exists before we judge any of it. Discover the environment. Map the process. Assess current AI. Evaluate maturity. Examine data and technology.
Steps 10–15 — Identify and prioritize: We separate what is material from what is merely true. Identify relevant use cases. Catalogue them. Score value and feasibility. Assess risk. Build the business case. Prioritize the portfolio.
Steps 16–19 — Turn findings into action: We give every finding an owner, a date and a measure. Build the roadmap. Plan the pilots. Support validation. Define measurement.
Steps 20–22 — Assure and report: We put the material findings, and only those, in front of the people who decide. Produce the final audit report. Present material findings. Support executive decision-making.
Steps 23–24 — Continue: We come back, because the system will not have stood still. Follow up. Re-test where required. Support implementation. Support scale.

Testing a control

Claim: Human review is required before a payment decision is actioned. Evidence: of 1,250 decisions, 1,201 carried a documented review and 49 had no documented review. Conclusion: control effectiveness partial, raised as F-03 — Human oversight exists but is not consistently evidenced.

F-01 (Context) — Model version changed without reassessment. Classified: Restrict. Material.
F-02 (Data) — Supplier master feed lineage not captured end to end. Classified: Remediate. Material.
F-03 (People) — Human oversight exists but is not consistently evidenced. Classified: Remediate. Material.
F-04 (Vendors) — No contractual notice of provider model change. Classified: Escalate.
F-05 (AI system) — Decision path not reconstructable beyond 30 days. Classified: Remediate.
F-06 (Instructions) — Instruction changes recorded but not reviewed. Classified: Accept.

Remediation

R-01 (0–30 days) — Enable review logging on the payment decision workflow. Owner: Finance Systems.
R-02 (0–30 days) — Hold the model version pending reassessment. Owner: AI Platform.
R-03 (30–90 days) — Restore lineage capture on the supplier master feed. Owner: Data Engineering.
R-04 (30–90 days) — Define the reassessment trigger for model and vendor change. Owner: Risk.
R-05 (30–90 days) — Re-test the oversight control on a full quarter. Owner: Internal Audit.
R-06 (3–6 months) — Change-notice clause in the AI provider contract. Owner: Procurement.
R-07 (3–6 months) — Decision reconstruction for the payment workflow. Owner: AI Platform.
R-08 (6–12 months) — Instruction change control in the release process. Owner: AI Platform.

AI System Assurance Report

Conclusion: Conditional. Reliable for the intended use, once F-01 to F-03 are addressed.

03 Material findings
4 / 7 Domains effective
124 Evidence items reviewed
08 Recommendations
90 days Re-test

Executive conclusion — What the AI can be relied on for, and what it cannot.
Scope & methodology — What was examined, how, and against what.
Material findings — Evidence, risk, recommendation, re-test — per finding.
Remediation & re-test — Owners, target dates, and what gets tested again.

Internal AI Audit Function Design

For organisations that want permanent capability rather than repeat engagements.

Method — The audit methodology, adapted to AI evidence.
Tools — Tooling and the data access an audit actually needs.
People — Training to the level required to audit, not to build.
Governance — The structure that gives the function independence.
Reporting — The line from a finding to the board.
First audits — A first cohort run under supervision, to validate the method.

Professional basis: COBIT, CRISC, CISA. Methodology, evidence standards, documentation and reporting follow ISACA frameworks and the professional standards of chartered accountancy.

AI in auditing

The same audit desk, doing more of the work that used to be sampled.

Enhanced sampling and coverage: Population coverage: How much of the ledger was actually looked at. From statistical sample to full population, or risk-stratified where it cannot be.
Continuous monitoring: Anomaly detection: Finding it as it happens, not at year end. From point in time to continuous, with thresholds tuned against alert fatigue.
Document analysis: Extraction accuracy: Whether the machine read the contract correctly. From manual review queue to extraction validated against a manual benchmark.
Risk assessment for internal audit: Audit universe: What the annual plan decided to look at. From interviews and last year's plan to transaction, control and external signals, ranked.
AI in finance audit: Materiality: The threshold above which an error would change a decision. From two methodologies, separately to financial audit discipline applied to ai systems, and back.

Output is evidence, not a substitute for professional judgement.

03 of 08 · Nakoda AI divisions

AI Audit is one of eight. They work as one.

Compare all eight

UAE AI ecosystem

Organisations this division works with

From the directory

Links for AI Audit

AI strategy & policies6