How we audit fintech codebases in the AI era (and what still needs a human)

Audit-readiness is still a developer responsibility

Fintech products carry other people's money and identity. So it was always our job to ship them with a sound security, privacy, and traceability base: an auditor showing up should be boring. That duty does not transfer to the model when the model writes the code. The less of the code we type, the more of the verification we own.

And the gaps that matter are rarely exotic. At Wolfpack Digital, integrating with national financial institutions has often meant working with legacy systems whose APIs were designed for internal use only. Their side can't give you a modern security base, so yours has to carry it: webhook validation through API keys, IP matching, certificate pinning.

Another common trap is mobile security the backend never mirrors: the app has 3DS and biometrics, but the endpoint behind it accepts the data without validating a token, so anyone calling the API directly bypasses every check the app makes. If someone on your team can manually unblock a customer from a dashboard, that action needs a paper trail, for posterity and for proof.

An audit touches far more than this: infrastructure, vendor contracts, incident process. We can't cover it all in one article. So today we focus on the code, and on one question: how do you decompose "are we audit-ready?" into something you can verify, now that LLMs do a lot of the code-writing?

What is a fintech audit loop?

A fintech audit loop is a repeatable verification workflow that breaks an audit into small checks, runs the mechanical ones as static checks and the judgment ones as schema-constrained LLM reviews, and requires every LLM flag to be reproduced before it counts as a finding. It is a loop because results feed back: recurring flags become new static checks for the next run.

The design principle behind it: no repro, no finding. An LLM step never emits a pass or a fail. It flags a risk with a location, and the flag becomes a finding only when something deterministic reproduces it: a failing test, a command anyone can rerun, or a static check that already covers it. Auditors, regulators, and due-diligence teams accept proof, not model output. And so should you. This is also where the loop differs from most "AI code review" tools, which hand you verdicts you can't rerun or are very hard to follow.

The loop, and the rule that shapes it

Everything in the loop follows one decomposition rule, applied to every audit question:

"Is any money column a float?" is grep with taste, so it stays deterministic. "What happens if this exact webhook payload arrives twice, concurrently?" needs reading three files and reasoning about a race, so it goes to an LLM, phrased as one narrow question rather than "review my code". Every fuzzy step gets deterministic validation of its output, so a hallucinated file path dies at the door.

There are many ways to build such a loop, and that's the beauty of it. For our use case right now, we want to build a workflow that only covers a few cases. I know fintech projects are very different, and some of these features are only the surface, but we need to start from a solid base in order to be able to build a complete audit. Applied to a simple codebase, the rule produces seven phases under one orchestrator:

The orchestrator is boring deterministic code, and that is the point. It owns sequencing, schema validation with retries, running each review question three times with a majority vote, budgets, and an append-only findings log: the audit's own audit trail. The LLM appears in exactly two places. Discovery agents each answer one inventory question (where is money represented, what are the write paths, where are the external boundaries, where does PII live), and every returned location is mechanically checked to exist. Review agents each answer one risk question over one slice of that inventory. Everything else, from the C1-C15 static check pack to reproduction and reporting, is plain code you could run in CI.

So what does a finding look like? Here is the double-credit case. The review step emits a flag under the finding schema:

That flag alone proves nothing. So the reproduce step turns it into a test (in Rails this would be a short RSpec case, in Node a Jest one, etc.):

The test fails, so the flag is proven, and status moves to proven with the test attached. What can't be reproduced goes to a needs_human pile that is counted and shows up in the report.

One more thing closes the loop. The second time a flag class shows up, it should have been a static check, so the codify step drafts one (a semgrep rule, an AST query, a SQL assertion), a human reviews it, and it joins the pack. This way, each run needs the LLM for less.

What should we build this as? The options we weighed

There is no single obvious shape for a workflow like this, and we went back and forth on it. The same loop could live in your editor, in a script, in your pipeline, or behind a dashboard. Each of those is a real option with real trade-offs, so it is worth walking through them the way we did, rather than jumping to a conclusion.

Option 1 - a Claude Code subagent pack. The audit lives where developers already work: a set of subagents in the repo, one per review question, invoked from the editor. This is the cheapest option to build by far. There is no infrastructure, prompt iteration is immediate, and a developer can point R1 at a webhook handler mid-task and get a useful answer in a minute. But the model owns the control flow, and that is the problem. Two runs explore the codebase differently and surface different flags, there is no append-only log unless you bolt one on, and a chat transcript is not something you can hand to an auditor as evidence. In our experience, this is a great prototyping bench and a poor system of record.

Option 2 - a plain script. A small CLI, written in whatever language your team already ships, that runs the phases in order and calls the LLM through the API the way it would call any function: schema in, JSON out, validated, retried on invalid output. The pros are exactly the properties an audit needs. Control flow is code, reruns are comparable, budgets and logs come naturally, and when an auditor asks "how did you produce this report?" the answer is "we ran this command, here is the log". The cons are real too. You build the plumbing yourself (validation, retries, the majority vote), prompt iteration is slower than in an editor, and someone has to remember to run it.

Option 3 - wired into CI/CD. Run the audit as a pipeline stage and every push gets checked, which sounds like the endgame. For the static checks it is: they are fast, free, and binary, so there is no reason they shouldn't gate a PR like any linter. The full loop is a different story. Discovery and review need whole-repo context, not a diff, and they cost real tokens, so running them per push is slow, expensive, and noisy. There is also the risk that finding reports on every PR trains developers to scroll past it within a week.

Option 4 - a hosted service. Scheduled runs, history across projects, a dashboard, findings diffed release over release. For an agency this is attractive, and honestly it is where a workflow like this wants to end up. But it is the heaviest build, and it front-loads the hardest problem: fintech clients handing repo access to an audit service is a security conversation in itself, and you inherit compliance obligations of your own the moment you store their findings. Building the platform before the loop has proven itself on real codebases is effort in the wrong order.

So we landed on a hybrid, and the split follows the decomposition rule from earlier. Subagents are where prompts get born, because iteration speed wins during design. The script is where the loop lives, because control flow must be code. CI is where the static check pack runs, because rules are cheap and cadence is free. And the hosted service stays parked until the loop has earned it. Your constraints may weigh these differently, and that is fine: the comparison is the part worth stealing, not our answer.

The script at the center of it fits in about twenty lines:

Every line that touches the model is wrapped the same way: schema validation with retry, a token budget, and a hard rule that nothing unvalidated flows downstream.

Keeping the loop honest

A loop like this degrades in two directions, and we guard against both explicitly.

Deterministic steps drift toward fuzziness. The temptation shows up the day a static check gets hard to write: the AST query is fiddly, so someone suggests just asking the model. Don't. The moment a check calls an LLM, it stops being evidence you can hand to an auditor and becomes an opinion with a config file. Our test is blunt: run the check pack twice on the same commit, and the output must be byte-identical. So tool versions are pinned, checks take no network calls, and the check pack is treated like production code: each check has fixtures of known-bad snippets it must catch and known-good ones it must pass.

LLM steps drift toward slop. Structure here is enforced, never requested. Every review step writes into the finding schema, and output that fails validation is retried, not parsed leniently: a finding without a resolvable file and line does not exist. Prompts are versioned artifacts in the repo, and they get regression tests too. The setup for that is a small seeded codebase with known planted bugs: if a prompt change makes R1 miss the double-credit it used to catch, that change does not merge. On top sit the runtime guards: three runs with a majority vote, and a watched needs_human rate. When that rate climbs, the fix is almost always the same: the question got too broad, so split it.

What still needs a human (and the mistakes teams make)

The loop verifies what exists. It does not judge what should exist. Whether your ledger model fits your settlement flow, what your reconciliation design leaves unreconciled, whether your risk scoring would hold up in front of a regulator, which early architecture choice breaks first at 10x: those questions stay with a senior reviewer, and no amount of codifying absorbs them. In our experience, even the paper-trail question is judgment first: a tool can verify that your support dashboard logs an actor, but only a person can decide whether that log would answer what an investigator actually asks.

Still, the common traps are not subtle. A few patterns to avoid:

- Treating LLM output as a verdict - shipping raw model opinion as an "AI audit". That is the slop pattern, and it is the one this loop exists to prevent.

- Refusing LLMs entirely - grep does not find race conditions, and pretending it does is the same abdication in the other direction.

- Trusting green CI - tests assert intended behavior; audits hunt unintended behavior. These are different jobs.

- Auditing once - scale moves the goalposts. The loop is a cadence, not an event. And compliance platforms like Vanta or Drata cover infrastructure and paperwork; your code is still yours.

When to run this

Run the full loop on a codebase takeover, before PSP onboarding, before due diligence, or after an incident. Run the static checks in CI from day one: they are just rules, and they are cheap. A classic manual audit is enough when you are pre-MVP, no real money moves, and the codebase is readable in a day.

For priorities: money-path checks first, because errors there compound silently. Audit trail second, because you cannot reconstruct it after the fact. Reconciliation third, because it is the net that catches whatever the first two miss.

Your loop will look different from ours, and that is fine. The checks are stack-specific and swappable; the decomposition rule is the part to keep. We're building this as a base workflow we tailor per project, and a follow-up article will show a full run on a real codebase, findings included. I hope this article helps you make your product audit-ready before someone else checks it for you.

Do you have a fintech product that needs this kind of look under the hood? Get in touch with us, and let's talk more!