Forge and Proof: Putting AI Agents Into Production You Can Defend
A half-day, hands-on workshop for model risk and compliance leaders. ForgeLoop builds governed agents; ProofLoop proves them — both on the Geometric Memory System.
We’re running a private, half-day workshop for model risk and compliance leaders, built around one question: how do you put an AI agent into a regulated workflow and still defend the decision afterward?
This post is the briefing for everyone we’ve invited — what we’ll cover, the one-pager to bring with you and the setup to run before you arrive.
Download the executive one-pager (PDF) — a single page you can print and bring along, or hand to a colleague who couldn’t make it.
Why this workshop exists
AI agents are moving into the workflows your organization actually runs — customer service, claims and disputes, compliance review, high-stakes decisioning. In each one, a wrong answer costs a customer money or puts a decision in front of a regulator. The way most of these agents are built today just doesn’t survive that bar.
The usual pattern is a system prompt to steer the agent, a flat file to remember and a second model to judge the first. It demos well, then fails in production in ways the demo never shows. A prompt isn’t an enforced boundary, so it breaks on the first unfamiliar or adversarial input. Flat memory can’t tell a real fact from a fabricated one, so a hallucination on one run gets cited as truth on the next. And when one model grades another, both have usually drifted the same way — “another model said so” isn’t evidence an auditor or regulator will accept. These failures are quiet when they happen and expensive when you find them.
The shift: replace hope with evidence
Knowlytix puts a thin, deterministic layer beneath the agent — the Geometric Memory System (GMS) — that checks every step and writes down a record: is this in scope, does this contradict what we’ve already accepted, here’s the signed receipt. The model still does the drafting. GMS is the part that makes its actions bounded, checkable and reproducible. Failures turn into refusals and escalations instead of confident fabrications.
The workshop is built around the two tools that sit on top of that substrate:
- ForgeLoop — build it. A framework for assembling governed agents: typed actions, a gate on every tool call, a hash-chained audit trail and escalation to a human only when it’s genuinely warranted. The human becomes a safety net, not a crutch for the unreliable parts.
- ProofLoop — prove it. Automated, end-to-end testing at scale. It generates the ground truth, the test scenarios and the weakness report for you, and it pins every failure on the input or component that caused it — instead of held-out accuracy and one model grading another, you get signed evidence.
What you’ll leave with
- Defensible deployment. Every action is bounded, checked and replayable — evidence you can put in front of model-risk review, internal audit and a regulator.
- Known risk, not hidden risk. Decisions carry a calibrated, measured error rate instead of an unknown one, and numbers are recalled exactly, never re-parsed from prose.
- No silent failures. On an unfamiliar input or a tool fault, the agent fails loud and escalates instead of pressing on with a corrupted result.
- Assurance that keeps pace. Ground truth and tests are generated, not hand-built, so coverage moves at the speed of release instead of lagging behind it.
Before the workshop: install and prerequisites
The session is hands-on, so please set up your environment ahead of time. That way we can start on the material instead of on installs.
Prerequisites
- macOS, Linux or Windows
- Python 3.12 (3.12 specifically — not 3.13 or newer)
- A clean virtual environment (recommended)
- A laptop you can install packages on (admin rights, no blocking proxy)
Install
The session uses two tools — ForgeLoop to build and ProofLoop to prove — on top of the Knowlytix family that provides the Geometric Memory System. Create a virtual environment and install all three:
python -m venv .venv # use Python 3.12 (not 3.13 or newer)
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install knowlytix
pip install forgeloop
pip install proofloopforgeloop— the transparent, auditable agent loop: policy-governed agents with runtime gates, audit and escalation. This is what you’ll build with.proofloop— test, validate and monitor agentic AI: design-of-experiments suites, validation reports and real-time runtime gates. This is what you’ll prove with.knowlytix— a meta-package that pulls in the Geometric Memory System family the two tools run on:
| Package | Purpose |
|---|---|
knowlytix-core | Geometric memory core (geometry + LLM-layer foundations) |
knowlytix-harness | Policy-governance harness |
knowlytix-knowledge | Geometric expert system with LLM-augmented learning |
knowlytix-benchmark | Benchmarks and reference results |
knowlytix-kal | Knowledge Adapter Layer for GMS |
The packages are live on PyPI at a stable 1.0 — installable today, with a settled API. We’ll hand out the workshop dataset and exercise materials on the day. If setup gives you any trouble beforehand, email us at hello@knowlytix.ai and we’ll get you unblocked before the session.
Workshop presentations
Both decks below run right in your browser — use the arrow keys or the on-screen arrows to move through them. They open in a new tab.
- Beyond Prompt and Pray — building governed agents with ForgeLoop: what an agent is, the minimal loop, typed actions and tools, safe execution, planning, memory and runtime governance.
- Beyond Ship and Pray — proving them with ProofLoop: why average accuracy isn’t enough, the system under test, knowledge graphs and the Geometric Memory System, building the oracle and attributing failure.
Stop shipping and praying. Deploy agents you can govern — and prove. See you there.