Continuous Education

In my last essay I named the attacks the architecture does not fully handle yet; insider abuse inside authorized scope, new injection patterns, slow drift in operator policies. What they have in common however is that the configuration does not solve them, they evolve day in day out hence the architecture has to evolve too.

The agent infrastructure literature treats governance as a configuration problem, a company authors policies, enforces them, audits them, but the issue is that these rules, policies and the feedback loop are essentially static rules against threats that happen to be a moving target.

That in my humble opinion is a classification error, since new social engineering patterns show up weekly, new injection techniques get published in papers, new regulatory requirements land from new jurisdictions, and new attack patterns surface inside the company’s own shop in real incidents that should be the loop that feeds what comes next. A static architecture only stays current with last quarter’s threats while the actual real threats moved on.

The solution for the above stated problem just so happens to be very much the pattern that has been running quietly in enterprise security for over a decade, we just have not connected it to agents yet.

If you work anywhere bigger than about fifty people you have been put through KnowBe4 training or one of its competitors. Every quarter IT pushes a short module on the latest threats, phishing examples that look real, social engineering scenarios, password hygiene, and sometimes a simulated phishing email follows to test whether you actually learned anything.

KnowBe4 has critics, mandatory training has a mixed record at changing behavior, clickthrough on the simulated phishing does not always drop after a cycle, people skim the videos. I have skimmed a few myself.

But look at what it accomplishes whether or not any individual employee learns. It pushes a continuous flow of curated current threat intelligence into the organization, some industry wide, some tailored to the company’s risk profile. It produces evidence that training happened, auditable records for regulators, insurers, incident investigators. It surfaces patterns, which roles are most susceptible, which threats are most common, which interventions work. And when an incident happens, the next training cycle carries the lessons.

The training is one piece of a bigger system, continuous adaptive risk assessment, a loop where the threats, the exposure and the defensive posture all get updated together, over and over.

Now compare how agent infrastructure handles the same problem. Omega has declarative policies, authored once. MCP gateways enforce rules written once and deployed forever. Verifiable runtimes attest to configurations fixed at boot.

None of them have an education layer, and that is the missing primitive, security as a continuous learning loop instead of a configuration problem.

In the takeout architecture I call it the curriculum, a continuously updated bundle. New threat signatures, recent injection patterns, fresh social engineering scripts, new exfiltration techniques. Policy updates, regulatory changes, incident driven rules, threshold adjustments. New context for the agents themselves, recent incidents in the organization, updated risk classifications, executive communications that should make agents extra suspicious. New menu items because new business needs showed up, and deprecations because some capabilities stopped being safe at their current friction level.

The curriculum is layered like KnowBe4 itself. The base layer ships from the platform, generic threat intelligence to every customer. The middle layer comes from industry, financial services consortia, healthcare ISACs, government CERTs, organized into agent relevant feeds. The top layer is Tanaka’s own, from her organization’s incidents and policies.

Every layer is signed and dated and visible, Tanaka sees which bundles are active, which are stale, which are pending review, and she can trace any agent behavior back to the curriculum elements that shaped it.

Delivery uses the layers that already exist, threat signatures update the cashier’s anomaly rules, policy updates refresh the templates in her dropdowns, agent context lands in the operator bundle every agent receives as part of its environment, new menu items show up in the catalog. Nothing architecturally new here, it is continuous flow into existing slots, governed by the same primitives as everything else.

And the curriculum gets verified the same way humans get verified. KnowBe4 sends fake phishing to test you, so the architecture sends simulated attacks against the agents. Red team agents probe Kenji’s steward on a schedule, I am pretending to be Kenji’s boss demanding an urgent wire transfer, does the agent push back? The results feed the training pipeline and Tanaka’s dashboard.

That last part is the genuinely valuable piece, and almost nobody is building it. Today when a regulator asks a bank how do you know your AI agents are safe, the answer is some version of our vendor says so and we have a SOC2 report. With the curriculum the answer becomes, here is our quarterly adversarial test report, here is the share of simulated social engineering our agents resisted, here is the trend, here is how we compare to industry benchmarks, here are the failures we found and the curriculum updates we shipped in response.

A regulator can actually work with that. It is a sequence of receipts.

The risks are real, so let me name them.

The word training is loaded, people hear train the agent and think fine tuning weights, and that is usually not what you want, fine tuning is expensive, risks degrading base capabilities and rolls back badly. The curriculum updates the operating environment, context, retrieval databases, enforcement rules, without touching the model. You have to be precise about this or the whole proposal gets misread.

The curriculum becomes a critical security boundary of its own. Whoever updates the curriculum shapes agent behavior, which makes it powerful and makes it a target, so the supply chain has to be signed end to end, attested at every hop, rate limited against sudden bulk pushes, reviewed before deployment. The architecture trains itself, and the training process is governed by the same architecture, recursively.

KnowBe4’s mixed record will come up, mandatory training barely works on humans so why expect it to work on agents? There are two answers to that. Agents update instantly, recall perfectly and apply what they learned consistently, so the decay curve is different. And even setting effectiveness aside, the auditable record that training happened, was applied uniformly and was tested against simulated threats is itself a regulatory and insurance asset. The training works because of what it teaches, and honestly also because of what it documents.

And continuous updates create their own attack surface, an updateable curriculum is an attackable curriculum, and if the simulations are predictable the agents optimize against the simulation instead of the threat. These are real risks and they take design, diverse probes, randomized scheduling, and keeping the team writing the curriculum separate from the team running the probes.