Most AI governance conversations I've been in start in a boardroom and end in a PDF. Meanwhile, in the same tenant, someone has published an agent to Teams that reads a SharePoint site nobody checked, using a connector nobody approved, owned by an account nobody can name.
That gap matters now. The EU AI Act's transparency rules apply from 2 August 2026, even though the Digital Omnibus pushed the high-risk rules out to December 2027 (Freshfields, 2026). The dates keep moving, but what they ask for doesn't: know what you've deployed, know who owns it, know why it decided what it decided, and be able to show someone.
You don't get that from a framework document. You get it from what your pipeline actually stops people doing.
Governance lives in your environment strategy
Two things shaped how I think about this. The first is delivery: building on Dynamics 365 and Power Platform, on a team of eleven sharing one DEV environment and promoting through four tiers to production. The second is studying for Anthropic's Claude Certified Architect – Foundations exam.
I expected that exam to be about API mechanics. It isn't. The biggest section is agentic architecture and orchestration, and nearly every question in it is really about limits: should this be an agent or would a workflow do the job, which tools should it be able to call, where does one agent's context stop and another's start (Anthropic, 2026).
Those are governance decisions in disguise. They look like architecture decisions, and they get made in a config file long before anyone sits down to talk about AI governance. Delivery taught me the same thing. That's what governance by design actually looks like. Three things have held up in practice.
Environments are the boundary, not the policy
Think of an environment as one of the separate spaces work moves through on its way to going live: development, testing, production, each with its own rules about what's allowed to connect to what. A Data Loss Prevention (DLP) policy set on one of those environments stops an agent reaching a connector. A principles document doesn't. The most useful thing we built wasn't a standard, it was an approval gate in the pipeline that a person signs off before anything reaches production. It's boring, and nobody can work around it. It's the same argument as data security being an architectural decision, not a compliance exercise, and it matters more now that Copilot Studio agents get the same environment, DLP, and Application Lifecycle Management (ALM) controls as apps and flows (Microsoft, 2026a).
Every agent needs a named owner and clear limits
An app does what a user tells it to. An agent decides for itself. The exam material is direct about this in a way governance documents aren't: whether you're designing one agent or a set of them working together, the question is always the same, what is this agent's actual job, and what is it allowed to touch to do it (Anthropic, 2026). That is least privilege by another name.
So the question at design time is never can it do this. It's who answers for it when it gets this wrong, and what is the smallest set of privileges that still lets it work?
We learned this the hard way. A missing write privilege on our SLA records only turned up when it blocked a resolution in testing. Asking the least-privilege question at design time would have caught it first.
You have to build explainability in from the start
To explain in six months why an agent sent a record one way and not the other, you need to capture the reasoning as structured output and save it somewhere you can query, at the moment the decision is made. That's a data model decision: it happens in sprint one or not at all.
Gartner (2025) expects more than 40% of agentic AI projects to be cancelled by the end of 2027, because of rising costs, unclear value and weak risk controls. It's the pattern behind why most AI projects fail. That last one is the quiet one: the agent worked fine and got switched off anyway, because nobody could show what it had been doing.
Source: Gartner, 2025.
Where governance goes wrong
None of this means you should govern less. Governance can fail in both directions: too little and you can't see what's happening, too much and people go around you. Here are three ways I've seen the second happen.
Lock things down too hard and the risk just moves
Every tenant starts with one shared environment that anyone can build in without asking. Restrict that enough and people don't stop building. They build somewhere you can't see, pasting customer data into a personal AI account.
Shadow AI usually isn't a sign that governance didn't go far enough, but that it went too far.
Treating everything the same is its own problem
Every organisation will only put up with so much process. Review a meeting-summary Copilot as strictly as a model that affects a customer's outcome and you've used that patience in the wrong place. Risk tiers only count if they actually change which controls apply.
Don't build your plan around someone else's deadline
The high-risk rules were expected in August 2026 and are now set for December 2027 (Freshfields, 2026). Anyone who built a fixed roadmap around the original date has spent a quarter reworking it. Controls built into your own delivery process don't need redoing when a date moves. That's why you judge a platform on what it lets you enforce, not on its feature list.
Conclusion
The organisations getting this right aren't the ones with the best AI policy. They're the ones where the policy and the pipeline are the same thing: environments that enforce data boundaries, an approval gate a person has to pass, an owner on every agent, and audit built into the data model instead of added later.
That doesn't slow adoption down. It's what lets an AI feature last in production, which is the whole point.
If you can't describe your governance model as something your pipeline enforces on a Tuesday afternoon, it isn't a governance model. It's a document.
If you want a second pair of eyes on your own environment strategy, that's a conversation we have regularly at riivo. Get in touch and we'll walk through what your pipeline is actually enforcing today.
FAQs
Isn't AI governance the compliance team's job, not delivery's?
Compliance decides what's required. Delivery builds it. "Tell the user when AI influenced a decision" isn't just wording on a screen, it means the system has to store that fact somewhere in the first place. If delivery isn't in the room when that requirement is written, it usually can't be built the way it was written.
Where do you start if you have nothing in place?
Start with an inventory. You can't govern agents you can't see. Find out what already exists across your environments, give each one a named owner, then apply DLP and connector policies per environment.
Does the EU AI Act matter outside the EU?
It applies directly if you serve EU users or work with EU firms. More usefully, it's the most detailed template out there, and other regimes are landing on the same principles. Building to a standard you can defend now is cheaper than redoing it later.
How do you govern agents without killing experimentation?
Split the environments. Give the people building these a space where they can try things with non-production data, and make the promotion path the point where governance kicks in. Restricting experimentation doesn't stop it, it just moves it.
What's the most common gap you see?
Ownership. Agents get built quickly, demo well, and reach production with nobody's name on them. When something goes wrong, that's why the answer is usually to switch it off rather than fix it.