150,000 Agents and No One Watching

Quick question. How many AI agents are running in your organisation right now?

If the answer is “no idea”, you’re in good company. Most organisations can tell you how many software licences they own. Very few can tell you which agents have access to which data, what they’re allowed to do, or who switched them on.

That’s about to matter a lot more. Let’s look at the size of the problem, what goes wrong when governance lags, and how where your AI lives changes the way you govern it.

The Scale Is Coming Fast

Gartner predicts that by 2028 the average global Fortune 500 company will have more than 150,000 agents in use. Only 13% of organisations believe they have the right governance in place, as Atlassian notes in its piece on built-in AI governance.

NZ organisations are smaller, sure. But the ratio is what counts. Agents are being added to every platform you already pay for, often switched on by default.

And then there’s the AI nobody signed off. Research cited by ClickUp in its context engineering guide puts it at 60% of employees using unapproved public AI tools at work, while 63% of organisations have no AI governance policy at all.

When It Goes Wrong

This isn’t theoretical anymore. Atlassian’s CAFE(S) framework points to a couple of examples worth knowing:

• Air Canada (2024): a tribunal held the airline liable after its chatbot promised a customer a refund the actual policy didn’t allow.
• EchoLeak (2025): a crafted email was used to get Microsoft 365 Copilot to leak data, the first documented case of prompt injection pulling data out of a production system.

Neither needed a sophisticated attacker breaking in. One was bad context. The other was trusted and untrusted information sitting side by side.

Where AI Lives Changes How You Govern It

This is where the governance question meets our panel question.

• AI embedded in a platform can inherit the permissions you’ve already built. If someone can’t see a Jira project, neither can their agent. That’s Atlassian’s core argument for Rovo, and it’s a fair one.
• A single AI layer across tools needs its own permission model, and it has to respect the rules of every system it connects to. Powerful, but more to get right.
• Running AI locally keeps sensitive data on your own hardware. ClickUp’s piece on local AI makes the case for a hybrid approach, local for sensitive work and cloud for the rest. The trade-off is real hardware cost and in-house expertise.

For NZ organisations there’s an extra wrinkle. Where your data is processed, and by whom, ties straight back into your privacy obligations. It’s worth knowing the answer for every AI tool you run, not just the big ones.

What We’ve Seen Work

Every organisation’s risk appetite is different, so take these as patterns rather than a checklist:

• Know what’s running. Keep an inventory of AI tools and agents the same way you do for software licences. Who owns it, what can it access, how is it billed.
• Give agents boundaries. Role-scoped permissions, restricted tool access and audit logs. Most major platforms now offer these, they just aren’t always switched on.
• Treat context like code. Key knowledge sources need an owner and a review cycle, because agents will act on whatever they read.
• Make the safe path the easy one. If governance slows people down, they’ll go around it, and that’s how shadow AI starts.

Where it doesn’t hold up: locking AI down so tightly nobody can use it. You won’t stop people using AI. You’ll just stop being able to see it.

Last Chance to Join Us

Data exposure and governance are a big part of the conversation at our upcoming breakfast panel, Where Should Your AI Actually Live? Tim Warren, Justin Flitter and Barfoot & Thompson CIO Simon Casey will share how they’re thinking about AI risk, cost and control.

If you’re an IT, finance, ops or transformation leader working this out right now, come along. Register here.