SecureWorld News

Agent Observability Shouldn't Just Be About Vulnerabilities

Written by Rick Doten | Tue | May 12, 2026 | 11:49 AM Z

Business leadership and boards are pushing for the use of AI and expecting to see ROI. CISOs are looking at agent monitoring tools that tell them about vulnerabilities, not business value.

I've talked to well more than a dozen startups over the last year who are focusing on AI agent security—from discovering what risks they have, evaluating attack surface, determining the accounts agents use and data they access, and even observing chain of thought to determine drift. While these are important for risk management, briefing this information to the leadership or board is not helping the AI rollout goals.

What leadership wants to know is how are agents being used? What business process are they supporting? Are we seeing productivity gains; do we see ROI toward our business goals? What are successful uses that we can operationalize and scale? They need this information to help them make decisions, not just give them metrics on use or vulnerabilities.

Most organizations are looking for fast AI adoption, but don't have strategy or governance, so they can't evaluate what's working. They are accepting the AI risks in the meantime. They expect the CISO to manage these risks as they evolve, without hampering experimentation or slowing the rollout.

And this isn't trivial. Enumerating what agents are running in an organization is not deterministic like the way we normally look for file names, app versions, or hashes. It is done through observing API traffic or API logs, triggers, code repos, account use, tool access, and data access. Developers are spinning up MCP servers, business units are connecting SaaS-embedded agents, and end-users are running desktop assistants. And these are the legitimate cases where the users are using sanctioned tools. Not to mention agents interacting with platforms not purchased or licensed by IT.

Building a real inventory of AI agents across the enterprise requires methodology, tooling, and persistent effort. The security teams are building something that didn't exist before: a live map of where autonomous processes are running inside the business.

That map is exactly what leadership needs—and the security team is the only function positioned to provide it. If we're already doing the hard work of finding agents, characterizing their behavior, and understanding what data and accounts they touch, we can also describe their business function and observed outcomes. This requires CISOs to work with business stakeholders to categorize what they're seeing to gain context for the agent's purpose and behavior. CISOs can choose to stop at "here are the risks" or go further and say, "here is what the business is actually doing with AI, and here is where it appears to be working."

Image prompted by Rick Doten and generated by Gemini

My suggestion is not to just brief the leadership and board how many agents are running and what accounts they use or the types of data they touch. Use that telemetry to show what categories of projects or tasks are being automated. Are most of the agents used by end-users on their desktop for personal assistance by filtering email, managing calendars, summarizing news, or creating presentations? Are some used by business analytics to collect data, analyze, and generate reports? Others used by IT to automate tasks, clean up logs, create tickets, collect data for reports? Agents used by security might be for log analysis, evidence collection, data correlation, or vulnerability prioritization. How many are used by software development for code generation, code review, and testing? And what others used by Marketing, HR, Finance, Sales, and Service desk exist?

That is what leadership wants to know: what is being automated, and what is working. This is important since many organizations don't have a clear AI strategy and plan. They need to know what different departments are doing with AI, and what positive outcomes they are seeing.

By having this information, leaders can ask questions about effectiveness, cost, and reliability of these outcomes. And they can make decisions on which area to put more investment in. Or to adjust their goals.

Each leader will have different goals. The CISO might choose to increase the focus used for report generation, that helps with compliance. Or use in security operations for evidence collection, and incident isolation, or want to increase use in vulnerability remediation to close more tickets faster. The CIO might want to grow use in development.

When business leaders see where agents are deployed and what outcomes they're producing, they can make real resource decisions. Here are examples of what is possible when security-sourced observability gets paired with business context:

  • Sales gets better analysis of pipeline prioritization and improved close rates through follow-up consistency.

  • Finance teams are using agents to run "what-if" simulations for capital allocation, reducing capital expenditure and improving revenue forecasting.

  • Manufacturing and supply chain agents can analyze demand signals and logistics constraints to reroute orders to prevent stockouts and reduced inventory carrying costs.

  • Customer service agents now have permissions to access databases, process refunds, and update accounts end-to-end. They can resolve issues instead of just recording them to solve customer problems faster, increasing first contact resolution rates.

  • Healthcare organizations are using agents trained on denial patterns to identify coding mismatches and documentation gaps before submission to reduce denials.

This is an exciting transformation of the business; CISOs are uniquely positioned with visibility they are getting in tracking the agent assets in the business to make sure they are secure, but also to help discover where AI is actually creating value. Being seen as a driver of business success is what we've been working toward for years, and we can choose to just report on risks that agents bring, or give leadership key insights to show their business value—and the insight to appropriately secure those workloads.