AI Month: Taming AI Sprawl and Shadow AI Through Strong Governance

The recent ISC2 Spotlight on Artificial Intelligence (AI) saw an industry panel session explore how strong AI governance practices can help organizations regain control before bad actors can weaponize and exploit these problems. Panelists discussed practical approaches for identifying shadow AI, governance and frameworks, reducing uncontrolled AI proliferation, and the challenges around AI visibility and accountability.

The discussion, entitled Taming AI Sprawl and Shadow AI Through Strong Governance, brought together Pamela Gupta, founder & co-president of Trusted AI; Dheeraj Gurugubelli, senior director of cybersecurity and AI for EY-Parthenon; and Michael Powell, director of cyber and infosec at car maker Aston Martin Lagonda.

Gurugubelli started by outlining what is meant by shadow AI and AI sprawl. Shadow IT, he explained, is what we do not see: personal ChatGPT being used by an intern; coding assistance that a developer is using; or unapproved browser extensions. AI sprawl, on the other hand, is: “uncontrolled growth of AI tools or models or copilots or agents,” even if those items are approved. He gave the example of a healthcare company he had been working with, carrying out an inventory of AI: “Turns out that they had 80+ agents just in the sales organization,” he said. “They knew that there was an approved tool, but the number of agents that were that spawn off that is something that they didn't imagine”.

Powell was then asked about the approach he and his firm were taking. “What we're trying to do is provide as much guidance as we can and allow people to essentially self-regulate,” he said, but noted that this is combined with an element of enforcement: “There are a number of tools that we've taken the decision, just for intellectual property or for confidentiality reasons, are out of scope for our organization. So, there's a little bit of blocking and tackling that we've had to do there to essentially prevent those tools from being used”.

Powell also spoke of taking a deliberate approach to the adoption of AI, particularly around not being at the bleeding edge. “We set a policy that says we are a fast follower,” he said, “so we're not using anything brand new, but we're happy to adopt things relatively quickly.” Leaning toward shadow AI, Powell noted: “We've made an open and honest discovery process to try and help people, when they identify something they want to use, push it to our center of excellence,” so the latter can consider if and how it might be adopted formally in the organization.

Governance and Data

The discussion moved on to discussing how data is being sent to AI engines. “We understand what information is being shared,” said Powell. “We're trying to quantify the value that we get back by sharing that information, because as long as the value that's being offered back to the company is more significant than risk, then in theory we should accept that risk”.

The conversation then turned to governance. Gupta was asked what governance model she believes works best. Her first point was that while governance needs to be effective, it must not get in the way of doing business. “How do we implement a strategic governance [approach] that achieves the business outcomes as well as doing what it needs to do and what it doesn't,” she mused. “Governance that [is only] going to say no is going to create shadow governance,” she pointed out, “but if it says yes and says we know governance, there is a gate”. She also noted that governance has to span departments and disciplines: “It's legal, it's marketing, it's the business owner and it has IT and security as a very foundational participant,” but Gupta reminded the audience that care has to be taken to not let this be an impediment to organizational progress.

Abandonment Issues

An audience question took the conversation back at what Powell had been discussing earlier around taking a balanced approach. “What I'm finding,” he said, “is sometimes these [AI pilot trials] get abandoned and now the sprawl is just the dead carcasses of agents cluttering up our enterprise.” Gurugubelli said that in larger organizations: “What we are seeing often … is, hey, let's set up a center of excellence, let's make sure there's a control tower, there's a good way to disseminate best practices across engineering or whichever teams that are building [AI], then disperse them to different business units.” Smaller businesses, on the other hand, have a greater tendency to miss the basics of security and hence introduce unnecessary risk. As he put it: “AI magnifies existing security gaps, right?”

Managing the Life Cycle

The panel was asked about the cycle of systems and software development. What is the model for beginning in a development environment, probably with little or no restriction on access to external AI, then tightening things up as we move into production realms? Powell responded: “We have a dev environment, essentially … so that people within the organization can experiment with the models. We're pretty explicit in that those models have to be taken offline … we're trying to use those offline models to do the work that we need on the datasets. If an online model is needed, essentially what we're doing now is just heavily restricting the data that you're allowed to put in scope.” He also spoke of the steps needed before something can go into production: security; governance and legal; licensing and tokenization – how much will it actually cost to run?

Powell was then asked about letting tools such as Claude Code interface with corporate systems. The response was that no, that’s not something the organization currently permits, but added that “some of that capability comes native and so we may not even have planned for its use in that particular environment, but we upgrade to the latest version of a software package and, you know, lo and behold, AI is included.” The risk was made clear: when this happens there are questions that need to be asked about data sovereignty, the risk of third parties, the risk of third parties’ third parties and so on.

Covert Deployment and Use

Gupta addressed the question of how to deal with AI “sneaking in” in this way. She noted that there is more to data than whether it is classed as personal data in a data protection sense – it could simply be sensitive or confidential corporate data. Differentiation of who can access what is essential, she said, as is the whole end-to-end reference model from the user interface right through to the AI back end. Gupta noted the ForcedLeak exploitation as an example.

In the closing remarks from the panel, Gurugubelli talked about controls: “One of the things that we all need to do, as security professionals, is keep thinking that any controls, any processes we put in place – how does this help move the business in the right direction? How can we do that fast? How are we enabling the business? How are we helping with enabling the customer trust?” He continued: “If you have good brakes in a high-performance car, you have greater control, you can go further in distance and fast, right?”

The motoring reference inevitably led to Powell next. “For us, it's been about starting somewhere.” he said, “defining guardrails and then being able to adapt quickly … I would say if the guardrails are too tight, you're going to slow everything down, right? That's not the business that we need to be in. If they're too loose, then the marble's going to bounce down the highway.” But we do not need to try to figure everything out from the ground up – instead, we can look around at others and learn from what they are doing, but always in the context that organizations are not similar. “When I talk to my peers in other manufacturing, other automotive [organizations], we're all approaching the problem slightly differently and I think that's okay”.

Gupta took the final opportunity to stretch the motoring analogy further. “You also need that steering wheel,” she pointed out, “and that's where governance comes in, right? What are you steering? Where? What do you avoid? How you get there? That steering wheel will help you get there”.

Related Insights

• ISC2 Research: Rethinking AI's Impact on Cybersecurity Roles
• AI Month: Are Incident Response Programs Ready for AI?
• For You, By You: Join Us and Build ISC2's AI Security Certification