Today, we will discuss a recent YDC project focused on Shadow AI Governance. We all know that AI introduces risk, which needs AI Governance and AI Risk Management. However, in this blog, we discuss the use of AI Governance to drive cost reduction by highlighting areas where organizations can highlight shadow AI to improve their negotiating posture and reduce costs.
The YDC team recently completed a study at a mid-size company. The hypothesis was that they had a number of applications with “Shadow AI.” We define Shadow AI as applications where vendors have added artificial intelligence capabilities into their application suite without the full knowledge of the company.
Inventory of CMDB COTS Applications
The ServiceNow team pulled a list of 800 commercial-off-the-shelf (COTS) applications from CMDB. The excel sheet included details such as Application Name, Vendor Name, Application Description, and Owner.
Appending Additional Information on Shadow AI
The YDC team conducted independent analysis and appended additional information to the CMDB file. This additional information included the existence of Embedded AI, the name of the AI, Vendor Privacy Policies on the Use of Customer Data to Train AI Models, and Opt-Out.
YDC_AIGOV AI Governance Agents Partially Automated the Process
The YDC team developed a number of agents to automate the extremely manual, time-consuming process of looking up vendor information relating to embedded AI, AI policies, etc. More of this in a future blog. For now, here is a sneak preview of our YDC_AIGOV agent in action in private preview on Hugging Face (I’ve hidden some of the details). We input the name of the vendor and receive the output in JSON format based on the YDC_AIGOV agents doing their work in the background.
Here are some of the key highlights from our analysis (we have disguised overall numbers but the percentages are pretty accurate):
-
- 32 Percent of Applications Included Embedded AI Necessitating AI Risk Assessments
59 percent of the applications did not have embedded AI, nine percent had embedded AI but excluded customer data from training AI models, and 32 percent (256 apps) had embedded AI but did not exclude customer data from training AI models. These 256 apps were the starting point for, what will likely be, a lengthy exercise. For example, each AI-embedded app constitutes an AI use case that needs to undergo an AI risk assessment.
- Tag AI-Embedded Apps in ServiceNow CMDB and Alert TPRM and Procurement to Improve Negotiating Posture
All 256 apps (32 percent) with embedded AI need to be tagged with an “Embedded AI” tag in CMDB. At the very minimum, the following additional steps need to occur:
- a.
 Update Third-Party Risk Management (TPRM): The TPRM process needs to be updated to ensure that vendor Master Services Agreements (MSAs) include clauses around the use of customer data to train AI models.
b.  Alert Procurement to Improve Negotiating Posture: If the vendors are using customer data for “free” to train AI models then they need to provide something in return (e.g., reduced pricing, vendor credits, free tickets to the vendor’s annual user conference).
- Populate 36 Percent of Application Records with Missing Vendor Name
The YDC team discovered that 36 percent (288 apps) had null vendor names. We had to populate those fields to make sure that the activities in steps 1 and 2 above were effective.
- Update 47 Percent of Application Records with Missing or Incorrect Vendor Name
The YDC team updated 47 percent (376 apps) with missing or incorrect vendor names. This list included the missing vendor names in step 3 above. Obviously, this step was critical to ensure that steps 1 and 2 above were effective.
- 32 Percent of Applications Included Embedded AI Necessitating AI Risk Assessments