# Does AI Create Trust or Uncertainty?

During a conversation with a prospective customer, a question came up that stayed with me. They wanted to understand what would happen to their institution's data if AI were introduced into the software. How could they be sure that their information would not be leaked, shared without their knowledge, or used to train AI models?

It was a reasonable concern, and one that made me think about the growing presence of AI in everyday software.

While technology companies are busy introducing AI-powered features, the people evaluating these products may have a very different question in mind: *What happens to our data?*

Working in educational technology, I understand why this question matters. A Student Information System holds information that institutions are responsible for protecting, including student identities, academic histories, attendance records, and other personal details. Introducing AI into such a system naturally raises questions about how that information will be accessed and handled.

But this concern extends far beyond education.

Organizations use software to manage finances, employee records, customer relationships, healthcare information, business operations, and confidential documents. As AI becomes part of these applications, organizations need to understand whether their information remains protected.

The question is no longer simply what AI can do. It is also **what AI can access, how that information is handled, and who remains in control of it.**

## Does Using AI Mean Our Data Is Used for Training?

One common concern is that whenever AI processes information, that information automatically becomes part of its training data.

That is not necessarily the case.

An AI system can use information to complete a task without using it to train or improve its underlying model. However, this depends on how the software is built, which AI providers it uses, and the policies and agreements governing those providers.

It is also important to understand that processing, storing, sharing, and training on data are different activities.

For example, a software application may send selected information to an external AI provider to generate a response. Even if the information is not used for training, it may still be retained temporarily or recorded in system logs, depending on the provider's arrangements.

This is why a vendor's assurance that customer data is not used for AI training is valuable, but does not answer every privacy question.

Organizations also need to understand where their information goes, how long it remains there, and who can access it.

And this brings us to an important point: trust cannot be established through a website headline alone.

Terms such as *secure AI*, *privacy-first*, and *enterprise-grade protection* may sound reassuring, but buyers need to understand what those claims mean in practice.

Rather than relying entirely on promises, they should know what to ask and what evidence to look for.

## Eight Questions Worth Asking Before Trusting AI With Your Data

These questions are relevant whether you are purchasing AI-enabled software or developing AI features for your own customers.

For buyers, they provide a starting point for evaluating vendors. For software companies, they highlight the safeguards that should be considered before introducing AI into a product.

### 1\. What information can the AI access?

Does the AI use only the information required for a specific task, or can it access entire databases, documents, and historical records?

AI should not automatically receive access to everything an application stores. Its access should reflect the task being performed and the permissions of the person using it.

### 2\. Is our data used to train or improve AI models?

Ask whether customer information, uploaded files, prompts, or AI-generated responses are used for model training or improvement.

If the software relies on third-party AI providers, understand whether the same restrictions apply to them.

These commitments should ideally be documented in writing rather than communicated only during a sales discussion.

### 3\. Where does our information go?

Does the data remain within the software vendor's infrastructure, or is it sent to another AI service?

Organizations should know where their information is processed, whether it crosses national borders, and which external providers are involved.

This becomes particularly important when industry regulations or contractual requirements restrict how information may be handled.

### 4\. How long is the information retained?

Are AI prompts, responses, uploaded documents, or activity logs stored? If so, for how long?

Can the information be deleted, and what happens to it when the customer stops using the software?

A promise not to train AI on customer data does not necessarily mean that the data is never retained.

### 5\. Who can access our data?

Are there controls preventing unauthorized employees, users, or other customers from accessing sensitive information?

Ask about user permissions, encryption, access records, and safeguards that prevent information from being exposed across different organizations.

The vendor should also be able to explain how it protects AI features against attempts to retrieve information that a user is not authorized to see.

### 6\. What happens when AI makes a mistake?

AI can generate incorrect or misleading information, sometimes with considerable confidence.

If an AI feature produces an inaccurate report, recommendation, or summary, can a person review and correct it?

Organizations should be especially careful when AI-generated information could influence important decisions or automatically update official records.

### 7\. Can we control or disable AI features?

Can administrators decide which AI features are available and who can use them?

Can certain capabilities be restricted or turned off without affecting the rest of the software?

Giving customers control over AI functionality allows organizations to adopt it at a pace that matches their needs and responsibilities.

### 8\. What evidence supports the vendor's claims?

Can the vendor provide relevant security documentation, independent assessment reports, privacy policies, contractual commitments, and details about third-party AI providers?

Security certifications can offer reassurance, but buyers should understand what those assessments actually cover.

The goal is not to expect software to be completely free of risk. It is to understand the risks, the safeguards in place, and whether they meet the organization's requirements.

## Not Every AI Feature Carries the Same Risk

As AI becomes part of more applications, it is important to recognize that not every feature requires the same level of scrutiny.

Using AI to improve the wording of an email is different from using it to analyze financial records, evaluate job applicants, interpret medical information, or recommend decisions affecting individuals.

The sensitivity of the information and the consequences of an incorrect result should influence how AI is designed and used.

For software companies, this means thinking carefully about what information each AI feature genuinely needs. Limiting access, providing user controls, and requiring human review where appropriate can reduce unnecessary exposure.

For buyers, it means evaluating individual AI capabilities rather than assuming that an entire product is either safe or unsafe simply because it includes AI.

A useful feature should not require unrestricted access to an organization's information.

## Building Trust Takes More Than Adding AI

AI offers real opportunities to make software more helpful, efficient, and accessible. But introducing AI also creates responsibilities that vendors and buyers cannot afford to overlook.

Software companies need to be transparent about how information is processed and protected. Organizations purchasing software need to ask informed questions and verify that the safeguards are appropriate for their data.

A product demonstration can show what an AI feature does. A website can explain its benefits. Neither, on its own, can establish how securely customer information is handled.

I often think back to the question that prospective customer asked.

They were not necessarily opposed to AI. They simply wanted confidence that adopting new technology would not compromise the information their institution was responsible for protecting.

And that is a concern any organization can relate to.

AI adoption should not require blind trust. It should come with clear explanations, appropriate safeguards, and meaningful control over data.

**Because the real question is not how much AI a software product offers, but how responsibly it handles the information entrusted to it.**