Trusting the machine – Part 3

Recap: In Parts 1 and 2, we explored why trusting AI is so difficult and how organisations might build systems that deserve an appropriate level of trust. In Part 3 we dive a little deeper into trust issues with data management and some practical tips to address these.  

AI is extraordinarily useful, increasingly cheap and becoming embedded in everyday work and life, but as with social media, the utility and value may be more complicated than on the surface. The more useful AI becomes, the more personal information we are likely to give it. That raises a simple question, are we just using the machine, or is the machine also learning to use us? Are we the real data product? Will our data be used to influence us? Is it being used to train AI that will render us obsolete?

These questions aside, it's not hard to imagine the inherent trust issues we are now facing.

The most familiar workflow for generative AI, usually means chatting with an LLM and sending that information to a server. A prompt leaves your device, is processed on infrastructure outside your direct control and then an answer comes back. For simple tasks like searching for something on the web or asking it to give you a holiday itinerary, it’s easy not to worry about where that information is going. 

What about more complex needs? What if the prompt contains a customer’s medical history, commercially sensitive intellectual property or the details of somebody’s financial situation?

Where the machine lives now becomes part of the trust equation.

Head in the clouds?

Cloud computing taught organisations an enormously successful model, centralise infrastructure, outsource much of its complexity and consume computing power when required. Generative AI inherited the same architecture.

The largest and most capable models require enormous computing resources, so it was the most obvious model for deployment. There are good reasons for doing this. The advanced models can be far more capable than smaller models. Cloud infrastructure can be easier to manage, update, secure and scale.

However, every time information leaves an organisation’s existing environment, another party, system, contract, network connection and set of controls enters the chain of trust.

The Australian Office of the Australian Information Commissioner already recommends that organisations avoid putting personal information, particularly sensitive information, into publicly available generative AI tools. It also specifically notes that deploying AI locally or on premises is likely to be more privacy preserving because it limits the risk of third-party access.

A practical example

Imagine a superannuation fund wants to use generative AI in its contact centre.

A member calls after losing their job. During the conversation they discuss their account balance, mortgage, health problems, family circumstances and financial stress. There is enormous potential for AI to help.

It could transcribe the call, summarise the member’s circumstances, identify actions, retrieve relevant policy information, draft a follow up email and reduce the administrative workload for the consultant.

However, thinking about the technical aspects, the audio might be sent to one cloud service for transcription. The transcript could be passed to another AI service for summarisation. Information might then be stored in a separate database so another model can retrieve it later.

A useful customer experience can quickly create a complicated data journey. None of this necessarily means the system is unsafe. Each provider might have sophisticated security, meet all data security laws and maintain strong privacy controls. However the organisation is asking the member to trust a surprisingly long chain of technology they will never see.

Now imagine the same process a few years from now.

The initial speech to text model runs locally. A smaller language model inside the organisation identifies personal information, creates the call summary and retrieves standard information from an internal knowledge base. For perhaps 90% of calls, that is enough. When the customer understands that their data is not being sent elsewhere, there are fewer points of access.

Perhaps the smaller model right now is not capable of such tasks, but things are improving at a rapid pace.

The smallest capable machine

Right now, in the early stages of consumer LLMs, we have become accustomed to assuming the best AI model is the most powerful one. More features, bigger context windows and longer tasks.

As technology improves and AI of things (AIoT) becomes upbiqutious, consumers will have more powerful devices and smaller models will be improving. We may also see increasng push back against the mega data centres now being built in the AI arms race. Beyond their unpopular water, energy and land usage, its not unimaginable current valuations will need to be corrected, impacting the viability of this solution long term. The best model for a particular task may actually be the smallest model capable of doing it “well” enough. Eventually it will be less about “what’s the biggest model I can use” and more about “what’s the most efficient model”.

Smaller models increasingly run directly on laptops, phones and organisation-controlled infrastructure. Microsoft now offers local AI infrastructure where prompts and outputs can remain on the device, while Apple has similarly adopted an architecture that attempts processing on device before using more powerful private cloud models for harder tasks.

Not every problem requires the biggest model in the room. Particularly if inviting that model into the room requires sending all your data elsewhere.

Where does AI live?

Local AI

Run the task on the user’s device or within an organisation-controlled environment.

Use this when information is highly sensitive and the task is sufficiently bounded for a smaller model.

Private AI infrastructure

For workloads that require more capability or need to serve many employees, models might run on infrastructure controlled by the organisation, a dedicated environment or privacy-enhancing cloud infrastructure. The organisation gains more capability while keeping tighter control over where information can go.

External AI

Use the largest externally hosted models where their additional capability genuinely creates sufficient value and the data is appropriate to send. This should not imply that local is good and cloud is bad. It means the architecture becomes proportionate to the task.

Local doesn’t automatically mean safe

There is an obvious trap here. Running an AI model locally does not magically solve AI risk.

A badly secured laptop is still a badly secured laptop. A local model can still hallucinate. It can still be biased. It can still make a bad recommendation confidently.

Sensitive information could still appear in logs, backups or monitoring systems. Internal employees with privileged access could still see things they should not. A compromised model or malicious document could still create security problems.

This is important because technological enthusiasm has a habit of replacing one simple story with another. First it was “the cloud is risky”, then “the cloud is safer than your own infrastructure”. We should be cautious about replacing that with “local AI is safe” but it certainly has advantages in certain situations.

Five things to start doing to improve trust in AI

The technology for this future is still developing, but organisations do not need to wait for it to be perfect before preparing.

1. Add data location to AI use case classification

Part 2 proposed classifying AI according to the stakes involved. Add another question: Where should the data be allowed to go?

An AI register should identify not only what a model does, but what information it can access, where inference occurs, what leaves the organisation and what is retained.

2. Stop making the most powerful model the automatic default

Benchmark smaller models against the actual task.

If a smaller model running inside your environment performs sufficiently well, the extra few percentage points offered by an advanced model may not justify expanding the trust boundary.

3. Bring models to sensitive data

Identify workloads where valuable organisational data is currently considered too sensitive for generative AI. These may become some of the strongest use cases for local AI.

Legal documents, employee records, customer case notes, research material, intellectual property and health information all contain enormous potential value precisely because they are information rich. Keeping them out of AI forever is one solution. Building an architecture capable of safely taking AI to them may be a better one.

4. Make external processing an escalation rather than the default

Hybrid architecture could become to AI what escalation pathways are to human decision-making. Try the lower risk option first. If it cannot do the job, move up a level.

Importantly, this escalation does not always need to send the original information with it. A local model may be able to remove identifiers, summarise the relevant facts or convert sensitive raw information into a less sensitive representation before another model sees it.

Look to disclose the minimum necessary information to achieve the task.

5. Benchmark trust as well as accuracy

The evaluation question should not only be: Which model produced the best answer?

It should also ask: Which architecture produced an acceptable answer with the least unnecessary exposure?

Accuracy, speed and cost remain important, but privacy, controllability, auditability and the consequences of failure should sit beside them.

Bringing the machine home

We probably won’t stop sending information to large cloud models. There will continue to be tasks where the capability of the most advanced systems overwhelmingly justifies using them.

However, AI may become less like one enormous machine we send everything to and more like a network of machines with different capabilities and different levels of trust. Some on our devices, inside our organisations, inside protected computing environments and some on the cloud.

This approach might prove to be one of the most practical ways of resolving some of our discomfort with trusting the machine. We need to transcend a decision of whether to trust or not to trust. Don’t trust a machine with everything, but contextually separate different trust levels with different requirements. Besides this is just like personal disclosure decisions we make on a daily basis with other humans when deciding how much of your personal life to divulge depending on how much you trust them.

Selected bibliography

• OECD, Sharing Trustworthy AI Models with Privacy-Enhancing Technologies, OECD Artificial Intelligence Papers, 2025.

• National AI Centre, Australian Government Department of Industry, Science and Resources, Guidance for AI Adoption, 2025.

• International AI Safety Report, International AI Safety Report 2026.

• Microsoft, Build, deploy, and govern sovereign AI with Foundry Local on Azure Local.

• Apple Security Research, Private Cloud Compute Security Guide and Expanding Private Cloud Compute.

Author

Anthony Zhang is a Senior Data Insights Consultant at MYMAVINS. An experienced economic and social researcher, Anthony expertise lays in quantitative research and data science.