01 · Data
What enters, leaves, and persists?
Include prompts, files, retrieval context, outputs, logs, telemetry, backups, support records, and deleted items.
Architecture comparison
On-premises private AI vs. public and hosted AI.
There is no safe architecture in the abstract. Choose the data boundary, control model, and operating responsibility that fit the work your firm will actually do.
Published by PrivateStride · Last updated August 11, 2026
Three categories
Separate where AI runs from how well it is governed.
“Private” can describe several different technical and contractual arrangements. Ask for the diagram.
A public AI service commonly runs in a multi-tenant, provider-operated environment, even when customer data is logically separated. Controls vary by product and plan. A dedicated hosted system reserves the applicable environment for one customer. An on-premises system runs on hardware in the firm’s environment.
Any of these can be implemented well or poorly. Architecture changes which parties and systems are involved. Governance shows whether the organization understands and manages those risks.
See the PrivateStride on-premises data boundaryPartner comparison
The tradeoffs in plain language.
Verify every statement against the provider’s current architecture, settings, and contract. Product names alone are not evidence.
| Decision point | Public AI service | Dedicated hosted AI | On-premises private AI |
|---|---|---|---|
| Inference location | Provider-operated environment | Single-customer hosted environment | Hardware in the firm’s environment |
| Public model API dependency | Usually central to the service | Depends on the provider’s architecture | Can be removed from the inference path |
| Administrative control | Provider controls the platform; firm configures available settings | Shared between firm and provider by contract and design | Firm controls the environment; provider duties must still be defined |
| Updates and maintenance | Mostly handled by provider | Mostly provider-managed, subject to the agreement | Requires local operations or a managed service |
| Internet dependence | Normally required for use | Normally required for use | Core inference can continue locally; updates and support may need controlled access |
| Best first question | Can this exact data and workflow use this exact service? | What is dedicated, who can access it, and what leaves it? | Can the firm operate, secure, and govern the local system well? |
Seven diligence questions
Ask the same questions of every option.
A comparable record makes the architecture decision easier to explain to partners, counsel, the Qualified Individual, and the firm’s IT provider.
02 · People
Who can see or administer it?
List users, privileged administrators, vendor personnel, support teams, and subprocessors. Verify approval and revocation.
03 · Purpose
What may the provider do with it?
Review service delivery, security monitoring, abuse review, product improvement, model training, and legal-response terms.
04 · Controls
What can the firm configure and prove?
Ask about identity, roles, MFA, encryption, network paths, retention, logs, exports, testing, alerts, and incident evidence.
05 · Change
Who decides when the system changes?
Document model updates, feature releases, integrations, data location changes, notices, testing, and rollback options.
06 · Resilience
How does work continue and recover?
Test internet loss, vendor outage, hardware failure, backup, restoration, degraded operation, and a safe manual workflow.
07 · Exit
How do you leave?
Define export, deletion, verification, hardware return, model or prompt portability, contract termination, and records retention.
When on-premises fits
Choose local processing for a reason you can state.
On-premises private AI can fit a firm that wants client-work inference in its own environment, has repeatable high-value workflows, and is willing to govern a dedicated system. It also needs an operating plan for hardware, patches, model changes, monitoring, backups, and support.
It may be a poor fit when the firm cannot assign owners, define approved workflows, coordinate with IT, or support the necessary controls. In that case, a carefully reviewed hosted service or a narrower non-client-data use may be the responsible first step.
Map your requirements in the auditDecision record
- Define the workflow and data class.
- Compare the complete data flows.
- Assign every operating responsibility.
- Review legal, privacy, security, and engagement duties.
- Test with synthetic or approved information.
- Approve expansion only after measured evidence.
Risk framework
Architecture is an input to the decision, not the score.
Evaluate risks in the context of the organization, users, affected people, intended purpose, and complete system lifecycle.
Primary sources: NIST describes voluntary AI risk-management practices in the AI Risk Management Framework and AI-specific considerations in its Generative AI Profile. The FTC’s Safeguards Rule guide discusses risk assessment, access controls, encryption, monitoring, service providers, change management, and incident response for covered financial institutions. See our claims methodology for the evidence behind PrivateStride statements.
Keep reading
Turn the policy into a working system.
These guides cover the decisions that sit next to this one.
ChatGPT and client data
Apply the architecture questions to a specific service and account.
Read the guideSafeguards Rule and AI
Connect the chosen system to the written security program.
Read the guideAI use policy starter
Translate the decision into rules people can follow.
Read the guideCapacity & AI risk assessment
Map the risk before you choose the tool.
In 30 minutes, we identify your highest-value workflows, likely shadow-AI exposure, and the controls a private AI program would need. The findings are yours to keep.
Book your audit 30 min · No preparation · Confidential