Where Your Company's Data Goes When You Use AI

A consumer AI account trains on your data by default - even when you pay for Plus or Pro. You pay twice: with the subscription and with data, yours and your customers'. The difference between "your data trains someone else's model" and "your data stays yours" often comes down to a single pricing plan. If your team works with customer data, the company pays for a business plan. A personal account holding customer data is a decision nobody at the company actually made.

One question before we start: when did someone at your company last paste something into ChatGPT? Exactly - you don't know. Below we show how to check and what to change.

Does AI train on your company's data?

It depends on one thing: which account your team uses. Consumer accounts - ChatGPT Free and Plus, Claude Free, Pro and Max - use your conversations for model training by default. Business plans - ChatGPT Business and Enterprise, Claude Team and Enterprise, the API, Microsoft 365 Copilot - don't, by default. So the first question to ask about any AI tool is which account your team works in.

If your team uses private consumer accounts for work, the content of those conversations - customer data included - can end up training someone else's models. Legally, and in line with terms of service everyone accepted without reading.

"Surely they ask for consent first" - no, they don't

On consumer accounts, training is switched on from the start. At both OpenAI and Anthropic the "improve the model" toggle is on by default, so the consent box was ticked for you. To withdraw it, you have to find the toggle in your settings yourself. We see this regularly when we talk to teams: most people have training switched on in their personal account. That's our observation, not a study.

Anthropic introduced these rules on August 28, 2025 - Free, Pro and Max users had until October 8, 2025 to make a choice. Anyone who changed nothing still has training switched on. Anthropic's announcement of the new consumer terms says it plainly: "We will train new models using data from Free, Pro, and Max accounts". That holds as long as the setting is on.

At Anthropic, opting out only works going forward: your data "will still be included in model training runs that are already in progress". Conversations that went into training before you switched it off stay there. Retention depends on the same choice: with training consent your data lives up to 5 years, without it - 30 days.

What's different about a business plan?

Business plans flip the logic: training on your data is off by default. OpenAI states on its Enterprise privacy page: "By default, we do not use your business data for training our models". The page explicitly lists ChatGPT Business, Enterprise, Edu and the API. Anthropic's commercial terms work the same way: no training on your prompts or code unless you actively join their partner program.

Microsoft 365 Copilot goes a step further because it operates under an agreement with your organization. Per Microsoft's enterprise data protection documentation: "the prompts, responses, and data accessed through Microsoft Graph aren't used to train foundation models". Microsoft formally acts as a data processor under a Data Processing Agreement (DPA). That is the legal construct privacy law expects whenever you hand customer data to a third party.

Companies using the API can also apply for Zero Data Retention. OpenAI then doesn't store request content at all, but it grants this only for eligible endpoints.

Watch the names: Microsoft now calls both the business version and the free app at copilot.microsoft.com "Microsoft Copilot". What protects your data is the account you sign in with. A work account is covered by the contract. A personal Microsoft account isn't, and it trains on your chats until you switch that off.

Is ChatGPT safe for business?

ChatGPT is safe for business on two conditions. First, your team works on a Business or Enterprise plan. Second, nobody pastes in passwords or data you're not allowed to share with a vendor. On Free and Plus, conversations can train OpenAI's models by default, and your company has no data processing agreement with the vendor. On Business and Enterprise, training is off by default, and OpenAI executes a DPA with your company through a form on its website.

The second condition depends on people. A well-configured plan won't help if someone pastes the password to your accounting system or a contract under an NDA into the chat. The table of what not to share is below.

Comparison: what each tool does with your data

The dividing line doesn't run between vendors - it runs between account types. OpenAI, Anthropic, Microsoft and Google all follow the same pattern: a consumer account trains models on your conversations by default, a business account doesn't. Before you pick a tool for your company, find the table row your team actually works in.

Current as of September 2026. Vendors change these rules, so check the last-updated date on their site before you decide.

What about Google Gemini?

Google Gemini follows the same split as the other vendors. The consumer version trains models on your conversations by default; Workspace and Vertex AI don't. On top of that, in consumer Gemini some of your conversations are read by humans. Google employs reviewers (including external contractors) who read selected chats - regardless of your settings. Conversations that went through review are kept for up to 3 years, even if you turn off activity history and delete everything. Google says it outright in its Gemini Apps privacy hub: "Please don't enter confidential information that you wouldn't want a reviewer to see".

On the business side it's clean. Google Workspace with Gemini doesn't train on your company's data and doesn't show it to reviewers without permission. Vertex AI (the API) carries a zero-training commitment. Workspace also lets you pin data storage and processing to a European region.

What if employees use AI outside company accounts?

The phenomenon has a name: shadow AI. It means AI tools used at work outside the company's knowledge and oversight, most often through private consumer accounts. Picture an ordinary Wednesday: your bookkeeper gets a spreadsheet of client records to sort. She pastes it into a free chatbot, because it's faster. From that moment, tax IDs, addresses and amounts sit on a server abroad, and under default settings they can end up in model training. Nobody at the company knows, because there's nothing to notice: no alert, no log. We broke down its scale and mechanics in a separate article on shadow AI at work. Oversight of that kind of use gets a broader treatment in the article on whether you can trust AI in business.

What does the law say - GDPR and beyond?

When your employee pastes customer data into an AI tool, the responsibility doesn't vanish on the other side of the screen. Your company remains the data controller. If you serve customers in the EU, GDPR applies regardless of where your company sits. The AI tool acts as a data processor, and Article 28 of the GDPR requires a Data Processing Agreement (DPA) for exactly this relationship. US state privacy laws such as the CCPA impose comparable contractual duties when you share personal data with a vendor.

Business plans give you that agreement. With OpenAI you execute it through a form on its website; at Anthropic it's automatically part of the commercial terms. Microsoft acts as a processor in Microsoft 365 Copilot, and Google does the same in Workspace. A consumer account offers no such agreement. With customer data, your company therefore has no Article 28 basis - and the conversations may train the model on top of that.

In December 2024 EU regulators added one point. A vendor's assurance that its model is "anonymous" doesn't spare the company using it from its own assessment.

As for the EU AI Act, the obligations for a company merely using ready-made AI tools are narrower than the headlines suggest. They come down to transparency (Article 50) and AI literacy for your team (Article 4). Data privacy is governed by the GDPR, not the AI Act.

What should you never put into ChatGPT or other AI tools?

Nothing goes into a consumer account that you wouldn't show someone outside the company. A business plan with a DPA moves that line. A few categories still stay out of AI for good, above all passwords and access keys. The table applies to every tool: ChatGPT, Claude, Gemini and Copilot.

Nobody on the team will remember seven rows. People get one rule, the same one as in our piece on shadow AI at work. Customer personal data, contracts and financials don't go into any AI tool. Passwords and keys stay out for good. Add the exceptions from the right-hand column only once the company has a business plan with a DPA.

Three things to do today

1. Check the toggles. ChatGPT: Settings → Data controls. Claude: privacy settings → "model improvement". Five minutes per account, zero cost.
2. If anyone at the company works with customer data - move to a business plan. A business plan switches off training on company data and gives you a data processing agreement. Without those two, customer data shouldn't go into any AI tool.
3. Agree on one team rule: what we never paste. Start from the table above; one page is enough. The full AI policy can come later, the page works from tomorrow.

One step further, when the data is meant to stay with you for good: your own Company Memory. Documents, procedures and decisions sit in one base the AI answers from, instead of travelling to somebody else's chat with every question.

Data is one side of the coin. The other is control over what AI does day to day, which we break down in Can you trust AI in your business?

One last question: which accounts at your company have training switched on today, and who knows about it?

If you don't know which accounts your team works in today, that's where our audit of where your data actually goes starts. We also deploy AI on business plans and train teams on what can and cannot be pasted.

Frequently asked questions

Is ChatGPT safe for business?

Yes, if your company uses a Business or Enterprise plan: training on your data is off by default there, and OpenAI signs a data processing agreement (DPA) with your company. Free and Plus accounts come without that agreement and can use your conversations for training by default. On every plan the same rule applies: passwords and access keys never go into the chat.

Does ChatGPT train on my data?

On Free and Plus accounts - yes, by default. Your conversations can be used for model training until you turn it off under Settings → Data controls. On Business, Enterprise and API plans - no, by default.

Does Claude use my conversations for training?

On Free, Pro and Max accounts - yes, if the "model improvement" toggle is on, and it has been on by default since August 28, 2025. Opting out only works going forward. Team, Enterprise and API plans don't train on customer data.

What is shadow AI?

AI tools used at work without the company's knowledge - most often from private consumer accounts. What the phenomenon is, how to check its scale in your own company and what to do about it, we cover in a separate article on shadow AI at work.

Is it safe to paste customer data into AI tools?

Not into a consumer account: there's no data processing agreement, and the conversations can train models. On a business plan with a DPA - yes, within the purpose you collected the data for: OpenAI's DPA is executed through a form, Anthropic and Google Workspace include it in their terms, Microsoft covers it in Microsoft 365 Copilot. Your company remains the data controller either way.

What should you not put into ChatGPT?

Passwords, API keys and login credentials - never, on any plan. On Free or Plus, also leave out customer and employee data, health data, contracts under an NDA, financials and source code. Until your company has a business plan with a DPA, the rule is: customer data, contracts and financials don't go into any AI tool.

Does ChatGPT store your data?

Yes. Saved conversations stay in your account until you delete them, and on Free and Plus they can also be used to train models unless you switch that off in Settings → Data controls. The exception is Temporary Chat: it stays out of your history and out of training, but OpenAI may keep a copy for up to 30 days for safety purposes.

Does ChatGPT delete your chats?

A deleted chat disappears from your account immediately and from OpenAI's systems within 30 days. It stays longer only if OpenAI had already de-identified it or must keep it for security or legal reasons. A deleted chat can't be restored.

Does my data stay in the EU?

Depends on the tool and configuration. One trap worth knowing: Anthropic models inside Microsoft 365 Copilot are excluded from the EU Data Boundary - Microsoft keeps them off by default for EU customers, and enabling them moves processing outside the EU.

How much does an AI-related data breach cost?

According to IBM's 2025 report, organizations with high levels of shadow AI - employees using AI tools the company doesn't control - paid on average $670,000 more per breach than those with low levels or none.