Fireworks AI Breach Exposed Notion AI Chats: What Store Owners Do Now

Fireworks AI Breach Exposed Notion AI Chats: What Store Owners Do Now
Fireworks AI Breach Exposed Notion AI Chats: What Store Owners Do Now
Affiliate disclosure: This post contains affiliate links. If you buy through them, I may earn a commission at no extra cost to you. Full disclosure

Hackers who breached AI inference platform Fireworks AI stole Notion AI chat transcripts, Notion confirmed this week, affecting fewer than 1% of its workspaces.

This matters to the store owners we write for at Ecommerce Paradise, because the leak did not start at Notion. It started at a vendor behind the vendor, one most Notion customers had never heard of. If your team pastes supplier quotes, customer data or login details into any AI chat, the question is no longer whether your own tools are secure. It is who else holds a copy of what you typed.

Stop losing the details of supplier and customer calls. Record, transcribe and summarize them in a tool built for it, so nobody has to paste call notes into a general AI chat. Try Otter.ai for call notes →

What Notion and Fireworks Have Confirmed

Fireworks AI is a platform developers use to run open AI models, and Notion used it behind two features. Notion’s own security notice says Fireworks served open-weight models for Notion AI and ran classification jobs for a feature called Insights. The notice is on Notion’s help center and carries no publication date.

The exposed data is chat transcripts. Notion says that “any input to or output from the model that was written to the chat is included in that chat transcript,” according to TechCrunch’s Nov. 12 report by Zack Whittaker. That means both what a user typed and what the model answered.

The scope is small by percentage. Notion’s notice says fewer than 1% of workspaces were affected, while TechCrunch put it at around 1% of customers. Notion also says it added no metadata linking most transcripts to a workspace, user or company, so much of the stolen data cannot be tied to a specific customer.

Notion says AI Meeting Notes data, meaning audio, transcripts and summaries, was not affected unless someone used it as input in a Notion AI chat. A third use also appears in the notice. A Learning and Early Access Program let volunteers contribute data for experiments in training task-specific models on Fireworks.

The detail that stands out is a broken promise. Notion says its investigation found that “data was retained from the Fireworks endpoints that we used,” even though it had a contractual zero-data-retention commitment from Fireworks. In plain terms, the stored copy that was stolen was never supposed to exist.

Fireworks tells a narrower version of its own story. Its latest bulletin, updated Nov. 9, says inference, its Training SDK and a product called Nexus are zero data retention by default and show no detected impact. The Notion incident and that statement are hard to square, and Fireworks says its investigation is ongoing.

The Timeline: Two Breaches in Under a Month

This is Fireworks’ second incident in as many months, per TechCrunch. The first started on Oct. 6, when Fireworks detected unauthorized use of internal credentials. It published its first bulletin on Oct. 9, according to The Register, and said attackers reached environment variables in evaluation jobs, including some customers’ API keys and tokens.

The second wave is the bigger one. A third-party cloud provider hosting Fireworks’ research and development machines was attacked on Oct. 23, per The Record. On Oct. 24 the attacker used that access to reach Fireworks development machines and ran an unauthorized workload. Fireworks says it detected the activity the same day.

Notion says it learned of the problem from Fireworks. The Record reports Notion was notified on Nov. 2. Notion’s notice says it disconnected from Fireworks on Nov. 3, deleted the data in the Fireworks system, halted all training runs and stopped sending traffic there.

Public disclosure came later. The Register’s story ran Nov. 10, TechCrunch followed on Nov. 12, and Notion’s help-center notice landed in between. So there was roughly a nine-day gap between Notion cutting the connection and the story reaching most customers. If you use Notion AI, a vendor cut it off before you could even ask a question.

The company brought in outside investigators. Fireworks says it hired Palo Alto Networks’ Unit 42 and notified law enforcement. It also says it revoked and replaced credentials, isolated affected systems and tightened access to its storage and analytics.

The Register quotes Fireworks CEO Lin Qiao saying “From what we see attackers are not after weights,” and “Most downloads were partial and stopped.” That is the company’s own assessment, and the Register notes the full scope of the stolen model weights and datasets is not confirmed.

What Else Fireworks Says Was Taken

The data list goes well beyond Notion chats. Fireworks’ bulletin lists resource metadata such as identifiers, display names and creation times. It lists billing information including billing and shipping addresses, invoice and payment history, account balance and credit usage.

Fireworks says payment card numbers are not stored and were not impacted. For a limited set of accounts it says secrets, datasets and incomplete models were affected. A secret here means a credential that grants access to a user or an app, such as an API key.

The number of customers affected has not been disclosed. Fireworks calls it a limited number of accounts and says everyone affected was notified. Its customer count itself varies by source, with TechCrunch saying tens of thousands and The Record saying more than 10,000.

Bloomberg, as relayed by The Register, reports Fireworks is working with customers including Cursor and Harvey to assess impact. Neither company has been quoted in the coverage I found, so I will not guess at what they lost.

Why This Is an LLMjacking Story, Not Just a Privacy Story

Fireworks says the evidence points to financially motivated criminals who sell LLM tokens obtained through unauthorized access. That practice has a name, LLMjacking, and TechCrunch says the attackers aimed to sell model access to third parties who then use it without paying.

The timing matters here. On Nov. 2, Google Threat Intelligence Group and Mandiant published a report on LLMjacking. It found fraudulent charges “exceeding $100,000 USD in a single day” and cumulative charges “at some organizations reaching $1 million USD before access was revoked.”

The same report says the dominant way in has been exposed or stolen cloud identity tokens. That includes API keys, service account keys and personal access tokens, and it describes corporate LLM keys harvested from public GitHub repositories. One actor, called Jet Night, ran more than 500,000 LLM API calls in under eight hours.

Put those two stories side by side and a pattern shows. Attackers want AI keys because AI keys are cash. A secret stored in an evaluation job, a chat transcript where someone pasted a key, or a billing account tied to a card all have a price.

My read is that the Notion story is the quiet version of this trend. The loud version is a $100,000 day of fraudulent API charges. The quiet version is a pile of chat logs sitting on an attacker’s drive, waiting to be searched for strings that look like passwords.

Use one AI account with a spending cap, not ten loose keys. Keep your AI work in a plan you control, with a billing limit set and keys you can rotate. Start with Claude →

What a Store Owner Probably Pasted Into an AI Chat

This is the part I would worry about, and it is a judgment call, not a reported fact. Nothing in the coverage says what the exposed Notion chats contain. The Register notes that Reddit users argued conversations could carry identifying details even without linking metadata.

So think about what actually gets typed into a workspace AI assistant. A store owner drafts a supplier negotiation and pastes the quoted wholesale prices. A virtual assistant asks it to clean up a customer list. Someone pastes an API key to ask why a Shopify integration is failing, or a refund policy that names a lawyer.

Those are hypotheticals, not findings. The point is that chat is a place where people hand over context they would never put in a public document. A transcript holds the question and the answer, so it holds the secret and the explanation of what the secret opens.

Notion’s own advice reflects that. The notice tells everyone to treat the exposed data as exposed, and says anyone who pasted passwords, API keys or other credentials into Notion AI should rotate them immediately. It does not set a date range beyond “the relevant timeframe,” so I would rotate anything you remember typing.

If you outsource work, the exposure grows. Hiring offshore help through a service like OnlineJobs.ph is a standard part of running a lean store, and those assistants often work inside shared tools. If a VA used your Notion AI to handle supplier emails, their chats may be in the affected set too.

What Changed for Ecommerce Operators

Three things shift because of this incident, and none of them is “stop using AI.” The first is that a vendor’s zero-retention promise is a contract line, not a technical guarantee. Notion had one, and the data was still retained.

The second is that subprocessors are now part of your risk. When you use a SaaS tool with AI features, your data may travel to a model host you never signed with. I covered the wider cleanup pressure in my posts on Shopify’s shorter erasure window. California’s SB 923 on deletion requests, which I wrote up in its own post, points the same way, and both assume you know where customer data lives.

The third is that the cost of a mistake is moving from reputation to cash. The GTIG numbers are enterprise-sized, but a small store with one leaked key and no spending cap can still wake up to a surprise invoice. Setting a quota takes ten minutes, and cleaning up after fraud takes weeks.

Customer data deserves its own paragraph. If a chat held names, emails or order details, you may have a notification duty under the privacy laws that apply to you. I am not a lawyer, so check with one before you decide whether a notice is needed.

The disclosure side also matters. If you run a store, your privacy policy should say which processors touch customer data, including AI vendors. A policy generator such as Termly can keep that page current as your tools change.

7 Moves to Make This Week

None of these costs money, and most take under an hour. If you use Notion, start at the top. If you do not, start at step three, because the same questions apply to every AI tool in your stack.

  1. Open Notion on a desktop browser, click the question mark in the sidebar and look for Security incident review. Workspace owners can download the affected member list, and owners with a verified domain can download affected content files, per Notion’s notice.
  2. Rotate anything sensitive you or your team typed into an AI chat: API keys, store passwords, supplier portal logins and payment links. Do it even if you were not notified. Put the new ones in a password manager, and my comparison of the best password managers for small teams covers the options.
  3. List every tool that has an AI feature switched on, from your helpdesk to your email platform. For each one, find out which model provider sits behind it and whether the vendor offers a zero-retention setting. If it is a customer-facing chat widget, check what it stores. Tidio is one example of a store chat tool worth asking.
  4. Set spending caps and quotas on every AI account that has an API key. GTIG’s mitigation list recommends granular quotas per model and budget limits, and I would add one rule: no key lives in a chat message or a public repository.
  5. Write a one-paragraph AI-use rule for your team and your contractors. List what must never be pasted into any AI chat, such as customer lists, card data and credentials. My cybersecurity tips for growing stores has a starting checklist.
  6. Check your customer-facing records for exposure. If a customer service person used Notion AI to draft replies, review what those drafts contained, and keep your helpdesk records in one place such as Gorgias so you know where the data sits.
  7. Turn on phishing-resistant multi-factor authentication for your email, Shopify admin and ad accounts. GTIG describes a session-hijacking page that got around passwords and MFA, so a code on your phone is not enough on its own. Use a hardware key where the platform allows it.

Run through the list once, then put a recurring reminder on the calendar every quarter. The tools will change faster than your habits do, and the audit is the habit.

What to Watch Next

Four things are still open. Fireworks has not said how many customers were hit by either October attack, and the full scope of the stolen model weights and datasets is unconfirmed. The cloud provider that hosted Fireworks’ machines has not been named, according to The Register.

Notion says its investigation is ongoing and that it will contact any workspace it later finds was impacted. It has disabled Insights during the investigation and stopped serving open-weight models through Fireworks. It also announced an audit of critical vendors, such as inference providers and data processors, to verify data minimization and zero-data-retention practices.

My bet is that other SaaS vendors get asked the same question this month. Procurement teams will start asking whether an AI feature sends data to a third-party model host, and whether that host retains it. Stores that can answer in one sentence will have an easier time with partners and payment processors.

If you are building a store from scratch, build it with fewer places for data to leak. Fewer tools, one helpdesk, one email platform and a short list of vendors you can name. That discipline is part of what my turnkey done-for-you service sets up. The model behind it is explained in my guide to high-ticket dropshipping.

Frequently Asked Questions

Was my Notion workspace affected?
Notion says fewer than 1% of workspaces were, and that it has contacted affected customers. Owners can check Security incident review in the sidebar help menu, and Notion says it will contact any workspace it later finds was impacted.

Did the breach expose Notion pages, databases or docs?
Notion says no other data from within your workspace is affected, according to an email quoted by The Register. The exposed material is Notion AI chat transcripts from the Insights feature and open-weight model serving, plus data from the volunteer training program.

Do I need to rotate my API keys?
Notion tells people who pasted credentials into Notion AI to rotate them immediately. Fireworks says customers whose secrets were affected are being notified with response actions, so if you also use Fireworks directly, contact security-notice@fireworks.ai. Rotating a key is cheap, and a leaked one is not.

Is it safe to use AI tools in my store at all?
Yes, with rules. Keep credentials and customer lists out of chats, set spending caps on keys, and ask each vendor which model host it uses. My guide to AI tools for non-developers covers how to pick tools. My post on protecting high-ticket stores from cyber threats covers the wider setup.

What is LLMjacking?
It is the theft and resale of access to AI models. Google’s GTIG and Mandiant report says attackers mostly get in through exposed or stolen cloud tokens and API keys, then run up charges on the victim’s account. Fireworks says the activity it saw matches financially motivated actors selling LLM tokens.

Where should a new store owner start?
Start with the niche, then the tools. My free high-ticket niches list is the fastest way in. The full niches guide explains how to judge one.

Want a store built with a clean, short tool stack from day one? My team sets up the store, the suppliers and the email so you are not stitching together ten AI add-ons. See the turnkey store build →

Subscribe to the YouTube channel for daily breakdowns. More breaking news coming through the day.

Related Articles

If this was useful, these go deeper: