They can explain a report, organise a timeline or help prepare questions for a doctor. The problem is sending your identity and private context when that convenient service might need only part of the medical information. Before uploading anything, there are 4 reasons to remove what the system doesn’t need.
1. Your vulnerability can become a sales signal. Health data does more than just describe a condition; it can show your fears and what you may be desperate to solve. If a service detects anxiety about fertility, cancer, weight, chronic pain, sexual health, or an unusual result, that signal can be used to target you with ads for supplements, tests, subscriptions, or “miracle” products when you are most vulnerable. The FTC’s BetterHelp case showed that this is not theoretical. The regulator alleged that sensitive questionnaire information was disclosed to advertising platforms despite privacy promises. Would you knowingly tell an advertiser your symptoms and fears before seeing what they want to sell you?
2. Your data can influence how institutions score you behind your back. Health information can affect risk assessments. Depending on the country, product, and legal framework, it could impact insurance pricing, underwriting, claims, or eligibility. The EU AI Act classifies AI used for risk assessment and pricing in life and health insurance as high-risk. A system can also misread an old medication, confuse a temporary condition with a chronic one, or create an inferred profile, based on assumptions, that you cannot see or correct. Could the service keep or pass on information that later influences a serious decision?
3. A cybersecurity breach gives an attacker more than just your name. Clinical documents often contain dates of birth, addresses, insurance details, medical record numbers, clinic names, appointments, medications, and family information. This data can facilitate medical identity theft, account takeovers, or sophisticated phishing. Health records are among the most valuable data on the dark web, and stolen data typically appears for sale within 24 to 48 hours of a breach. An attacker who knows your provider and next appointment can impersonate a hospital, pharmacy, or insurer far more effectively than a generic scammer. On June 26, 2026, a healthcare business associate reported a server hacking incident to the US Department of Health and Human Services affecting over 1.2 million people. If the service were hacked tomorrow, what could someone reconstruct from what you uploaded today, and how fast could it be resold?
4. Your data can have a “second life.” While not every AI provider trains on every interaction, "not used for training" doesn't tell the whole story. Information may still be retained in logs, reviewed for safety, processed by contractors, included in analytics, or used for product improvement. There is also a direct resale market: data brokers actively buy and sell health information, and it is not theoretical. Researchers who contacted 37 data brokers about mental health data found 11 willing sellers offering lists of people with depression, anxiety, or PTSD medication history, inside a roughly $200 billion US industry with no dedicated federal law governing it. The FTC has warned AI companies that using customer data to train models contrary to their commitments may create legal liability. Once health information is copied across systems, or sold to a broker, deleting a conversation may not delete every downstream copy. You should ask: what is kept, who can review it, which processors receive it, is it sold to a broker, and does a deletion request reach all of them?
Simply removing a name is often not enough. A health note can identify someone through exact dates, locations, rare conditions, or family relationships. HHS guidance recognizes that even de-identified health information retains a small risk of re-identification, and its “Safe Harbor” method requires removing many categories beyond names. Remove too little, and you remain identifiable; remove too much, and the note loses its clinical value. Dates are particularly difficult because they are necessary for a clinical timeline but can also reveal identity.
This challenge inspired the first open-source tool I built: github.com/otmanm/health-data-anonymiser a local health-note anonymizer. The workflow was narrow by design: clean the note on your machine, review the result, and only then share the specific data needed. Later, I discovered OpenMed started by Maziyar Panahi, an advanced local-first healthcare AI platform, and contributed fixes for date shifting, format preservation, and safer offline workflows. The goal wasn’t to solve the entire problem with a single prototype, but to reduce the amount of sensitive data crossing unnecessary limits.
Before uploading health information, ask: What exact task is the system performing? Which identifiers are required? What stays on my device? How long is the data kept? Is it used for training or advertising? Can it be deleted completely? Convenience always has a price, especially when the data that makes a tool easy to use can later be turned against you. With cybersecurity incidents becoming more frequent, it is vital to think twice before sharing sensitive information with AI labs.
Use tools when they help, but provide only the minimum information required. The best way to protect your privacy is simple: don’t share anything you don’t absolutely need to.
If you are looking for a safer way to use healthcare AI, consider trying OpenMed. For those who prefer a non-technical setup, you can download the OpenMed app directly from the App Store or visit their website at openmed.life to get started with a local-first approach. Unlike cloud-based AI apps that store your data elsewhere, this solution makes sure your health information never leaves your device and stays safe.
Finally, if you are a builder and wanna improve your vibe coding abilities or join open-source projects based on useful but focused on solving meaningful human problems, you can join the community.
Sources
[1] Federal Trade Commission, “FTC Gives Final Approval to Order Banning BetterHelp from Sharing Sensitive Health Data for Advertising,” 2023. The FTC alleged that BetterHelp disclosed health-questionnaire data and other personal information to advertising platforms despite its privacy promises.
[2] European Commission, “Navigating the AI Act.” The Commission identifies AI used for risk assessment and pricing in life and health insurance among the Act’s high-risk use cases.
[3] US Department of Health and Human Services, Office for Civil Rights, Breach Portal. The portal lists an MCBS, LLC network-server hacking incident submitted on 26 June 2026, affecting 1,261,464 individuals.
[4] Federal Trade Commission, “AI Companies: Uphold Your Privacy and Confidentiality Commitments,” 2024. The FTC warns providers against using customer data for undisclosed purposes such as training or updating models contrary to their commitments.
[5] US Department of Health and Human Services, “Guidance Regarding Methods for De-identification of Protected Health Information.” HHS explains the Safe Harbor and Expert Determination methods and states that correctly de-identified data still carries a small, non-zero identification risk.
[6] Flare, “What is the Cost of Your Data on the Dark Web?” Health records are among the highest-value stolen data categories and typically appear on dark web markets within 24 to 48 hours of a breach.
[7] Duke Sanford Tech Policy program, study contacting 37 data brokers about mental health data; 11 of 26 responding firms were willing to sell it, in an unregulated US market estimated at $200 billion.