top of page

When AI is involved, you are the product.

9 hours ago
4 min read

A few weeks ago, one of the biggest text-and-video therapy platforms in the US held an investor call. Its CEO wasn't talking about client outcomes. He was talking about inventory.

The company has compiled roughly 8 billion words, 140 million messages, and 6.2 million assessments, pulled directly from actual sessions between real clients and real, licensed therapists on its platform. That corpus is being used to train an AI chatbot built to sit inside the therapeutic relationship. And the stated business goal isn't better care. It's getting insurance companies to reimburse the AI tool the same way they'd reimburse a session with you.


The sequence matters. Client says something vulnerable to a therapist. Platform logs it. Platform feeds it into a model. Model gets pitched to the people who write the cheques, not as a supplement to therapy, but as a cheaper way to skip it.


The Business Was Never the Session

Every therapy platform, everywhere, tells you the same story when it signs you up: we handle the admin, you focus on the client, everybody wins. Free scheduling. AI-assisted notes. Session summaries typed up before you've even logged off. It's positioned as a favour to you.


It isn't a favour. It's how the platform gets its raw material.



A chatbot that sounds like a therapist doesn't come from nowhere. It comes from exactly the kind of data an AI note-taker, an AI session-summariser, or an "AI-assisted" video platform sits closest to: unfiltered, real, high-stakes human disclosure, generated for free by clients who came to you because they trusted the room, and handed over by therapists who wanted their Friday evenings back.


The platform above was unusually honest about this on an investor call, because investors are the one audience a company has to be honest with. Everyone else, including the therapists on the platform and their clients, gets the softer version: "AI-assisted," "quality and training purposes," "helping us serve you better." Same mechanism. Different audience, different words.


It's Easy to File This Under "Not My Problem"

If you practise in Ahmedabad, Mumbai, Bangalore or Delhi, the instinct is to read this and think: we don't have insurance-driven therapy here, so this doesn't touch us.

That's true. It's also not the point.


What actually threatens you isn't the insurer waiting to swap you out. It's the data. And collecting the data doesn't need an insurance system in place. It just needs you, or the platform you use for notes, transcription or session support, to keep feeding a system that has no obligation to stay small, stay local, or stay out of your caseload.


India's Digital Personal Data Protection Act is still finding its feet on enforcement. Most Indian therapists have no idea which country a given "AI-assisted" tool actually processes their session data in, let alone what its owner's five-year plan is. A platform doesn't need to be big in India today to be training on Indian sessions today. It just needs an Indian therapist who signed up for the free tier.


We've written before about the specific trade platforms offer therapists here — more referrals, less admin, in exchange for session access. This is what that trade looks like once it scales: not a hypothetical, an actual investor call.


If AI Doesn't Belong in the Room, Why Is It There?

Most of us skip past this question, so it's worth sitting with: if you wouldn't let an AI conduct the session, why does it need to be present during the session at all?

There's a real difference between a tool that helps you after the fact, where you write your notes and run them through something to tighten the language, and a tool that's listening, live, while your client is talking. "AI-assisted notes" and "AI-assisted sessions" get marketed with the same three words, and most therapists using them couldn't tell you, off the top of their head, which one they've actually signed up for, or where that recording or transcript goes once the session ends.


So ask plainly: what exactly are you feeding into this system, and who benefits from it once it's in?


Ten minutes saved on writing up a session is a real, tangible relief at the end of a long day. It is not a fair trade against hours of someone's most guarded, most personal disclosures sitting on a server you've never seen, governed by terms of service you skimmed once, feeding a model whose entire purpose is to eventually do your job for less.


The Two Questions Worth Asking Before Your Next Session

What is this platform's most valuable asset? If the honest answer is "the data our therapists' clients generate," that tells you exactly whose interests come first when the platform makes a decision. Not yours. Not your client's.


Would you say it out loud to your client? Every therapist tells clients some version of "what you say in this room stays between us." That's not a formality, it's the entire foundation the work rests on. So the question is simple: before your next AI-assisted session, would you be willing to say, plainly, "this session is also being processed by an AI system," or does the platform's language let you avoid saying it at all?


If you wouldn't say it to their face, that's the answer to whether the consent you're relying on is real.


Clients don't read the terms of service on the tools their therapist uses. They can't; they don't even know those tools exist. That's not a loophole you get to use. It's exactly why the responsibility sits with you, not them.


Choosing better here doesn't mean rejecting every piece of technology in your practice. It means asking, before you adopt anything, what it actually collects, where that data goes, and what the company selling it to you is really building with it. Ten minutes of admin time was never worth trading that away.


Nobody is going to send you a notification when your client's words end up training the thing meant to replace you. That check has to be one you run yourself, before you sign up, not after you find out.

 
 
 

Comments


bottom of page