Here’s a risk most AI deployment plans never account for: the system works exactly as designed, and care gets worse anyway.
Seven current and former Kaiser Permanente nurses told CalMatters that workplace surveillance is eroding their ability to care for patients, according to a report picked up by Hacker News this week. Nurses who spend more than 15 minutes on a call with a patient routinely face criticism from management or get pulled into performance evaluation meetings. Call time factors into their monthly performance scores. Kaiser also runs software that predicts, daily, whether a nurse is being unproductive or too slow to pick up. And AI systems have been used to rate their empathy and tone of voice.
Read that last part again. An algorithm scoring a human being on how much compassion she showed a dying woman.
What’s actually at stake
The timing matters. The California Nurses Association started negotiating a new contract with Kaiser this month, and AI is expected to be a central issue. The union is bargaining for 25,000 nurses, roughly 1,000 of them in call centers. Kaiser nurses already struck against AI for a day in March and picketed last fall.
California lawmakers are moving too. Several bills are in play, including one that would shield doctors and nurses from retaliation when they override automated care recommendations. That bill tells you where the fight is heading: the question isn’t whether AI gets used in healthcare, it’s who gets punished when a human disagrees with it.
Kaiser pushes back hard. A spokesperson said the company “does not use Average Handle Time to assess agent performance or enforce call time metrics,” and that contact center tools “support our quality assurance efforts and have human review and oversight.” Spokesperson Vincent Staupe added that Kaiser deploys AI responsibly while “prioritizing patient safety, privacy, and equity,” but declined to share specifics about internal systems.
Why this sets a precedent
Kaiser is the largest private employer in California. It serves more than 9 million people in the state and 3 million more elsewhere. Whatever it normalizes becomes the template.
Two precedents are being set at once:
- Algorithmic management of professionals. Not warehouse pickers or delivery drivers. Licensed clinicians whose judgment is the whole point of the job.
- The automation-versus-presence tradeoff in healthcare. How much human touch does a health system give up to cut costs, and who decides?
The nurses’ stories make the cost concrete. Raquel Alvarez Sanchez, an advice nurse since 2010, described a call with a suicidal patient that ran over an hour because she had to wait for police to arrive. She stayed on the line knowing it would drag her average handle time down for weeks. “I think at some point all of the nurses have been talked to about their average handle time,” she said. “The only thing I can think of is they’re doing it for profit.”
Another nurse, speaking anonymously out of fear of retribution, took a call from an elderly woman who’d just received a terminal cancer diagnosis. The woman wasn’t suicidal. She was in shock and needed to talk. The nurse wanted to comfort her, then stopped herself. “I had to ask myself: Am I going to get disciplined for going off script or saying more than what is necessary?”
The measurement trap
What stands out is the failure mode. Nobody at Kaiser wrote a rule saying “be less compassionate.” The system just measured what was easy to measure, and nurses adapted. A 2024 records request found no patient complaints against Kaiser over call times, but that proves little. These nurses lose contact with patients the moment they hang up. The harm, if it exists, happens somewhere they can’t see.
If you’re deploying AI to monitor knowledge workers, the lesson is blunt. Optimizing a proxy metric doesn’t just miss the real goal, it actively teaches people to sacrifice it. Consumer Watchdog advocate Michele Ramos put the broader pattern this way: “Kaiser’s been known through the years to manage dollars over managing care, which is only going to fail patients.”
Watch the CNA contract talks and the California bills. Both will shape how far employers can push algorithmic oversight into professional judgment, and other sectors will copy whatever survives. Full details are at the original source.