data security – ��

Data Privacy Advocates Raise Alarm Over NYC’s Free Teen Teletherapy Program

Michael Elsen-Rooney — Thu, 12 Sep 2024 12:30:00 +0000

This article was originally published in

New York City’s free online therapy platform for teens may violate state and federal laws protecting student data privacy, lawyers from the New York Civil Liberties Union and advocates charged in a letter Tuesday to the city’s Education and Health Departments.

, a $26 million partnership between the city Health Department and teletherapy giant Talkspace launched in late 2023, connects city residents between ages 13 and 17 with free therapists by text, phone, or video chat.

In less than a year, roughly 16,000 students have signed up, Health Department officials said. Sign-ups disproportionately came from youth who identified as Black, Latino, Asian American and female and live in some of the city’s lowest-income neighborhoods, .

Get stories like this delivered straight to your inbox. Sign up for �� Newsletter

Information shared with a therapist is subject to stringent protections under the federal Health Insurance Portability and Accountability Act, or HIPAA. But before connecting with a therapist through Teenspace, teens go through a registration process that asks for personal information like their name, school, mental health history, and gender identity. Advocates are concerned such information is being improperly collected and could be misused.

For one, teens enter the registration information before securing parental consent – a possible violation of federal student privacy laws, the letter contends.

And families don’t get a chance to review the privacy policy – which discloses that registration information can be used to “tailor advertising” and for marketing purposes – before entering the registration information, advocates allege. There’s an option for teens to request that their data be deleted from the company’s platform, but it’s hard to find, according to advocates.

“It’s all very invasive,” said Shannon Edwards, a parent and founder of AI For Families, an organization that seeks to help families navigate artificial intelligence, who co-authored the letter along with NYCLU and the Parent Coalition for Student Privacy. “It’s also very unclear that parents understand what they’re getting themselves into.”

Advocates also pointed to the risk of a potential data breach – something the city has in recent years.

Advocates say similar about have been circulating for years and questioned whether city officials did sufficient due diligence or built in enough additional privacy safeguards before inking the contract.

“It’s the opacity of the relationship here, and the failure to make manifest what the city is doing to ensure there isn’t this data accumulation and sharing for inappropriate purposes,” said Beth Haroules, a senior attorney at the NYCLU who co-authored the letter.

Health Department spokesperson Rachel Vick said the agency has “taken additional steps to protect the data of Teenspace users and ensure information is not collected for personal gain, including stipulations that require all client data to remain confidential during and after the completion of the city’s contract and barring use of data for any purpose other than providing the services included in the contract.”

Client data is destroyed after 30 days if a teen doesn’t connect with a therapist, officials said.

A spokesperson for Talkspace referred questions to the Health Department.

The extent to which Teenspace is subject to state and federal laws governing student privacy in educational settings is somewhat murky, given that the contract is with the city’s Health Department, not its Education Department.

But NYCLU attorneys contend “the City cannot absolve itself of its responsibility to provide the protections inherent in federal and state laws…simply because the contract sits with DOHMH instead of DOE. The service is promoted on public school websites, and it is DOE’s responsibility to ensure that student data is protected, regardless of which City agency signs the contract.”

Parents may be more inclined to trust the platform because it has a “stamp of approval” from the school system, Edwards added.

A Health Department spokesperson didn’t specify whether the program is subject to education privacy laws, but said it’s “not a school based service.”

Teenspace has been the city’s highest-profile effort to address the ongoing youth mental health crisis.

“We are meeting people where they are with a front door to the mental health system that for too long has been too hard to find,” said Ashwin Vasan, the city’s health commissioner, in May.

Some teens have praised the program, noting it’s a way to bring mental health care to young people who may not otherwise have access.

But some mental health providers have argued it can’t replace the kind of intensive care a clinician provides, especially for kids with severe mental health challenges.

Company officials shared in May that they had helped 36 teens navigate serious incidents including reports of suicide attempts and abuse – cases they referred to child protective services, in-person therapists, or hospitals.

Talkspace CEO Jon Cohen previously told Chalkbeat the company uses an artificial intelligence algorithm to scan transcripts of therapy sessions to help identify teens at risk of suicide.

Even advocates critical of Teenspace’s privacy protections acknowledge the severe shortage of mental health providers and say teletherapy can play a role in filling the gap.

“We know you cannot find providers … there is such a need,” said Haroules. But advocates said the city can do more to ensure its vendors are meeting strict standards for data privacy, especially with such sensitive information.

“Everyone thinks, well, mental health is important for kids, these kids of services are required … when on the other side is: ‘How are they getting to it?’” said Edwards. “It doesn’t matter what the app is, there has to be a standard.”

This was originally published by Chalkbeat. Chalkbeat is a nonprofit news site covering educational change in public schools. Sign up for their newsletters at .

L.A. Schools Probe Charges its Hyped, Now-Defunct AI Chatbot Misused Student Data

Mark Keierleber — Wed, 10 Jul 2024 10:30:00 +0000

Independent Los Angeles school district investigators have opened an inquiry into claims that its $6 million AI chatbot — an animated sun named “Ed” celebrated as an unprecedented learning acceleration tool until the company that built it collapsed and the district was forced to pull the plug — put students’ personal information in peril.

Investigators with the Los Angeles Unified School District’s inspector general’s office conducted a video interview with Chris Whiteley, the former senior director of software engineering at AllHere, after he told �� his former employer’s student data security practices violated both industry standards and the district’s own policies.

Whiteley told �� he had alerted the school district, the IG’s office and state education officials earlier to the data privacy problems with Ed but got no response. His meeting with investigators occurred July 2, one day after �� published its story outlining Whiteley’s allegations, including that the chatbot put students’ personally identifiable information at risk of getting hacked by including it in all chatbot prompts, even in those where the data weren’t relevant; sharing it with other third-party companies unnecessarily and processing prompts on offshore servers in violation of district student privacy rules.

Get stories like this delivered straight to your inbox. Sign up for �� Newsletter

In an interview with �� this week, Whiteley said the officials from the district’s inspector general’s office “were definitely interested in what I had to say,” as speculation swirls about the future of Ed, its ed tech creator AllHere and broader education investments in artificial intelligence.

“It felt like they were after the truth,” Whiteley said, adding, “I’m certain that they were surprised about how bad [students’ personal information] was being handled.”

To generate responses to even mundane prompts, Whiteley said, the chatbot processed the personal information for all students in a household. If a mother with 10 children asked the chatbot a question about her youngest son’s class schedule, for example, the tool processed data about all of her children to generate a response.

“It’s just sad and crazy,” he said.

The inspector general’s office directed ��’s request for comment to a district spokesperson, who declined to comment or respond to questions involving the inquiry.

While the conversation centered primarily on technical aspects related to the company’s data security protocols, Whiteley said investigators probed him on his personal experiences with AllHere, which he described as being abusive, and its finances.

Whiteley was laid off from AllHere in April. Two months later, a notice posted to the said a majority of its 50 or so employees had been furloughed due to its “current financial position” and the LAUSD spokesperson said company co-founder and CEO Joanna Smith-Griffin had left. The former Boston teacher and Harvard graduate was successful in raising $12 million in venture capital for AllHere and appeared with L.A. schools Superintendent Alberto Carvalho at ed tech conferences and other events throughout the spring touting the heavily publicized AI tool they partnered to create.

Just weeks ago, Carvalho spoke publicly about how the project had put L.A. out in front as school districts and ed tech companies nationally race to follow the lead of generative artificial intelligence pioneers like ChatGPT. But the school chief’s superlative language around what Ed could do on an individualized basis with 540,000 students had some industry observers and AI experts speculating it was destined to fail.

The chatbot was supposed to serve as a “friendly, concise customer support agent” that replied “using simple language a third grader could understand” to help students and parents supplement classroom instruction, find assistance with kids’ academic struggles and navigate attendance, grades, transportation and other key issues. What they were given, Whiteley charges, was a student privacy nightmare.

Smith-Griffin recently deactivated her LinkedIn page and has not surfaced since her company went into apparent free fall. Attempts to reach AllHere for comment were unsuccessful and parts of the company website have gone dark. LAUSD said earlier that AllHere is for sale and that several companies are interested in acquiring it.

The district has already paid AllHere $3 million to build the chatbot and “a fully-integrated portal” that gave students and parents access to information and resources in a single location, the district spokesperson said in a statement Tuesday, and “was surprised by the financial disruption to AllHere.”

AllHere’s collapse represents a stunning fall from grace for a company that was named among the world’s top education technology companies by Time Magazine just months earlier. Scrutiny of AllHere intensified when Whiteley became a whistleblower. He said he turned to the press because his concerns, which he shared first with AllHere executives and the school district, had been ignored.

Whitely shared source code with �� which showed that students’ information had been processed on offshore servers. Seven out of eight Ed chatbot requests, he said, were sent to places like Japan, Sweden, the United Kingdom, France, Switzerland, Australia and Canada.

‘How are smaller districts going to do this?’

What district leaders failed to do as they heralded their new tool, Whiteley said, is conduct sufficient audits. As L.A. — and school systems nationwide — contract with a laundry list of tech vendors, he said it’s imperative that they understand how third-party companies use students’ information.

“If the second-biggest district can’t audit their [personally identifiable information] on new or interesting products and can’t do security audits on external sources, how are smaller districts going to do this?” he asked.

Over the last several weeks, the district’s official position on Ed has appeared to shift. In late June when the district spokesperson said that several companies were “interested in acquiring Allhere,” they also said its predecessor would “continue to provide this first-of-its-kind resource to our students and families.” In its initial response to Whiteley’s allegations published July 1, the spokesperson said that education officials would “take any steps necessary to ensure that appropriate privacy and security protections are in place in the Ed platform.”

In in the Los Angeles Times, a district spokesperson said the chatbot had been unplugged on June 14. �� asked the spokesperson to provide documentation showing the tool was disabled last month but didn’t get a response.

Even after June 14, Carvalho continued to boast publicly about LAUSD’s foray into generative AI and what he described with third-party vendors.

On Tuesday, the district spokesperson told �� that the online portal — even without a chatty, animated sun — “will continue regardless of the outcome with AllHere.” In fact, the project could become a source of district revenue. Under the contract between AllHere and LAUSD, which was obtained by ��, the chatbot is the property of the school district, which was set to receive 2% in royalty payments from AllHere “should other school districts seek to use the tool to benefit their families and students.”

In the statement Tuesday, the district spokesperson said that officials chose to “temporarily disable the chatbot” amid AllHere’s uncertainty and that it would “only be restored when the human-in-the-loop aspect is re-established.”

Whiteley agreed that the district could maintain the student information dashboard without the chatbot and, similarly, that another firm could buy what remains of AllHere. He was skeptical, however, that Ed the chatbot would live another day because “it’s broken”