Google AI flagged parents’ accounts for potential abuse over nude photos of their sick kids

guest · Aug 23, 2022

The pictures were taken to help doctors diagnose infections
August 21, 2022

A concerned father says that after using his Android smartphone to take photos of an infection on his toddler’s groin, Google flagged the images as child sexual abuse material (CSAM), according to a report from The New York Times. The company closed his accounts and filed a report with the National Center for Missing and Exploited Children (NCMEC) and spurred a police investigation, highlighting the complications of trying to tell the difference between potential abuse and an innocent photo once it becomes part of a user’s digital library, whether on their personal device or in cloud storage.
Click to expand...

According to the NYT, Mark received a notification from Google just two days after taking the photos, stating that his accounts had been locked due to “harmful content” that was “a severe violation of Google’s policies and might be illegal.”

Like many internet companies, including Facebook, Twitter, and Reddit, Google has used hash matching with Microsoft’s PhotoDNA for scanning uploaded images to detect matches with known CSAM.
Click to expand...

A Google spokesperson told the Times that Google only scans users’ personal images when a user takes “affirmative action,” which can apparently include backing their pictures up to Google Photos.

Mark ended up losing access to his emails, contacts, photos, and even his phone number, as he used Google Fi’s mobile service, the Times reports. Mark immediately tried appealing Google's decision, but Google denied Mark’s request.
Click to expand...

While protecting children from abuse is undeniably important, critics argue that the practice of scanning a user’s photos unreasonably encroaches on their privacy. Jon Callas, a director of technology projects at the EFF called Google’s practices “intrusive” in a statement to the NYT.

“This is precisely the nightmare that we are all concerned about,” Callas told the NYT.
“They’re going to scan my family album, and then I’m going to get into trouble.”
Click to expand...

The New York Times: A Dad Took Photos of His Naked Toddler for the Doctor. Google Flagged Him as a Criminal.

Google has an automated tool to detect abusive images of children. But the system can get it wrong, and the consequences are serious.
Click to expand...

Log in or Sign up

Google AI flagged parents’ accounts for potential abuse over nude photos of their sick kids

guest Guest

Log in or Sign up

Google AI flagged parents’ accounts for potential abuse over nude photos of their sick kids

guest Guest

Useful Searches