A San Francisco stay-at-home dad used his Android phone to take pictures of his toddler son's inflamed penis to send to a doctor—and ended up in the middle of a nightmare as a result. Two days after taking the photos in February 2021, the man, identified only by his first name, Mark, was alerted that his Google account had been disabled due to "harmful content" that may be illegal. The photos he takes on his phone automatically back up to the Google cloud, and Mark realized the photos of his son's groin area may have been flagged automatically, reports the New York Times in an extensive look at the case. Indeed, he ultimately found out Google had flagged the photos as child sexual abuse material (CSAM) and reported it to the police, who launched an investigation.
Police ultimately cleared Mark after looking at all the information Google handed over, including his internet searches, location history, messages, documents, photos, and videos. Even with the police report clearing Mark on hand, however, Google refused to return Mark's access his account—and, thus, his access to not just that list of things but also more than a decade's worth of contacts, his email account, and the phone number he had through Google Fi. The Times reports on a similar case out of Texas, in which the father was cleared by police but never got his Google account back. Experts who spoke to both the Times and the Guardian say these cases highlight longstanding concerns about the problems of such automated content-flagging systems, and tech giants acting as "proxies" for law enforcement. See the Times' full piece here. (Read more Google stories.)