Google Home
When one thinks of “digital assistants,” such as Amazon Echo and Google Assistant, one might automatically think of these devices as being used in private, in one’s home. However, these devices are also being promoted as useful in the classroom (e.g., see here and here and here).
Although one might view digital assistants as merely helpful, hands-off operators of one’s other digital devices, there’s more to the story of who, exactly, is being “helped.”
It seems that those digital assistants are also created to gather information on their users, for the benefit of the corporate world.
In December 2017, Consumer Watchdog posted this 15 page summary report of patents filed by Google and Amazon related to their digital assistants’ information-gathering capabilities. Consider the following, keeping in mind the privacy concerns (and associated liabilities for teachers and schools) that such capabilities pose in the classroom setting:
A new study reveals that Amazon and Google have filed patent applications for a number of technologies that would dramatically expand surveillance of consumers’ private lives. These patent applications show how technology companies use home data to draw disturbing inferences about households, and how the companies might use that data for financial gain. The technologies detailed in the patent applications include:
** A system for deriving sentiments and behaviors from ambient speech, even when a user has not addressed the device with its “wakeword.” In 2014, as Amazon was preparing to roll out its Echo smart speaker, the company patented an algorithm that could listen to human speech, including phone conversations, for statements of interest such as “I love skiing.” The algorithm would then process the statements into key words that could be used to target advertising. The patent describes transmitting the keywords to Amazon servers as text, which could allow the company to spy on conversations while technically keeping its promise to only store and analyze audio recordings that a user intends to share.
** Multiple systems for identifying speakers in a conversation and building interest profiles for each one. Both Google and Amazon offer users the option of creating acoustic “voice profiles” for voice-activated smart devices in their homes. These profiles can help the devices tailor services to the person speaking. Patents show that both Amazon and Google could also use voice profiles to associate behaviors with individual members of the household, in order to better target ads.
** A method for inferring users’ showering habits and targeting advertising based on that and other data. Dozens of patent applications for Google’s smart home devices detail scenarios in which Google may share data from smart home devices with third parties, including businesses, who can then use the data to make inferences about users’ sleeping, cooking, entertainment, and showering schedules. These inferences, Google says, “may help third-parties benefit consumers by providing them with interesting information, products and services as well as with providing them with targeted advertisements.”
** A system for recommending products based on furnishings observed by a smart home security camera. A particularly troubling patent application describes how Google could use video feeds from smart devices to determine users’ behaviors and characteristics, including “gender, age, fashion-taste, style, mood, known languages, preferred activities, and so forth.” The patent application describes smart devices that target advertisements based on the title of the book by a user’s bedside, the presence of a guitar or basketball in the room, and the face of a famous actor on a user’s t-shirt.
** A methodology for “inferring child mischief” using audio and movement sensors. Another Google patent describes a smart home system that monitors the activity of every member of a household and reports back to a designated “policy holder.” One application uses voice fingerprinting to identify the presence of children and motion sensors to detect activity, and reports inferred mischief when children are both active and quiet. This patent also mentions advertising applications.
** Systems for inserting paid content into the responses provided by digital assistants. Both Amazon and Google have patented methods for serving ads to users through their smart devices. An Amazon discusses a system of “intelligent sponsorship based on knowledge generated by reference to the human being entity using the system.” A Google patent shows how advertisers could bid to have the company’s digital assistant feature their products in its voice-based search results.
While Amazon and Google may be focused on the commercial implications of their inventions, these technologies also have troubling legal and ethical implications. Smart devices with cameras and microphones inevitably capture video or audio of people who did not consent to being recorded. Third parties can also access data from smart devices; a recent study found that hackers could easily access insecure wifi-enabled “smart” toys and talk to children. Law enforcement has also begin to subpoena data from smart devices in criminal investigations.
The 15-page Consumer Watchdog summary also includes some enlightening information on the background and capabilities of the Amazon Echo and Google Assistant and is well worth a full read. However, here is a final bit that caught my particular attention:
Many other patents for Google’s smart home products contain boilerplate describing an “advertising/communication paradigm […] that estimates characteristics (e.g., demographic information), desires and/or products of interest of a user based on device usage.”
One patent application, ironically named “Privacy-Aware Personalized Content for the Smart Home,” goes even further:
“In some embodiments, the robot may use its sensors to measure sounds and appearance of people and/or objects in the room and send audio and/or visual features that can be used to assess properties of the objects and/or people. For example, for people, the audio and/or visual features may be used to ascertain gender, age, fashion, taste, style, mood, known languages, preferred activities, and so forth.”
The application goes on to describe marketing sports camp to a 15-year-old boy holding a basketball in the living room, combining browser search history with an image on a user’s t-shirt to infer an interest in the actor Will Smith, and recommending a TV program based on the book on a user’s bedside table. …
Google’s smart home patent applications also describe the company’s interest in users’ health data. Several applications describe creating an audio signature for the master bathroom that will record the number of times that a toilet is flushed. Another application describes monitoring a user’s nighttime breathing patterns to detect a sleep disorder, and storing the information in a user’s “sleep profile.” While these patent applications do not describe systems for advertising to users based on this information, Google has demonstrated a willingness to advertise to users based on health data derived from other products. For example, a 2015 patent describes a scheme to advertise vacations to individuals whose heart rates indicate that they are under stress.
Google’s vast repository of information about its users has made the company the single biggest recipient of digital advertising revenues.
Information is power, and digital assistants are the power harnesses delivering constant, detailed, highly personal information into the hands of those who want it simply to make a buck.
And whereas it might seem neat to have a talking, computerized assistant overseeing your other electronics, even helping you in your classroom, keep in mind that the device is also monitoring you, gathering massive information about you, even in places where you believe your privacy is a given.
***
Originally posted 12-16-17 at deutsch29.wordpress.com.
***
