Last Week Amazon Unveiled its newest bit of hardware, a “personal robot” on wheels armed with a screen, cameras, and their Echo (aka Alexa) “smart” speaker built in. The $999 machine (for “early invitees,” $1,499 when fully released), says Gizmodo, “finally answers the question: ‘What if Wall-E were real, evil, and knew how to beatbox?’”
The best way to get invasive tech into your house is to make it cute, and seem useful. “What could go wrong?” Gizmodo asks.
My wife and I noticed that when we talked around our Alexa boxes at home, very often we would see online ads for what we talked about shortly after.
For example, we discussed getting a home security system. We did get one, but not from any of the ads that followed us around the Internet for days afterward. It wasn’t just one time, but example after example after example — things we only talked about, and had not searched for on our computers or phone.
We had three of the things, and I pulled the plug on all of them. Sure enough, the phenomenon ceased. Hm.
You might say I don’t trust Amazon, even though it has made life easier for rural dwellers like me. If you did say that, you’d be right.
Motherboard, the tech publication of Vice, also ran an article about the Amazon ’bot, dubbed “Astro” (though it looks nothing like The Jetson’s space mutt).“Astro is terrible and will almost certainly throw itself down a flight of stairs if presented the opportunity,” said one of the developers who, naturally, is kept anonymous.
“The person detection is unreliable at best, making the in-home security proposition laughable. … They’re also pushing it as an accessibility device,” the developer continued, but rather it is, “at best, absurdist nonsense and marketing and, at worst, potentially dangerous for anyone who’d actually rely on it for accessibility purposes.”
And with Alexa built in, you know that its security app, called “Sentry,” will be listening to everything you say. Motherboard got hold of some internal documents from the developers. “Sentry is required to investigate any unrecognized person detected by it or Audio Event in certain set of conditions are met [sic],” they quote one of the documents as saying. “When the person is identified as unknown or 30 [seconds has] passed, Sentry should start following the person,” it continues.
Yeah, an intruder wouldn’t notice a cheesy toy robot following them around the house and decide to use the firearm they brought along “just in case,” which sounds like a wonderful way to escalate an already dangerous situation.
Of course, for it to be able to recognize that someone in the house is “unknown,” you have to “register” the face and voice of everyone who normally has access to the house. And we can be sure that Amazon won’t abuse knowing who everyone is, and what each particular person says. Right…?
Let’s just say that there’s no mention of the Three Laws of Robotics. Hell, at least Roomba cleans the floor while it runs around the house! That’s useful.
To be fair, facial recognition is done “locally,” Amazon carefully points out in that blog post — not uploaded to their servers — as part of its “Privacy by design.” But I note they don’t say a word about the privacy of conversations in your house on that page. But you can trust their other, unlinked, policies. Right…?
But then there’s that quality problem again. “The source also corroborated that Astro’s facial recognition abilities perform poorly,” Motherboard says, “which is concerning for a device designed mainly to follow people around and determine if they’re a stranger or not.”
“In addition to consulting with several Amazon Scholars who specialize in computer vision,” Amazon said in a blog post in anticipation of criticism, “we also consulted with an external expert in algorithmic bias, Ayanna Howard, dean of the Ohio State University College of Engineering, to review the steps we took to enhance the fairness of the [visual identification] feature.”
It is well known that facial recognition systems are particularly apt to err when trying to recognize people of color. “U.S. government tests find even top-performing facial recognition systems misidentify blacks at rates five to 10 times higher than they do whites,” Wired magazine reported in 2019.
It hasn’t gotten better since then. Facial recognition “is more than just a gimmick,” wrote Harvard University in a blog post a year ago. “It is employed for law enforcement surveillance, airport passenger screening, and employment and housing decisions. Despite widespread adoption, face recognition was recently banned for use by police and local agencies in several cities, including Boston and San Francisco. Why? Of the dominant biometrics in use (fingerprint, iris, palm, voice, and face), face recognition is the least accurate and is rife with privacy concerns.”
Speaking of Evil, a month ago Facebook was embarrassed (if that’s actually possible) by a racist visual identification gaffe, Engadget (among others) reported.
I really hope A.I. becomes a useful tool for all of us, rather than just governments and giant corporations. But we’re definitely not anywhere close to that yet.
(Is it merely a coincidence that Kit asked last week if we could watch The Matrix*, which is all about A.I. run amok? We did in fact watch it — my first time since seeing it in the theater in 1999 — 22 years ago?! Time flies when you’re whiling away your time in your pod, but the film holds up well. Maybe entirely too well.)
So Think Carefully About whether you want the richest man in the world to take your hard-earned money from you in exchange for putting an “Evil Wall-E” in your house. There won’t be an Astro in my house, but someday if I do suddenly find one following me around here, I just might employ that firearm option….
July 2022 Update
“I sacrificed myself,” said Voice AI expert Sarah Andrew Wilson, by “falling” onto the floor in view of Astro and asking it for help. It’s not just “potentially dangerous for anyone who’d actually rely on it for accessibility purposes,” as a developer warned (see above), it’s criminally negligent in emergency situations:
Video #19 with #AmazonAstro. One use case highlighted by Amazon is that Astro can be helpful for people who live alone.
So what happens if someone is by themselves and falls? I sacrificed myself by falling 3 times right in front of Astro & asked for help. Astro was not helpful. pic.twitter.com/O2wDYgdNU7
— Sarah Andrew Wilson (@SarahAndrewWils) July 24, 2022
A phone would do better. Unfortunately, it cannot be programmed to follow you around.
* FTC Notice: If you buy products through links on this site I may receive an “affiliate” fee, which does not affect the price you pay. Details on True’s Privacy page.
– – –
Bad link? Broken image? Other problem on this page? Use the Help button lower right, and thanks.
This page is an example of my style of “Thought-Provoking Entertainment”. This is True is an email newsletter that uses “weird news” as a vehicle to explore the human condition in an entertaining way. If that sounds good, click here to open a subscribe form.
To really support This is True, you’re invited to sign up for a subscription to the much-expanded “Premium” edition:
Q: Why would I want to pay more than the minimum rate?
A: To support the publication to help it thrive and stay online: this kind of support means less future need for price increases (and smaller increases when they do happen), which enables more people to upgrade. This option was requested by existing Premium subscribers.