Uncanny Disruption When tech firms simulate humans, it gets creepy fast

摘要

I recently returned from a vacation to find that Google's algorithms had created a customized slide show of my trip. I hadn't asked for one. But the company's software robots apparently noticed I'd traveled somewhere and taken a flurry of photos, which likely indicated I'd been vacationing. Now, I actually enjoy some of Google's simpler customization tools, like autocomplete. But this unbidden slide-show curation seemed too humanlike. The machine had anticipated desires I didn't have yet. I actually yelped when I saw it. 1 Recent experiments suggest that I'm not alone in my discomfort. Colin Strong, a marketing consultant in the UK, storyboarded several high tech customization scenarios, ranging from the simple (targeted direct mail) to the sophisticated, like health insurance companies crawling info on your food purchase habits to adjust your premiums. When he showed the scenarios to subjects, he found that the more personalized the services got, the more people liked them-until they got too personalized. Then they seemed freaky.