The Machine as a Mirror

The Upside of the Digital Panopticon

“It’s disgraceful. You should be ashamed. She’s still in High School.”

The father was not happy. His daughter had only recently acquired a loyalty card and already it was sending her vouchers for nappies, bottles and the other necessities for those within months of increasing the world’s population.

The manager apologised profusely and promised to look into it. The loyalty programme was run by a computer and well, sometimes it gets things wrong. He was terribly sorry for any offence caused.

A couple of weeks later, the father returned, but it was now his time to apologise. His daughter really was, he had discovered, about to increase the world’s population. The loyalty card had just worked it out before she did.

This is not a new story. I read it in a book over a decade ago. But, in another way, it is a new story.

For the machines which ran the loyalty programme had analysed billions of transactions and found a pattern. People who bought a certain combination of goods reliably went on to buy another combination of goods. Designed to persuade consumers to spend as much of their money in their shop, the machines took steps to nudge the humans to make these next purchases in their store. They didn’t know the daughter was pregnant, but knew that, given her earlier purchases, she was likely to want nappies, bottles etc. in the near future, and set out to ensure that she bought them from their outlet.

We have no reason to believe that humans have suddenly become more predictable. Rather, we now have enough data to allow us to identify the patterns which enable us to predict them. Most of us have no problem with this. We evolved in a resource-constrained environment. Free stuff, to our caveman brains, is good stuff.

But we don’t think much about the implications. We think of ourselves as the masters, to an extent, of our destinies. We make rational choices after careful consideration. The message of loyalty programmes is that we are not. Our behaviour often follows patterns of which we are unaware, but which machines can detect. If you do b after doing a, just like everybody else, are you exercising your Free Will, or are you just using an autopilot you didn’t know you had?

If you’re not the customer, you’re the product, they say and nowhere is that truer than in social media. We go to the shops a few times a week. We spend hours a day scrolling Facebook and the like. We go to specific shops for specific goods – you don’t book your singles holiday in Tesco. Yet. We put our whole lives on our socials. These companies have information on us retailers could only dream about.

But “a lot” is seldom “enough”. Social media companies want to keep you on their social media sites because the more time you spend, the more likely you are to click an ad which means they get paid. They are companies not charities. To ensure your attention is “sticky”, they need to show you content you want to attend to.

Much as we might like to think of ourselves as reasonably constant, unchanging entities, we are not. Things happen. The daughter in the story above probably had little interest in childcare videos until she fell pregnant. We might accidentally discover a new genre of music we really like. Our interests change and, if it wants to keep us engaged, our social media feed has to change too.

The sites monitor the content we engage with so they can show us more of it. Their tracking cookies report back on our off-socials behaviour so they can update our profiles with information we have yet to share. Regular readers (there must be some) may remember my adventures last year learning Italian with A.I.. It was only a few weeks into the project that my frequent trips to Google Translate tipped Facebook off that something was up. And from that point on, my feed was flooded with “Learn Italian” posts. For which I was very grateful.

The point of learning Italian, however, was not to learn Italian for the rest of time. There was a hope that some improvement might be seen. That is generally how learning works. We level up.

That being the case, it would not make sense for Facebook to keep on showing me the same type of posts. It would be the equivalent of giving me, a reasonably literate adult, an “early reader” book. I would get bored and wander off. And take my eyeballs with me.

It was a few weeks ago that I first noticed the change. I was shown a video of a cycling race, with Italian commentary. Facebook knows I like cycling. It knew I was learning Italian. Why not? The problem is that sports commentary is fast. Very fast. It needs to be. Highlights reels show the bits where things are happening, when there is a lot to be said and not much time to say it. It was a bit beyond me. The number of words I was understanding did not entirely match the number of words I was hearing.

No matter. Perhaps reading was more my thing. I started seeing the sort of posts Facebook likes to show me in English but in Italian. Little poems. Inspirational quotes. Influencer chat. I learned that the “10 Life Lessons” a Japanese monk offers a Sicilian are exactly the same as those he gives a Scotsman. The more I read, the more I was shown. Until, eventually, over half of my feed was in Italian to the extent I was starting to miss the English language stuff I had become used to. If we worry about humans becoming addicted to social media, perhaps we should also worry about the machines being addicted to supplying us, like a needy teenager throwing presents at a girlfriend to keep her onside.

The initial Italian posts had been an experiment. An attempt to find a way to retain my interest. And they were good experiments. Low cost – just stick a couple of posts into my feed. High return – I showed interest, so Facebook had another way to keep me engaged. And, like any good experiment, if it works, it can be tweaked and be repeated. My profile notes that I went to school in Greece. Maybe I speak Greek? A couple of posts will find out. Knowing one Romance language might mean I know others. Throw some French into the mix. What about Spanish? Let’s have some of that too. It wasn’t long before my feed started to look like a meeting of the European Commission. (There was some Latin too – presumably the Vatican was in attendance).

But there is a wrinkle. In allowing me to while away my hours watching Italian gym-bunnies telling their audiences how to avoid age-related physical deterioration, Facebook is treating me like an Italian-speaker. And I do not see myself as an Italian-speaker.

In part this is due to the way I learned – having focused on being able to read historical fiction, I know there are spazi vuoti on my mappa. I have no formal qualification in the language – no-one else has told me I speak Italian. And I tend to take an “aut Caesar, aut nihil” (“Either Caesar or nothing”) approach to things – there is total victory and shades of failure. Until I can hold a discussion on quantum physics for an hour (a bar I might not clear in English), I am reluctant to declare myself fluent.

But Facebook was showing me Italian slop and I was consuming it just as eagerly as I consumed its English slop. It was treating me as an Italian-speaker because I was displaying the behaviour of an Italian speaker. I fitted the pattern. Whatever story I was telling myself, it was looking at my actions, and seeing me as I really am. And as I am increasingly becoming – my vocabulary for joint-friendly, low impact exercises having come on by leaps and bounds. Just as the girl with the loyalty card was pregnant when she received the vouchers – she just didn’t think she was.

We like to think of social media as a place for display. We curate our lives and put the best bits on our feeds, seeking to gain status and inspire jealousy. But it is also a place of revelation. A tool which responds to our actions, not our images, and reflects them back at us. A tool which sees us as we are rather than as we think we are. There are countless reasons why we fail to update our self-stories – humility, distraction to name but two – but the machine experiences none of them. It just sits there, watching, and shows us what it sees. It does not “know” us, but it knows us better than we do ourselves.

There this piece would have ended. I decided to let it marinate overnight before posting it. I started to flick through Facebook. I saw some videos. Many were in Italian. One was about quantum physics. Chance? Possibly. Does Facebook want me to think of myself as an Italian-speaker? Who knows. I watched it anyway.

Do I speak Italian? Si. Un po’.