Big Brother Can Watch You- From Your Pocket

Finding that (Sherlock) Holmes was too absorbed for conversation, I had tossed aside the barren paper, and, leaning back in my chair I fell into a brown study. Suddenly, my companion’s voice broke in upon my thoughts. 

“You are right, Watson,” said he. “It does seem a very preposterous way of settling a dispute.”

 “Most preposterous!” I exclaimed, and then, suddenly realizing how he had echoed the inmost thought of my soul, I sat up in my chair and stared at him in amazement.

 “What is this, Holmes?” I cried. “This is beyond anything I could have imagined.”

 He laughed heartily at my perplexity.

The Resident Patient – The Memoirs of Sherlock Holmes by Sir Arthur Conan Doyle.

In the above story, Dr. Watson is rendered speechless by Holmes’ almost eerie ability to read his mind purely through his extraordinary powers of observation.  Imagine, then, this power magnified hundreds of times, and then linked to a supra-normal intelligence. Such a being would be able to reach to our innermost self, our most private thoughts and emotions, and lay us naked before its gaze. It almost seems too sci-fi to contemplate, yet this is the reality that is taking shape even as this article is being written.

It all started in the 1970s when a psychologist called Paul Ekman catalogued over 5000 muscle movements of the human face and indexed them to underlying emotions. Called the Facial Action Coding System (FACS), it has today been picked up by an array of companies who have extended the scope of the original study several-fold by using advanced computer generated algorithms. These split facial muscle movements into their smallest individual component called micro-expressions. These usually flit across the face in a fraction of a second, but that is all that’s needed for a hi-res high speed video camera to instantly feed into its system and accurately reflect the underlying feelings that they represent.

Companies like Emotient, Eyeris and Affectiva are building on emotion recognition (ER) technology for an array of applications. Imagine you are driving your car. The dashboard has also an embedded video cam that’s scanning your micro-expression in real time. Now, the interesting thing is that you can be from any country, male or female, of any age. The company that’s supplied the emotion reader has on it database billions of expressions codified from almost every major nationality on the planet and reflects your specific cultural make-up. Basis its reading of your emotional state at any point in time, it can adjust the music that floats from your speakers, or perhaps the type of podcast you wish to listen to. If it senses you’re anxious about someone in your family and this anxiety is peaking, it can even trigger a call to that person. And, in one of the important features that the car manufacturer will highlight, it will instantly detect when you’re fatigued and about to fall asleep at the wheel and warn you – and even slow the car down on its own.

ER cuts to the very heart of our sense of what it means to have a private life. As with all these intrusive tools, it is the benefits that get touted first by the ER companies. The safety feature it imparts to the driver of a car is clearly one such. Patients with severe physical disabilities including mental illness that prevent normal expression can be gauged much better with a properly designed FACS. Psychologists and therapists can possibly understand their patients inner workings better using ER. In a somewhat grey area, it is being applied to gauging customer reaction to products. Sometimes this is done with consent and in a controlled test environment, but it is very possible for these video-cams to be mounted in stores to scan the emotions of every shopper without their being aware of it. In turn, this data can, and is, already being applied to elements of store and product design.   

As with all such technologies, the greatest potential applications appear to lie in the realm of law enforcement. ER is already being seen as an effective interrogation tool and The Wall Street Journal has reported its use in the Abu Ghraib prison in Iraq. Crowd emotion can be mapped by facially reading each participant’s face and feeding the data into the police network which is linked seamlessly to police officers’ smartphones in real time – allowing local cops to calibrate their responses accordingly. ER firm Eyeris confirms that it has tied up with law enforcement authorities for unspecified applications.

With privacy laws barely covering the ER domain, it appears that these companies operate in a sort of unregulated twilight zone, making up the rules as they go along. Companies such as Affectiva claim to delete all ER data on individuals post capture, but there is nothing in the law in most countries that mandates this and the logic of counter-terrorism and profiling weigh strongly against privacy concerns.

We live in a world where our real persona is increasingly mirrored in its digital counterpart. Our personal smartphones already capture very many important aspects of our physical life – where we travel, what we buy, what we see and hear, and who we call. Our digital self is now our shadow, cast over the screen that is the internet. With emotion recognition generating increasing interest with companies such as Facebook and Google, the time is not far away that when you look at your mobile phone it is not just looking right back at you, but looking into you.

Those who find this hard to believe, reflect on this: Paul Ekman, the inventor of FACS has gone on record to say that he has definite concerns about how this entire capability could be seriously misused if the law does not apply constraints upon it. However, he remains a remains a consultant to Emotient, and towards end 2015/early 2016, Apple acquired Emotient.

In what is possibly the greatest dystopian novel of a totalitarian world ever written – 1984 by George Orwell – one of the characters says, “If you want to keep a secret, you must also hide it from yourself.”

That world may be much closer than we think.

(Aroon Raman is a Research and Innovation entrepreneur and the author of the national bestsellers The Shadow Throne and The Treasure of Kafur. His latest thriller, Skyfire, has been nominated as one of Amazon’s must-read picks of 2016. He can be reached at www.aroonraman.com)