Literacy and Minority Report

You surely remember those amazing scenes in Minority Report where Tom Cruise manipulates several video streams, comparing them, replaying them, using hand gestures.

The company that created the interfaces seen in the movie is now releasing info about a similar commercial product. It is a fascinating opportunity to extend our ability to interact with computer data.

However, I have the vague fear that such non-verbal interfaces hold the false promise of liberating the human race from efforts, and in particular from the effort of learning languages.

This is obviously not a criticism of this specific (and awesome) project. It’s rather an observation about the mental state such projects are fostering, a certain type of magical thinking, where machines improve their understanding of human desire, to the point that we no longer need to verbalize it.

We believe that through technical progress, expressing our will shall eventually be replaced with merely manifesting it.

Actually, this may very well work out.

Better interfaces that support much more human context (in this case space, which the movie shows us can be used to manipulate time with some convenience), or provide it, but also brain scanners that enable machine to read thoughts (however crudely for the moment), the proliferation of sensors (GPS, camera, etc.) we carry around to supplement our memory with little effort, implanted chips that interact on our behalf with the environment, or network-enabled devices that allow querying massive databases of knowledge, anywhere, anytime.

In general, we’re searching for ways for computers to enhance our human abilities, and we’re creating and integrating technology that does just so. But at a cost.

The Kindle and iPad are asymmetrical devices that give rise to a class of consumers who obtain more abilities to consume content, but aren’t empowered for creation. In the collective expectation, users are excused from understanding the software they work with and from authoring it.

This reflects a fundamental trend in our thinking about tools in general, and computers in particular. We’ve been slowly moving out of the pre-industrial phase where each artisan had his own set of tools, extremely efficient but that could not be transferred easily to another worker. We’re “Taylorizing” software: rather poor, standard tools are made available, that can be used by anyone with little training, for a limited set of known uses.

The industrial revolution showed that the undeniable loss of individual productivity or quality is offset many times over by the massive cost advantage of legions of incompetent laborers who perform systematic tasks, following the blueprint of one competent designer.

Moving away from a situation where all computer users were programmers, the gap is now widening between content consumers and producers, as well as between content producers (who are software users) and software producers. And only software producers are left dealing with language.

The notion of “computer language” is a convenient anthropomorphic metaphor, but not a very accurate one. For example, software programs don’t exchange information (“talk to each other”) using computer languages. Computer languages are a way for people to order computers around (a very limited part of what human languages can be used for). I’m not talking here about the command of computer languages, I’m talking about human languages, about literacy.

Indeed, language is at the heart of computing (possibly more so than mathematics), in that it is a conventional, structured, complex way of expressing thoughts. Our relationship with computers has, so far, been based on projecting our desires in a conventional manner. We don’t emit raw thoughts, we express them in a pre-defined way that has been agreed with the machine (or with its makers) to have the expected effect.

Increasingly sophisticated interface devices (more or less metaphorical, such as windows, pointers, the trash can, drawers, menus, etc.) are slowly lowering the entry barrier, by making extremely sophisticated computing tasks accessible to people with little technical knowledge.

But this movement is possible only because the computing tasks that people undertake have been identified. In Minority Report, Cruise interacts with an extremely specialized system, that has been designed to deal with three video feeds that must be viewed, analyzed, understood — all of that extremely quickly. Someone designed that system. Actually, someone spent an awful lot of (fictional) time developing and then tweaking the interface, to ensure it best supports Cruise’s tasks. (I seem to remember the system designer was Cruise himself, in the movie — in an artisan role!)

In fact, in that movie, someone designed a system which supports a non-verbal approach to reality. The minds of the three psychics are mined by a computer, and yield video streams. Language isn’t part of it. Then Cruise navigates the streams using gestures. Not language (unlike Harrison Ford in Blade Runner, talking to his image processor).

Symbolically, the big innovation of this entire Minority Report software system is that it does away with language: with expressing one’s will through words, as well as with expressing one’s perception in that way. Words are implicitly shown as not worthy of trust, as a superfluous layer of potential conflict and inaccuracy, as a hurdle in the process of taking in reality.

Aside from the chilling human and social pitfalls this entails (some of which are the point of the movie, some of which are ignored), this run from language also highlights the creation of a class of workers who will no longer need language to function. And right now, software development is similarly hell-bent on freeing computer users from the requirement of any level of command of a language.

This is an interesting conundrum. On the one hand, I strongly support software development that enables users to do more, especially users with little technical knowledge. Usability is a very democratic concern. On the other hand, pushing the logic too far, we’ll make modern life increasingly easier for illiterates, thereby reducing the need to be literate, thereby perhaps increasing the number of illiterate people, which is not quite the intention.

Literacy isn’t only used for communication — actually, language is primarily a thought process. The ability to shape thoughts is closely related to the ability to express them (in most cases, with the notable exception of art, which can manifest itself in many non-linguistic ways).

In our modern society that relies so heavily on computer systems, we excuse the majority of the population from the requirement of literacy. The creators of software, on the other hand, are in charge of building tools for everyone else. In turn, such tools largely define the way people relate with the world.

It strikes me that computers, more than any other human device save for books, are borne from language. Rather than an artisan’s tool of enlightenment and human improvement, are computers, unwittingly, being industrialized into a sorcerer’s tool of enslavement?