Superhuman AI may be the next Pied Piper of Hamelin for our kids

Without getting into philosophy, let us also accept the prediction that this ‘intelligence’ would enable behaviours and actions which in humans arise from capacities that we consider particularly human and often base our decisions on, such as empathy and moral-ethical judgement. Let’s call this SHAI—short for Super-Human AI. Let us examine the use of SHAI in education.

SHAI can play four kinds of roles in education. As an assistant to the student. As an assistant to the teacher. As a teacher for the student. And in assisting education administration (which we will not touch upon here).

Even before AI reaches super-human levels, it must be used in education only with extreme caution. The principle should be: ‘Use last in education and only once we are fully certain it has no risk.’ There are four intertwined reasons to adopt this approach. The force of the four varies for each of the roles; and this applies to learning in both children and adults.

Hollowing: With SHAI as an assistant, neither teachers nor students need to think. If you stop using a capacity, you lose it (or it fails to develop). 

Given what SHAI would be, its ‘thinking’ would extend to crucial human capacities like empathy and moral judgement. With SHAI in education, cognitive hollowing is inevitable; and perhaps also the hollowing of almost all human capacities.

Form factor: This is a ‘techie’ phrase for what kind of physical form one needs something in—boxy, small, soft and so on. AI that interacts with us through a phone has a form factor that is very different from that of an AI robot which can kill. 

Our desires from education and the processes of learning are such that a phone or computer interface form factor will sharply limit SHAI in education, given the nature of human learning.

Effective learning requires understanding the student’s ‘then’ state of mind and body, capturing her attention and guiding her carefully through steps that constitute learning. 

And this must be done with a group of students, as we want children to acquire social, emotional and moral capacities, all of which can only be learnt in a social setting involving interactions with other people.

All of this implies that SHAI must operate in a setting much like the current classroom: with a group of students. SHAI trapped inside a computer box or screen will have severe limitations. What’s needed is SHAI in robotic form. 

Some sages of AI have only half-jokingly mentioned a plumber’s job as the safest from the AI onslaught, as it requires a near-humanoid form factor along with high multidimensional intelligence. They could just as well have spoken of a teacher’s job as the last frontier—because it requires SHAI residing in a humanoid form factor.

Control: Social science uses the word ‘agency,’ but ‘control’ better describes this phenomenon. If SHAI is indeed SHAI, do we want to give it control over our individual and collective lives and destiny in the immediate world and for eternity? 

Inevitably, SHAI will exercise control, whether as our teachers or in the guise of assistants. Should we take an irreversible leap of faith into this unknowable unknown? Remember, what controls education controls us.

Amplification: This is the most here-and-now reason. Technology amplifies human frailties, addictions and pathologies. It hardens our inequalities: of class, among nations and of social categories such as ethnicity and caste. 

The best and most useful stuff is always in the hands of the rich, while the poor get tokenism to carry them along and maintain social and economic order. Other amplifications, such as those of addictions exemplified by social media, will use education as gateway to every mind.

All these things are true not only for SHAI, but every step of AI development, right from what we have today. Any benefits of AI in education will accrue differentially to the rich, be it people or nations, and its unintended consequences will be borne first and mostly by the disadvantaged, and then by all humanity. 

Those who think I am going too far with all this need to be reminded that it is leaders of AI technology who are predicting these advancements in AI capability and performance. I am merely drawing the logical conclusions of these predictions.

The ethically committed among the pioneers of AI are deeply concerned and are grappling with ‘(super)alignment’, i.e., ensuring that SHAI is aligned with human well-being and will avoid these catastrophes. 

But most people in this field are shying away from or deliberately obfuscating the logical conclusions to be drawn from AI’s trajectory. This may be because of a commitment that arises from their own commercial interests and/or overestimations of the human ability to resist adversities of our own making.

So, do we want to hand over education to something that may seem indistinguishable from a human being even if it is not human but more powerful, and which will hollow out students’ minds and be in a position to control us? Something that could amplify inequality and strife?

Eagerly handing over our children and future to the Pied Piper of Hamelin is not a lesson we have learnt from education—have we?