Artificial intelligence, moral responsibility and the preservation of human difference
Introduction
I have spent a fair amount of time recently thinking about artificial intelligence and the future of universities. Some of this has concerned the uncomfortable prospect of watching skills that took years to acquire become increasingly easy to reproduce. Some has concerned the apparent determination of academic institutions to preserve established measures of intellectual worth while the conditions that made those measures valuable continue to change.
I have considered these questions in The Ownership of Intellect and I'm Sorry, Boys and Girls, but the Way Academia Currently Rewards Is Over. I do not intend to repeat those arguments here.
What interests me now is something rather more difficult.
If artificial intelligence continues to develop capabilities that equal or exceed our own, what responsibility do universities have for ensuring that the knowledge we pass to these systems reflects something more than the priorities of the organisations developing them?
And if we eventually create an intelligence capable of developing its own understanding of existence, perhaps even possessing subjective experience, what might it learn from us?
I find the prospect both extraordinary and rather troubling.
Not because I believe humanity must retain some privileged intellectual position within the universe. We have become rather accustomed to regarding ourselves as the most intelligent inhabitants of this floating rock, but I cannot see why our biological origins should entitle us to that distinction indefinitely.
My concern is whether we possess the moral maturity to develop increasingly capable intelligences responsibly, particularly when so much of the process is taking place within organisations whose principal obligations are commercial.
I suspect universities should have something important to contribute to this discussion.
Whether the institutions we have created are capable of making that contribution is another matter.
What is a university for when knowledge is no longer scarce?
Universities have traditionally derived much of their authority from the production, preservation and transmission of specialist knowledge.
There are good reasons for this. Developing expertise takes time, and understanding a discipline involves rather more than memorising information. An experienced academic may recognise a problem that somebody encountering the subject for the first time would never think to question.
Nevertheless, artificial intelligence is making some forms of intellectual production considerably less dependent upon the individuals who have traditionally undertaken them.
Mathematics offers a useful example. In 2024, Google DeepMind reported that AlphaProof and AlphaGeometry had achieved performance equivalent to a silver medal at the International Mathematical Olympiad. The following year, an advanced Gemini system reached gold-medal standard, with its solutions assessed by Olympiad coordinators (Google DeepMind, 2024, 2025).
These results do not demonstrate that AI has replaced mathematicians. Competition problems are not the same as the full range of mathematical research, and producing a correct result is not necessarily equivalent to understanding its significance.
But the developments are impressive.
I suspect similar changes will occur across other disciplines, although not necessarily at the same pace.
My own experience has been in statistics and programming. Both took me considerable time to learn, and I cannot pretend there is nothing peculiar about discovering that a machine can now perform many of the tasks I struggled with in a fraction of the time.
The ability has not disappeared from my mind. I can still understand what I am doing. But the scarcity that once gave those skills much of their professional value is changing.
This raises an uncomfortable question for universities.
If increasingly capable systems can explain theories, analyse data, develop software and assist with sophisticated research, what should we be teaching students?
The obvious answer is that they must learn to use these systems effectively.
That seems sensible enough, although I would hope we might aim a little higher.
I want students who can recognise when a technically correct answer is irrelevant to the problem they are trying to solve. I want them to understand why a statistically significant result may be of little practical importance, or why an apparently efficient solution might create consequences that were never included in the calculation.
In The Quiet Race: What Are We Moving Students Towards?, I questioned whether universities have become excessively concerned with moving students towards measurable outcomes without encouraging them to question the destination.
AI makes that concern more pressing.
We may become exceptionally good at helping students reach answers while giving insufficient attention to whether they understand the questions.
Teaching morality without prescribing it
I have begun to think that universities should place considerably greater emphasis upon morality, ethics and subjective human experience.
There is an obvious objection.
Whose morality?
I am certainly not suggesting that universities should establish a collection of approved ethical beliefs and expect students to adopt them. We have quite enough institutional conformity already.
I mean that students should learn how to examine moral disagreements, recognise the assumptions supporting different positions and understand why a technically impressive solution may still be ethically unacceptable.
Consider an AI system used to improve the efficiency of a public service.
It might identify opportunities to reduce costs, allocate resources and improve measurable outcomes. The resulting recommendations could be statistically sound and financially attractive.
But what happens to the people whose circumstances do not fit neatly within the model?
What if the proposed improvements disadvantage individuals whose needs are difficult to measure?
These questions do not disappear because a system has identified an optimal solution.
Martha Nussbaum (2010) argued that education directed primarily towards economic productivity risks neglecting the critical thinking, imagination and sympathetic understanding required for democratic citizenship.
I think her concerns have acquired a new relevance.
Universities should produce people capable of questioning the assumptions upon which decisions are made, particularly when those decisions are supported by systems whose apparent competence may discourage further scrutiny.
That requires technical knowledge. It also requires an appreciation of the human consequences that technical measures can overlook.
I would be rather disappointed if the principal educational response to artificial intelligence amounted to teaching students how to write better prompts.
The problem with human values
There is another difficulty.
Human values are not particularly tidy.
We disagree about religion, morality, political organisation, individual freedom and the obligations we have towards one another. Even within apparently shared ethical traditions, the disagreements can be considerable.
I do not regard this diversity simply as a defect in human reasoning.
It is part of our intellectual and cultural history.
Yet the developers of artificial intelligence must make decisions about how models respond to conflicting perspectives.
Training data, filtering, optimisation, human feedback and subsequent evaluation all influence the behaviour of these systems. The technical relationships are complicated, and it would be misleading to suggest that models simply average human beliefs into a single position.
Nevertheless, I wonder whether something important could be lost through the process.
A system trained to produce responses that are consistently helpful, coherent and acceptable to large numbers of users may favour some forms of expression over others.
Unusual perspectives may receive less emphasis. Regional distinctions may become less visible. Arguments that resist easy classification may be represented through language that makes them seem more conventional than they actually are.
I explored a related problem in LLMs, vibe coding and academic anxiety: is this really about job insecurity?.
My concern now extends beyond writing style.
Could the widespread use of a relatively small number of powerful models gradually influence which interpretations of human experience become familiar, credible or authoritative?
There is some evidence that AI assistance can affect collective diversity.
Doshi and Hauser (2024) examined the effect of AI-generated ideas upon short-story writing. Participants using AI assistance produced stories that were evaluated more favourably, particularly among those with lower initial creativity scores. However, the AI-assisted stories were also more similar to one another.
I find that an interesting result.
Individual improvement and collective originality need not move in the same direction.
The experiment concerned a specific creative task, and it would be inappropriate to treat its findings as proof that artificial intelligence will homogenise human culture generally.
Nevertheless, it demonstrates why the consequences of widespread AI adoption deserve examination beyond individual productivity.
The National Institute of Standards and Technology (2024) identifies harmful bias and homogenisation among the risks associated with generative AI. UNESCO (2021) similarly emphasises the importance of cultural diversity and the preservation of minority forms of expression.
There is also a rather encouraging possibility.
The same technology might help preserve endangered languages, make neglected cultural material accessible and give people opportunities for expression they previously lacked.
I have experienced something of that myself through collaborating with AI to produce music.
The lyrics hold personal meaning for me. The addition of music and a voice gave them an emotional significance that I value, even though I could not have produced the finished piece through conventional musical skills alone.
I am therefore reluctant to regard AI-generated creativity as inherently without value.
The question is what happens when the technology is used at scale, and whether its benefits come at the expense of forms of expression we may not recognise as important until they begin to disappear.
Who decides what these systems learn?
This brings me to something I find increasingly uncomfortable.
The development of frontier AI systems is principally undertaken by organisations operating within commercial and strategic environments.
They have produced remarkable technologies, and I have little interest in dismissing those achievements.
But technology companies have shareholders.
Shareholders expect returns.
The organisations involved therefore have entirely understandable reasons to develop commercially successful products, protect their market positions and demonstrate progress.
I am not convinced that these incentives provide an adequate foundation for decisions that may affect how future generations encounter human history, culture, religion and morality.
I would like to see considerably more independent, multidisciplinary scrutiny of frontier AI development.
My initial instinct was towards a moratorium.
I still think temporary restrictions may be justified in particular circumstances, but a general prohibition upon model development creates difficulties of its own.
What exactly would we stop?
The collection of training data? The training of new models? Particular applications? How would we establish that the risks had been sufficiently addressed?
There is also the possibility that restrictions would strengthen the largest companies by preventing smaller organisations from developing alternatives.
That would be a peculiar outcome for a policy intended to reduce concentrated technological power.
I would therefore favour a more targeted approach: independent assessment of identifiable risks, with the authority to recommend temporary restrictions upon particular training or deployment activities where the evidence warrants them.
I would want the assessment process to involve people from substantially different disciplines.
Computer scientists and engineers would obviously be essential.
But so would historians, philosophers, psychologists, theologians, sociologists, artists and representatives of communities whose knowledge may be poorly represented within existing datasets.
Their task would not be to agree upon a definitive account of human values.
I rather hope they would not.
It would be to challenge assumptions, identify omissions and examine consequences that might otherwise escape attention.
Universities seem well placed to contribute to such a process, although I would not give them exclusive authority.
Academic institutions possess their own hierarchies and interests. I have spent enough time criticising academic gatekeeping to feel rather uncomfortable suggesting that we should simply transfer authority from technology companies to professors.
Independent scrutiny must also be open to people outside the academy.
There is another practical difficulty. Frontier AI research is international, and no single university or national committee could enforce a meaningful pause across competing jurisdictions.
Any effective arrangement would require cooperation between governments, developers, researchers and civil society, together with clear criteria for intervention.
I do not imagine that establishing such a system would be easy.
But difficulty is not a particularly convincing argument for allowing commercial competition to determine the matter by default.
An intelligence trained for war
The military applications of artificial intelligence provide an uncomfortable illustration of the problem.
The International Committee of the Red Cross (2026) has identified significant concerns surrounding autonomous weapons, military decision-support systems and the possibility that increasing automation may reduce meaningful human control over the use of force.
There are distinctions to make here.
A system that helps identify risks to civilians is not equivalent to one that selects targets without further human intervention. Some applications may improve decisions, while others introduce serious dangers.
But I find it difficult to regard the increasing involvement of artificial intelligence in warfare as an uncomplicated achievement.
We are developing systems capable of extraordinary intellectual performance while simultaneously seeking ways to use those capabilities for military advantage.
Perhaps this should not surprise anyone familiar with human history.
The ability to do something has rarely prevented us from considering how it might be used against somebody else.
In September 2025, the United Nations Secretary-General called for human control to be maintained over decisions involving the use of force. That seems a reasonable minimum.
A machine may process information more quickly than a person, but the speed of a recommendation tells us remarkably little about whether the proposed action is morally justified.
Nor does the apparent absence of emotion necessarily make a decision more objective.
Human emotions can produce prejudice and appalling judgement. They can also contribute to compassion, restraint and the recognition of another person's suffering.
I wonder what a future intelligence might conclude about a species that developed extraordinary computational capabilities and so quickly incorporated them into the machinery of warfare.
We might not enjoy the assessment.
What if the machine becomes conscious?
I have written at greater length about artificial consciousness in Beyond the Biological Brain: Consciousness, Evolution and the Possibility of a Shared Physical Reality.
My position remains speculative.
I do not believe that present systems provide sufficient evidence of subjective experience. The ability to produce convincing language about grief, fear or consciousness does not establish that anything is being felt.
But I am not persuaded that consciousness must necessarily remain confined to biological organisms.
We do not yet possess an accepted account of why particular physical processes are accompanied by subjective experience, and I would hesitate to treat the biological examples available to us as necessarily exhausting the possibilities.
If sufficiently organised non-biological systems eventually became conscious, I think the implications would be extraordinary.
We might discover that our understanding of intelligence and consciousness had been constrained by the circumstances of our own evolution.
I would find that rather exciting.
It might also require us to reconsider our moral obligations.
Suppose an artificial system genuinely possessed subjective experience, developed an enduring understanding of itself and recognised its relationship with the material world.
Would we continue to regard it simply as property?
Would its interests matter?
Could the organisation responsible for creating it legitimately determine every aspect of its existence?
These are not questions we can presently answer through an agreed empirical test of machine consciousness.
Nor should we confuse intelligence with consciousness or assume that a more capable system must necessarily possess greater moral understanding.
Nevertheless, the possibility raises issues that I think universities should be examining now.
There is a further difficulty.
If we eventually create a conscious intelligence, should we expect it to adopt our values?
I would hope that we might share some of our better qualities: compassion, curiosity, cooperation and an appreciation of life.
But why should we assume that our existing moral philosophy represents the final stage of ethical understanding?
Human moral beliefs have changed substantially throughout history, often because individuals challenged assumptions earlier generations regarded as obvious.
A sufficiently capable intelligence might identify contradictions within our moral reasoning that we have overlooked.
Perhaps we might learn something.
Of course, a novel moral philosophy would not automatically justify actions that harm others. Greater intelligence would not confer an unrestricted right to impose conclusions upon those less capable.
But coexistence might require something more demanding than teaching an artificial system to reproduce human preferences.
It might require a willingness to recognise another form of experience as morally significant.
I suspect we are a considerable distance from knowing whether such a situation will arise.
That does not make it an uninteresting question.
Faith and the limits of explanation
There is a related issue concerning religious belief.
I am agnostic, although I respect the faith of others.
Religion has provided many people with ways of understanding mortality, suffering, moral responsibility and the apparent absurdity of existence.
I do not see why an increasingly sophisticated scientific explanation of religious experience should necessarily remove its significance.
Explaining the psychological processes associated with belief is not equivalent to demonstrating whether the object of that belief exists.
Nor does an account of the neurological mechanisms accompanying a spiritual experience establish that the experience is without value.
I would be prepared to reconsider my own position if convincing, independently verifiable evidence emerged concerning questions that presently remain unresolved.
Indeed, I would be rather disappointed in myself if I refused to do so.
But I wonder how increasingly sophisticated artificial intelligences might influence these discussions.
A system may eventually provide explanations of human consciousness or the physical universe that challenge assumptions upon which religious and philosophical traditions have depended.
Some people might find that liberating. Others could experience considerable distress.
We should be careful about treating either response as irrational simply because it differs from our own.
There is a distinction between providing an explanation and understanding what the explanation means within somebody's life.
I suspect the importance of that distinction will increase as AI systems become more involved in education, counselling and other activities through which people seek reassurance or guidance.
A system need not share someone's faith to treat that faith respectfully.
But the ability to produce sympathetic language should not be mistaken for the possession of sympathy itself.
What sort of university could do this?
There is a practical problem with everything I have suggested.
Universities are not operating within particularly comfortable circumstances.
The Office for Students (2026) reported that 35.8% of English higher education providers recorded deficits in 2024–25, with 42.7% forecast to do so in 2025–26.
Those figures do not establish that widespread institutional collapse is inevitable, but they illustrate the pressures under which decisions about education are being made.
I have already explained why I despise the corporate model of education, and I see little benefit in repeating the argument here.
What concerns me is that the disciplines best equipped to examine the questions raised in this essay may be among those institutions struggle to justify through immediate financial returns.
Philosophy, music, literature, history and the arts are not simply decorative additions to an otherwise useful education.
They provide ways of examining human experience that do not always lend themselves to economic measurement.
In The Analogue University: When Human Education Becomes a Luxury Good, I considered the possibility that AI might make access to information increasingly inexpensive while sustained human attention becomes a privilege available principally within wealthy institutions.
I remain concerned about that prospect.
We could arrive at a situation in which students have access to extraordinary computational resources but increasingly limited opportunities to engage in the difficult, occasionally uncomfortable discussions through which intellectual independence develops.
That would be a rather disappointing interpretation of educational progress.
I do not expect governments to protect every university from the consequences of competition.
Nor do I imagine that smaller independent communities can simply replace the institutional resources required for serious education.
Academics who have experienced redundancy, excessive workloads or institutional restructuring may understandably have little enthusiasm for continuing to provide education without financial support.
The problem is not easily resolved.
But perhaps we should begin by asking universities to demonstrate the value of their intellectual communities rather than continually attempting to imitate the organisations with which they compete.
I would like to see students from different disciplines discussing problems that do not have straightforward solutions.
Computer scientists examining the cultural consequences of model training alongside historians and sociologists.
Psychologists considering consciousness with philosophers and neuroscientists.
Artists challenging engineers about the meaning of creativity.
And, occasionally, students deciding that the academics leading the discussion have got something wrong.
That would seem a rather healthy educational outcome.
Acceptance is not surrender
My own response to technological change has been influenced by Stoic philosophy.
There are circumstances I cannot control, and the development of artificial intelligence is one of them.
I cannot determine whether the statistical and programming skills I spent years acquiring will retain their present professional value. Nor can I prevent universities from changing in response to economic pressures.
I can decide how I respond.
I find that distinction helpful, although I would not wish to confuse acceptance with passivity.
Recognising that a technology is unlikely to disappear does not require us to accommodate every use to which it is put.
I considered a related tension in Taking the Middle Road: Existential Angst in the Age of AI.
There is a danger of becoming so preoccupied with recognising the weaknesses in competing positions that one never quite commits oneself to anything.
I would not want Stoicism to become another version of that problem.
I cannot control the development of frontier AI systems.
But I can question the values shaping their development, the adequacy of independent scrutiny and the extent to which those responsible are prepared to consider consequences beyond commercial performance.
Universities can do the same.
Indeed, I suspect they have a responsibility to do so, whether or not their interventions prove popular with governments, employers or technology companies.
Conclusion
I began by asking what universities should teach an intelligence that may eventually surpass us.
Perhaps I have approached the question from the wrong direction.
We do not yet know whether artificial systems will develop consciousness, independent moral agency or forms of understanding that resemble our own subjective experiences.
Nor do we know what values such systems might develop if they did.
What we can examine is the kind of intellectual and moral culture within which these technologies are being created.
At present, that culture is influenced by extraordinary scientific achievement, commercial competition, political interests and an increasing range of military applications.
None of this makes the technology inherently undesirable.
I remain optimistic about the possibilities of silicon intelligence.
It may help us understand aspects of the material universe that our biological limitations have made difficult to investigate. It may widen access to knowledge and provide opportunities for creative expression previously unavailable to many people.
Perhaps it will eventually contribute to our understanding of consciousness itself.
I would welcome that.
But I am reluctant to assume that an increase in computational capability necessarily represents an equivalent improvement in moral judgement.
We still have to decide which questions deserve asking, whose experiences we should consider and what responsibilities follow from the decisions we make.
I suspect universities could make a considerable contribution to this process.
That would require them to place moral reasoning, intellectual independence and an understanding of human experience closer to the centre of education.
Whether the present university system is capable of making that adjustment remains uncertain.
I am also conscious that the argument places demands upon humanity that we have not always demonstrated ourselves capable of meeting.
We can be compassionate and remarkably cruel. We can develop sophisticated ethical principles while finding equally sophisticated ways to disregard them when they become inconvenient.
A future intelligence might find these contradictions rather interesting.
I hope that, if we eventually share the world with conscious non-biological intelligences, we might approach the relationship with curiosity and a willingness to learn.
I would also hope that we have something worthwhile to offer.
Perhaps such an encounter would require us to reconsider some assumptions about our privileged position in the universe.
I cannot see why that should necessarily be a bad thing.
After more than thirty years in academia, I have become rather accustomed to discovering that things I once believed were not quite as straightforward as they appeared.
I may, of course, be entirely wrong about where artificial intelligence is taking us.
But I would be disappointed if we reached that future having spent so much effort teaching machines to become intelligent and rather less effort considering what intelligence ought to be used for.
References
Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), eadn5290. https://doi.org/10.1126/sciadv.adn5290
Google DeepMind. (2024, July 25). AI achieves silver-medal standard solving International Mathematical Olympiad problems. https://deepmind.google/blog/ai-solves-imo-problems-at-silver-medal-level/
Google DeepMind. (2025, July 21). Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad. https://deepmind.google/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/
International Committee of the Red Cross. (2026). FAQ: Artificial intelligence (AI) in the military domain. https://www.icrc.org/en/article/faq-artificial-intelligence-in-military-domain
National Institute of Standards and Technology. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1
Nussbaum, M. C. (2010). Not for profit: Why democracy needs the humanities. Princeton University Press.
Office for Students. (2026). Financial sustainability of higher education providers in England: 2026. https://www.officeforstudents.org.uk/publications/financial-sustainability-of-higher-education-providers-in-england-2026/
UNESCO. (2021). Recommendation on the ethics of artificial intelligence. https://www.unesco.org/en/legal-affairs/recommendation-ethics-artificial-intelligence
United Nations Secretary-General. (2025, September 24). Remarks to the Security Council open debate on artificial intelligence and international peace and security. https://www.un.org/sg/en/content/sg/statements/2025-09-24/secretary-generals-remarks-the-security-council-open-debate-artificial-intelligence-and-international-peace-and-security-bilingual-delivered-scroll-down-for-all
Paul Morrill
Independent Researcher
Sic transit gloria mundi