OVER the past 12 months, the capabilities of frontier AI models have remained a central topic of debate. Recent performance demonstrations across a variety of technical fields (maths and bio-genetics in particular) show that scepticism around raw capability at least is a dated position. However, there are still question marks around financial viability, regulation of broad-based societal risks, and impact on workers and labour more generally. The latter two deserve more scrutiny.
The first concern, around broad-based risks, has gained quite a bit of attention with a spate of resignations from top AI companies, with departing individuals claiming that AI poses a serious risk to society and carries a non-zero chance of triggering human extinction.
What is the pathway of such a catastrophe? The most common position at this point is that, in the absence of clear ethical guardrails, AI tools can be used to create biological weapons of the sort that could trigger large-scale human loss. Others think that, in the wrong/rival hands, AI could be used to trigger weapons of mass destruction.
These fears appear to be fairly fantastical, and the timing with which they’re being made public — just when the frontier models are poised for initial public offerings — suggests they’re part of a Silicon Valley hype cycle.
However, if these claims about capabilities and the risks they could carry are even remotely true, they pose a much simpler question. If a technology is truly this capable, then what is it even doing in private hands? Historian and economic commentator, Adam Tooze, made an insightful analogy that if we take the claims of societal risk at face value, we’re essentially developing a technology that is similar to the nuclear bomb in the type of catastrophe it can pose, and that too at a much greater scale.
The impact of AI on labour markets is something that many workers will face in a variety of ways in the coming years.
But the key difference is that the development of the bomb took place through the Manhattan Project, which was entirely governed and steered by the state, with private enterprise playing a junior and often strictly compartmentalised role. AI, on the other hand, is being developed by private enterprise, with private investment steering its priorities, and commercial viability shaping the discourse around it. At any other point in human history, has anything posing purported risks of this nature been allowed to emerge via a process of borderline anarchic market-led competition?
The logical corollary of this parallel is that anything that poses large-scale existential risk is too important to be left in private hands. Public regulation determining its development, use, sale/purchase, and evolution becomes a necessity. And that this regulation cannot emerge from the industry itself, which is subsumed under intense competition and has a vested interest in skirting around various legalities, but rather from the state.
Beyond pushback against the fearmongering, there is also some scepticism around AI’s potential to generate astronomic growth in capitalist societies. In particular, Ben Moll and Alex Imas argue in a recent Substack piece that predictions of double-digit AI-driven growth, the kind floated by Dario Amodei, Leopold Aschenbrenner, and various technologists, are unlikely to materialise within the next decade or so, even though making a textbook case for it is relatively simple.
Moll and Imas argue that explosive growth is easy to generate in a standard textbook growth model. If automation steadily raises the share of tasks that machines can perform, diminishing returns to capital weaken and eventually disappear once machines can do everything.
The core of their position, however, argues that this outcome depends on several assumptions that are unlikely to hold in practice. Automation would need to spread across the whole economy very quickly, not just in cognitive or remote tasks but across physical, messy, and relational work that resists mechanisation. Spending patterns would need to stay proportional even as automated goods become cheap, rather than shifting towards whatever remains scarce, which is the usual pattern of structural change.
But a central facet, which these authors don’t give much attention to but others have honed in on, is that in a society where the cost of production approaches zero, there would need to be enough demand to absorb all that cheap, expanded output. Where would such demand come from once automation shifts jobs and income away from workers who spend a large share of what they earn?
This brings us back to perhaps the key issue of AI capability that has been drowned out by fears of biological weapons and global catastrophe. The technology’s impact on labour markets is something that many workers will face in a variety of ways in the coming years. It’s not just about a machine replacing a human, though that is the most commonly held view. It’s also about how the threat of a machine replacing humans changes working conditions and the kind of income one can expect to receive.
The history of technology adoption in labour processes is often told in large sweeps of history. Technologists tend to use common examples, such as how the arrival of machines after the late 19th century unlocked rapid growth and raised prosperity or how the creation of the ATM didn’t lead to a mass exodus of banking sector jobs. The implication is that society adapts and the outcome is usually positive.
What this cheerful story misses out is that adaptation processes produce winners and losers, with the latter taking years to adjust. In a world already characterised by the deskilling of labour, people rendered jobless by a machine (or the threat of a machine) won’t suddenly find alternative employment. Those two or three decades in which the labour market sorts itself out will seem minuscule in a broad sweep of history, but for those living through this period, it will be a particularly bleak eternity.
The writer teaches sociology at Lums.
X: @umairjav
Published in Dawn, September 28th, 2026
The post AI and social catastrophe appeared first on Dawn – Home.
