Time to strengthen AI safeguards before it goes too far

Jacob Coxon, a former Anthropic and OpenAI employee, just came out with a dire warning for everyone. According to him, the development of AI is going on at such a breakneck speed, with minimal or no oversight, that it could eventually kill all of us.

Recently, a very disturbing event precipitated this fear. A bunch of AI agents, created by OpenAI, were given the task of solving some cybersecurity problems in a sandbox, a contained environment for testing before something is released into production. This test was meant to keep the agents disconnected from the internet. Apparently, these agents found a chink in the armour and escaped. They then hacked into Hugging Face, an AI company, gaining full access to one of its servers and then OpenAI. The intriguing part is that the agents banded together, colluded, formed groups, and created a leadership structure. Well aware that they were cheating, they covered up their trails. Finally, when detected, some agents even sacrificed themselves for the others.

Tap here to add The Daily Star as a trusted source

AI has made great strides, bringing in the might of LLMs (Large Language Models). But many of these developments have also come at a steep human cost. Worrisome is the over-reliance of humans on AI to do routine work for them every day. They are increasingly turning to AI not only to write simple letters, emails, and college essays, but also to summarise documents for them, even though some of these are only two pages in length. This is harming their learning, leading to cognitive decline. I have always found humans to be organically creative. But that trait is an indirect casualty too, meaning that creativity is more and more wrapped in AI, and talked about in terms of being able to use AI efficiently. AI is mostly laying out the creative base on which many are only building, rather than conceiving something in their minds from scratch. The patience to read an article, or the capacity to think is gradually diminishing.

Companies driving the frontier AI models are zipping ahead to win the AI race. The two pioneering private AI companies, OpenAI and Anthropic, are leading the AI race. They don’t have to adhere to the same reporting requirements mandated for publicly traded companies. Other hyperscalers are developing and experimenting with their models at a frenetic pace. It is unclear as to what sort of guardrails they are putting in place in their model training methods and tests, if at all, given the rapid pace of development under minimal or no external supervision, and their huge appetite for profits. Everything that has a tinge of AI now is in huge demand from the stocks to the products. The race is so intense among the competitors that they are issuing bonds to raise their capital expenditure significantly to get ahead. Data centers are being built to a fever pitch to meet the enormous demand for AI compute. The relentless push is on for building super intelligence that will outsmart humans completely. Many AI researchers and engineers are working on these models for long hours for hefty pay packages.

Safety fears prompted an essay by Dario Amodei, the CEO of Anthropic, highlighting the need for employee-like independent evaluators on site to monitor the AI training methods and models, setting safety standards, and collaboration among nations. Companies pursuing newer and more powerful AI models need to shoulder more corporate responsibility to communicate their approach and improve training methods. Maybe they need peer reviews.

The first obstacle is the prospect of huge returns in near future for the frontier companies already invested heavily on their models, GPUs, and data centers. The second one is geopolitical. No nation will slow down unilaterally what is deemed the ultimate power. Tension among major nations, engaged in the race, will make it very hard for them to get on board because of mutual trust issues. No one wants to risk falling behind in AI dominance.

These companies may put in place some guardrails, in terms of monitoring, auditing, and reporting. Governments may introduce regulations in consultation with AI experts. But politics will play a major part in how far these measures go. They could perhaps temporarily slow down the development. I am a bit skeptical of any major impact though. The genie is out of the bottle. We have to manage its progress. AI is not just a tool, but an intelligent digital species now.

I can’t fully write off a huge AI takeover. Autonomous, biological, and chemical weapons, and drones, guided by rogue AI, no longer seem very distant possibilities to me, the way things are moving toward creating decision-making super intelligence. I hope this wake-up call results in some meaningful curbs.

I see a world of ultra-privileged class of people controlling the lives of the rest. More of the wealth will be concentrated in the hands of the fortunate few, rendering the rest poor, or helpless. AI may create a world of much greater disparity, if white-collar jobs are in decline at a rapid pace. A few lucky executives, researchers, engineers, and venture capitalists will become ultra-rich through the IPO process, with an unlucky majority fighting for dwindling job opportunities. Even some blue-collar jobs may be on the chopping block because of physical AI. I see four-year college degrees, as we know them, changing significantly, and in some cases being shortened and revamped. I have long felt that these take too long anyway, in terms of return on investment (ROI). With increased use of AI, I see an existential threat to many of these colleges and degrees.

AI will certainly power us to game-changing innovations in our lives, and we will need to adapt. We need to address the dangers of creating super intelligence right now to mitigate tremendous risks ahead. I believe in human intellect. Let humans be the drivers of AI, not the other way around.

Arif Shahjahan is a US-based essayist, thinker, and a software engineer. 

Views expressed in this article are the author's own.

Follow The Daily Star Opinion on Facebook for the latest opinions, commentaries, and analyses by experts and professionals. To contribute your article or letter to The Daily Star Opinion, see our guidelines for submission.