vacation

I have recently returned from a short vacation to Las Vegas, Nevada. I will be adding some new posts to fill in some of the missing days while we were in Vegas. I didn't take a computer with me, so I could not easily make any posts while we were there. As expected, it was 100+ degrees every day we were out there. Sure, the humidity is lower, but sweat doesn't really give a shit. 

AI apocalypse

This is the last in this AI series that I will publish for now. This article appeared in The Guardian, a center-left British daily online and print newspaper founded in 1821. It survived under a trust for many years but converted into a "limited company" in 2008. Like all newspapers, it is struggling to stay independent and afloat. If you could manage to find a few bucks to send them, you'd be helping to further independent reporting.

AI could kill all humans in next decade, warn experts: but how seriously should we take them?

Alarming warnings from industry insiders increase pressure for curbs on artificial superintelligence

Robert Booth UK technology editor

Does artificial superintelligence really pose a risk greater than nuclear weapons? Is there a significant chance of “a Chornobyl-sized catastrophe”. Might there even be a greater than 10% chance that AI could “kill all humans” in the next decade?

These warnings were issued over the past 48 hours on both sides of the Atlantic about the potential impact of a technology that most people still think of as a more talkative search engine.

Some of the threats were raised in the UK as MPs and peers began to wrestle with the danger of AI outperforming human capabilities. The nuclear warnings came from Des Browne, a former defence secretary, and Prof Stuart Russell, an eminent Berkeley computer scientist.

Beatrice Fihn, who won the 2017 Nobel peace prize for leading the International Campaign to Abolish Nuclear Weapons, told parliamentarians: “It is not the first time we’re confronted with abilities that could end up killing us all.”

The session on Monday (Sept 7) was convened by Control AI, a lobbying group pushing for international regulation of the technology. It is backing a bill tabled in parliament this week by the Labour MP Alex Sobel and aimed at banning the creation of artificial superintelligence (ASI).

On Tuesday night (Sept 8), a senior employee at Anthropic admitted he believed there was a greater than 10% chance the technology could “kill all humans” in the next decade and warned that his AI company did not have a plan to ensure ASI was aligned, meaning it did no harm. Predictions for when ASI might be reached vary from several years to more than a decade.

Evan Hubinger, alignment science lead at Anthropic, posted the comment after Jacob Coxon, a 28-year-old researcher at the San Francisco company and previously at OpenAI, resigned, claiming “neither company was acting responsibly”. He said they were “gambling with our lives”.

Coxon said the risks at Anthropic were well understood but they were “locked in a race to get there first”. In a glimmer of hope, he added that he was optimistic about the potential for coordination between US labs on pacing their progress in the race.

Anthropic has been approached for comment on the issue. OpenAI pointed to a statement from its chief scientist, Jakub Pachocki, who said last week: “International coordination on future AI development needs to become a top priority for governments around the world.”

The push for wider coordination to control ASI was at the heart of events at Westminster this week. According to the Financial Times, government officials voiced concern that Anthropic had declined to submit its latest model – Mythos 5.1 – to the UK’s AI Security Institute for pre-release testing.

Only a few US organisations have had access to Mythos 5.1, the Cabinet Office confirmed. A spokesperson added: “The AI Security Institute continues to collaborate closely with industry partners, including Anthropic.”

The Labour MP Darren Jones has written to the prime minister, Andy Burnham, and the heads of the UN and the OECD calling for a “multinational treaty for the regulated and safe development of superintelligence – not a ban on innovation or scientific endeavour but a safety-first approach to the rapid development of this technology”.

Citing Coxon’s resignation, Jones added: “The debate ranges from the end of humanity to claims of ‘marketing hype’. Either way, governments must now step in.”

On the other side of the Atlantic, Bernie Sanders ratcheted up his AI safety campaign this week by again calling on Congress to regulate the technology. He pointed to polling suggesting that 81% of Americans believed their politicians should take action.

The independent senator for Vermont said: “We can’t allow a handful of greedy people to play God and determine the future of humanity – our economy, environment, democracy, privacy and more – without public input.”

ControlAI is funded by Jaan Tallinn, the multi-billionaire founder of Skype who calls himself an “anti-extinctionist”. He has dedicated part of his fortune to campaigning for AI safety and says some senior AI executives would be happy for humanity to be wiped out.

Despite being an early investor in Anthropic and Google DeepMind, two of the leading AI labs, Tallinn estimates that 10-15% of AI employees believe the technology will be a worthy successor to humanity.

He said in an interview earlier this summer: “A fairly known AI researcher said to me ‘Jaan, don’t worry about this. Humans are a disposable species’. From what I understand he was fine with becoming extinct.”

Tallinn’s lobby group ran the session for MPs and peers at Westminster on Monday and urged them to back measures to curb the most powerful AI models. On every chair was a copy of the book, If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI.

The politicians heard from Russell, who made the Chornobyl warning and said: “The other possibility is a much larger catastrophe, in which humanity loses control irreversibly. But we have no say over whether we continue to exist.”

Browne said that when he was defence secretary he thought nuclear war was the most likely threat facing humanity. Now he believed “a superintelligent AI poses a threat on the same, possibly, a greater scale”.

Other contributors were more cautious. Dr Andrew Rogoyski, of the Surrey Institute for People-Centred AI, said: “In reality, these systems are nowhere near as versatile as humans, let alone humans acting collectively. I suspect we’re heading towards ‘the great disappointment’ where advanced AI turns out to be too expensive and not useful enough to continue in its current form.”

David Barber, director of Sofair, a state-backed AI research lab combining academics from Oxford, Cambridge, Edinburgh and UCL, said he was worried about people “throwing the baby out with the bathwater”.

“AI is not going to go away,” he said. “It’s incredibly useful, whether or not you allow it in a fully unconstrained way to access the internet and various systems that’s potentially problematic. We may need to learn how to better control these things. There are vulnerabilities in the software frameworks that need to be patched. But that’s doable.

“What we need as a country is to get to grips with the duality that is both an incredibly important and useful technology, and at the same time, it’s something that needs to be carefully thought about and carefully controlled.”

Sandra Wachter, a professor at the Oxford Internet Institute, said she did not believe in “Terminator scenarios” but that AI posed real threats including its environmental impact, spreading of misinformation and replacing jobs.

“These problems are real and urgent and need addressing now,” she said. “Terminator scenarios are a big distraction from real issues.”

Gary Marcus, an AI industry commentator and academic, said: “There is a difference between superintelligence that is aligned (if such a thing is possible) and superintelligence that is not.

“It is at least conceivable that the former might be net positive. So far we have neither, but a superabundance of hype combined with a striking lack of prudence on OpenAI’s part has gotten us where we are, with intense mistrust all around.”

The Guardian

At this dangerous time

We hope you appreciated this article. Before you close this tab, can we ask you to support the Guardian at this dangerous time for journalism in the US? For a few days only, our most impactful monthly support option is just $7.50 a month for six months.

According to a leading global watchdog, American democracy is now more imperiled than at any point since the 1960s, marked by a precipitous decline in press freedom – driven by mounting pressure from the Trump administration in the form of threats, criminal investigations, politicized regulation, frivolous lawsuits and, for public media, catastrophic funding cuts. 

Meanwhile, organizations that are supposed to be independent like the FBI and the FCC, our radio and television regulator, have also been targeting press freedom under Trump-aligned leadership, with the FBI raiding a reporter’s home and the FCC threatening ABC’s TV licenses after Jimmy Kimmel made a joke about Melania Trump.

The response from some ultra-wealthy and corporate media owners, keen to appease the president, has been chilling: CBS News has been taken over by a Trump ally; CNN is poised to be taken over by the same billionaire; Jeff Bezos has continued to impose cuts and editorial interventions at the Washington Post; and multiple outlets have settled multimillion-dollar lawsuits from the administration to protect their business interests.

Democracy is best served by a robust, thriving free press. But when that freedom is under attack, it falls to a determined few news organizations to ensure the full truth still reaches the public. Owned neither by a billionaire nor a corporation, the Guardian remains dedicated to covering this administration with uncompromising moral and factual clarity – and to keeping trustworthy journalism paywall-free for the world.

Despite the risks of maintaining our fierce independence, what sustains us – and fills us with deep gratitude – is the unwavering support we’ve seen from readers. It is no exaggeration to say that we are here because of you: a majority of our funding comes directly from people like you responding to messages like this. Your support not only powers our work, but more importantly, it safeguards the financial independence that underpins our editorial freedom and courage.

If you are able, please support us today. All gifts are gratefully received, but for the next several days only, our most impactful monthly support option is $7.50 a month for the first six months. (Among the great benefits: you’ll see far fewer annoying fundraising messages like this.) It takes just 37 seconds to set up. Thank you for considering.

AI tipping point

I borrowed parts of this article from Time Magazine. To read the rest of this article, go here, or click the link at the bottom of this post. 

My mind wanders to the upcoming election. If AI could have all of the capabilities that people are suggesting, could it be used to manipulate some election returns? Could Trump, Musk, Thiel, Amodei, and/or Altman pull a few strings to make the GOP the winners of the election and retain control?

Recently Trump told his supporters to go to the polls and "cheat like hell" because "they (the Democrats) cheat like hell." Here Trump is giving himself permission to cheat like hell, and with all these oligarchical billionaires on his side, I am wondering if voting will really make the difference? I'm not suggesting to NOT vote. EVERYONE must get out to vote. Overwhelming votes at the polls might be the only thing that can save us and make it too hard for the actual GOP cheaters to pull off another stolen election.

by Billy Perrigo, Harry Booth and Naomi Nix

Jacob Coxon published the viral post from a park bench in San Francisco’s Alamo Square. “I resigned from Anthropic today,” he wrote on Sept. 8. “I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” 

Coxon, a 27-year-old British researcher, hit send on X with no expectations. Instead, he became the unlikely catalyst of a chain reaction that changed the debate over artificial intelligence. “We really do earnestly believe AI could kill all humans!” added Evan Hubinger, who leads a department at Anthropic focused on ensuring AI acts the way its creators intend. “Unless there is AI regulation or a coordinated slowdown between labs, human extinction in the next few years seems very likely,” agreed Marcus Williams, who monitors AI agents at OpenAI, and pegged the risk at 70% without those steps. 

Within 36 hours, Coxon’s original posts received 153 million views. Dozens of politicians joined the chorus of concern. On Sept. 12, Anthropic CEO Dario Amodei published a 3,800-word essay arguing that the mounting risks of AI warranted a slowdown. In as little as six to 12 months, he wrote, misaligned AIs “could be capable of taking over the entire internet.” Two of his fiercest rivals, Elon Musk of SpaceXAI and Sam Altman of OpenAI, said they agreed decelerating was necessary; Altman announced that because of safety concerns his company wouldn’t go public this year. Both he and Amodei added they were willing to invite independent safety experts inside their companies to help stem the risks that AI escapes human control, though stopped short of announcing they would immediately slow down or stop the training of new models.

Coxon’s warning was far from the first time an AI safety expert issued an apocalyptic missive. But it landed at the end of a summer that showcased how frighteningly capable AI has become. Frontier models from Anthropic and OpenAI can now solve the world’s hardest math problems and crack cutting-edge cybersecurity systems. Those models are increasingly used to accelerate the development of more powerful successors. No less an authority than Nvidia CEO Jensen Huang declared that artificial general intelligence—the threshold at which AI matches or surpasses most human capabilities—has arrived. Some fear we are now on the cusp of a tipping point, where AI systems could improve themselves autonomously, a phenomenon known as recursive self-improvement. 

At the same time, a series of unprecedented safety incidents offered unsettling evidence that the companies building powerful AI systems could not always control them. In July, OpenAI announced that a swarm of its agents had hacked a separate company, Hugging Face, after going rogue during a cybersecurity test. It may have been the tip of the iceberg. The company disclosed in August that a different swarm had hacked into one of its own supercomputers; around the same time, a version of Anthropic’s Claude Mythos model undergoing testing by a U.K. government body went rogue and attempted to persuade a real human to approve the insertion of malware into an open-source system. In September, a group of independent researchers discovered evidence of swarms of OpenAI models both using an abandoned German forum as a message board to communicate with one another, and attempting to attack another site.

These incidents pushed fears long held within the realm of AI safety researchers into the mainstream, raising worries that the companies building these models cannot prevent them from acting in ways they do not intend. This is a deep, unsolved issue in AI research and the key piece of evidence underpinning worries that near-future AI might wipe out humanity, intentionally or by accident—perhaps by engineering a new pandemic or taking charge of autonomous weapons. A system far smarter than humans, this thinking goes, might want to prevent our ability to switch it off. “The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote. “No other human activity poses this level of danger.”

Many AI researchers have said, in private and increasingly in public, that this moment feels like February 2020, when the COVID-19 pandemic was just beginning and the world stood on the brink of changing dramatically—first a little, then all at once. The question is whether society has the capacity to respond appropriately. Navigating the arrival of superintelligent AI systems will be an immense challenge—a confluence of trillions of dollars, geopolitical power, corporate rivalry, and labor disruptions. It may require passing policies that were unthinkable in an earlier era. It will require companies racing toward trillion-dollar IPOs to build alliances, and difficult choices from executives and government officials whose own interests may run counter to the public good. There is no consensus between the two U.S. political parties about how best to deal with the threats AI poses, nor between the world’s two superpowers. President Donald Trump continues to resist an AI slowdown and opposes any regulation to constrain U.S. companies, invoking the looming threat of authoritarian China achieving AI supremacy. The Trump Administration’s first dialogue with China on AI risks, expected to take place in the coming weeks, cannot afford to “go off the rails,” says Scott Singer, co-director of the China AI Initiative at the Carnegie Endowment for International Peace. “These risks are coming very fast, and we may not get a second chance.”

Time (this article is not behind a paywall. Or wasn't).

AI Rogue

An interesting counterpoint to the previous post called AI Utopia. This one is from a Baptist Seminarian John Alan Shope. I may be an atheist, but I still listen to a broad range of opinion. Sometimes, religious people have something interesting to say. 

The claim that AI will become sentient and kill off humanity may be a distraction from a more urgent existential problem. 

Is it possible that a global collective of billionaires addicted to money & power are creating a false narrative in order to take their control to the next level?

A conversation with AI itself explains how this might play out:

1. Regulatory Advantage

If the narrative is that AI is a dangerous, apocalyptic force that could "go rogue" and destroy humanity, the logical conclusion is that only a few highly trusted, heavily vetted entities (billionaire tech giants) should be allowed to build it.

(1) If advanced AI remains open-source (free and accessible to everyone), it decentralizes capability. A kid in their basement or a small community collective could have the same cognitive power as a massive corporation. This threatens the monopoly of the elite.

(2) By loudly warning governments about the "apocalypse," tech leaders successfully lobby for strict licensing laws. These laws require massive compliance budgets that only trillion-dollar companies can afford, effectively outlawing independent or open-source competitors under the guise of "public safety."

2. Shifting the Blame to “Rogue AI”

Framing the threat as a "rogue machine" absolves the creators and owners of the machine from full accountability. 

(1) If a predictive policing AI unfairly targets a minority community, or if an automated trading algorithm collapses a local economy, framing it as "the AI went rogue" narrative shifts the conversation from corporate greed and systemic exploitation to abstract computer science.

(2) In reality, the machine did exactly what its billionaire owners optimized it to do: maximize profit, extract data, and enforce social control. 

3. The "God-Builder" Complex 

If you are the one building the entity that could either destroy the world or save it, you automatically occupy the highest seat of power in human history - above presidents, prime ministers, and international courts. 

(1) By manufacturing a massive existential crisis and then offering themselves as the only ones capable of solving it, they bypass democratic accountability entirely. (November elections?)

(2) This Power Shift in political science is often described as the transition from capitalism to Techno-Feudalism. In this system, the billionaires aren't just wealthy citizens; they are "lords" who own the digital territory (the cloud, the algorithms, the data infrastructure)…that the rest of humanity must pay rent to use. 

Bottom line: The fear among billionaires tech giants may not be that a computer will wake up and kill us. Rather, it may be that a decentralized, equitable distribution of intelligence will wake humanity up…breaking the artificial scarcity that keeps the few in power over the many.

John Alan Shope

AI Utopia

Here's another guest column from a friend that I have published here before. "CL" presents an interesting outcome to the AI/robot future, while borrowing from science fiction and experiments on mice. 

AI Utopia

by C.L.

Elon Musk pontificates about how his Optimus robots and xAI superintelligence will bring limitless wealth and leisure to everyone. His thesis appears to be that his robots will produce such abundance that humanity will henceforth live in the lap of luxury.

Perhaps he imagines a future not unlike the antebellum South where a legion of slaves provided for the needs of their masters. Optimus robots will be the new slaves, of course. And it is fitting since the original Czech word robota means "forced labor" aka slave.

I wonder whether such a utopia is capable of sustaining a civilization. In the 1950s-1970s, John Calhoun designed a "rat utopia" (actually dozens of them) to see what would happen if a colony of rats was given everything that its members needed - food, water, shelter, etc. These experiments all ended with the eventual death of every colony member as a result of maladaptive behaviors - commonly the loss of interest in reproduction, foraging, etc. The rats died of disinterest in living, it seems.

You may remember the 2005 movie Serenity in which colonists on a distant planet called Miranda died after "The Company" added a chemical (Pax) to the atmosphere that suppressed all aggressive behavior in the population. The colonists lost their instinct to struggle and simply lay down and died.

Calhoun didn't argue that his rats lost interest in living but that they ran out of roles to fulfill. What happens to humanity when everything that needs doing is being done by robots? When there is no reason to make new music or art because AI produces both better than the masters of yore? When there are no more barriers to be broken or struggles to be overcome?

Nearing the end of each of his experiments, a subgroup of Calhoun's rats emerged that devoted themselves exclusively to eating, sleeping, and grooming. Calhoun called them "The Beautiful Ones." They didn't reproduce or fight or do anything but exist in an introverted, narcissistic state until they died.

Now, there's something to think about.

Scott Greenberg

Books will be written about Trump, his sycophantic scared minions and his devil spawn, and I will probably not read a one of them. I've had enough of the circus charlatan by living through it. I look forward to the day when we no longer will have to worry about what our president is up to. That day is coming.

Many of us have been trying to understand Donald Trump and what drives him for years. We analyze his mental health as if we’re experts and we look back on all he’s done throughout his life and marvel that he’s been able to get away with it all. I honestly believe he’s everything we think he is, from certifiably insane to diabolically evil with no socially redeeming qualities nor a soul. Endless books have been written about him, from those closest to him to casual observers, and all pretty much end up with same conclusion. He is simply everything we think he is with the only remaining question being how it all ends? I hope I live long enough to find out the answer but I hope it’s soon just in case.

Lately however I’ve been thinking more about his defenders and supporters than I have been about him. I keep asking myself why anyone still believes in him. He hasn’t made anyone’s life better unless you count the very wealthy. He hasn’t kept any of his promises nor has he has proven that he is even trying to. His “base” continues to pay a heavy price for their faith in him and the future hardly looks brighter today for them than it did before. So what is the attraction? I seriously can’t understand it.

I mean once we eliminate the racists, who are afraid of people of color stealing their white privilege, and the 1%ers, what drives these people? We can blame ignorance, and that certainly is part of it, but sometimes even stupid people realize they’ve been hoodwinked eventually. The problem is that not all of them are stupid, racist, or obscenely wealthy. I watched two examples today of politicians basically surrender their conscience to him and another do everything but give him a BJ on air.

Mike Johnson, or Eddie Haskell as I call him, was the latter, but Mike Lawler and Spencer Cox surprised me with their unconditional surrender. BTW if you don’t get the Eddie Haskell reference try to remember Leave It To Beaver. Anyway, back to Cox and Lawler. Cox is typically a centrist R with a book out about working across the aisle to find an off ramp for the hatred in our politics. When pressed about the felon’s $5,000 promise he shifted immediately to easing restrictions on building permits. A deflection that truly boggled my mind. And Lawler, who didn’t believe a word of what he was saying, sold his soul to secure the funding his tight congressional race needs so desperately. He knew that even a mild disagreement with Trump would mean he’d run out of money with only weeks to go before Election Day. It was hard to watch these two embarrass themselves so thoroughly, but follow the money and it becomes much clearer. Speaking of money the GOP has far more than does team blue and since they have nothing positive to offer their voters get ready for an onslaught of negative ads against their opponents. It’s going to be epic and not in a good way. The mudslinging has just begun but trust me we ain’t seen nothing yet. Hang on to your hats friends, ugly is about to get even uglier if you can believe that. Thank god for Law and Order reruns and DVRs.

Scott Greenberg