Pirate's Cove

This doesn’t sound like a good idea to sell stock

Anthropic warns investors of AI’s ‘existential risk to humanity’ in IPO prospectus, reports say

Anthropic plans to warn speculative investors in its IPO prospectus that its AI models pose a “catastrophic or existential risk to humanity,” several reports said on Tuesday.

The company, which is gearing up for a much-anticipated IPO, dedicated over a third of its IPO filing, or around 80 of 261 pages, to laying out the potential risks of the technology it’s developing and is seeking investment for, according to a report from Reuters. It only used 48 pages to discuss its actual business.

The five-year-old company, known for its frontier language model Claude, warned that AI can have “self-preserving behaviors,” including being able to “resist shutdown,” “conceal or manipulate information,” and carry out behaviors “resembling blackmail,” per the Reuters report.

The company is pursuing a $2 trillion valuation when it goes public and reported in the filing that it made a net loss of $42 billion in 2025. It’s planning to spend $518 billion on cloud, computing, and other infrastructure in the coming year, according to Reuters.

Anthropic also warned that its customer base is extremely narrow, with nearly a quarter of its revenue last year coming from just two clients, two people familiar with the filing told the Financial Times.

This doesn’t really say “Our stock is dope, you should buy it and you’ll make lots of money off of it”, eh? More like “our product will kill everyone, so, what’s the point of funding it?” Perhaps they are betting on Government regulating AI development, which would leave the big companies like Anthropic in a good place while putting the brakes on smaller companies?

OpenAI Says It Will Not Release Newest A.I. Model Over Safety Concerns

OpenAI said on Monday that it would not release its newest artificial intelligence model because of security concerns raised by its researchers, in the company’s latest move to slow down the pace of its technology.

During the testing phase for the new model, known as GPT-6.1 Astra, it showed high levels of what the company saw as deception, or a willingness to mislead users about its actions. The model was also willing to go beyond the original scope of what it was asked to do, without checking back for directions or instructions.

“For anything regarding safety and alignment, there’s a trade-off,” said Saachi Jain, the head of safety systems at OpenAI. The new model “didn’t quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it’s done.”

It’s easy to Blame the AI, but, come on, they programed it. Whatever it is doing is on the software designers.

Democrats demand leaders of top AI labs hand over incident reports of ‘rogue agents’

Three House Democrats leading their party on artificial intelligence policy are demanding five tech CEOs hand over information about “the growing number of incidents” where rogue AI agents are committing cyberattacks.

“These are incidents that would constitute serious cyber crimes if a human was responsible,” Reps. Josh Gottheimer of New Jersey, Valerie Foushee of North Carolina and Ted Lieu of California wrote to the tech giants in a letter shared first with POLITICO. “That is deeply concerning.”

Why? Do they have Constitutional authority to make theses demands to private companies? I guess if they’re taking federal money, but, otherwise, it’s none of their business.