Sometimes people say that it's much less valuable to do AI safety research today than it will be in the future, because the current models are very different—in particular, much less capable—than the models that we're actually worried about. I think that argument is mostly right, but it misses a…

One risk of “vibe-coding” a piece of software with an LLM is that it gets you 90% of the way there, but then you’re stuck—the last 10% of bug fixes, performance improvements, or additional features is really hard to figure out because the AI has written messy, verbose code that both of you struggle…

isolated demand for rigor

dynomight.net/blood In a recent post about trading stuff for money, I mentioned: Europe had a [blood plasma] shortage of around 38%, which it met by importing plasma from paid donors in the United States, where blood products account for 2% of all exports by value. The internet’s reaction was: “TWO…

For months, I had the feeling: something is wrong. Some core part of myself had gone missing. I had words and ideas cached, which pointed back to the missing part. There was the story of Benjamin Jesty, a dairy farmer who vaccinated his family against smallpox in 1774 - 20 years before the…

[previously in series: 1, 2, 3, 4, 5] When that April with his sunlight fierce The rainy winter of the coast doth pierce And filleth every spirit with such hale As horniness engenders in the male Then folk go out in crop tops and in shorts Their bodies firm from exercise and sports And men gaze at…

[previously in series: 1, 2, 3, 4] It has been three weeks since Sam Altman was fired, but the conversation won’t move on. “What did Ilya see?” asks your Uber driver, on the way to the airport. “What

[previously in series: 1, 2, 3] You spent the evening agonizing over which Bay Area House Party to attend. The YIMBY parties are always too crowded. VC parties were a low-interest-rate phenomenon. You

[Previously: Every Bay Area House Party, Another Bay Area House Party] People talk about “fuck-you money”, the amount you’d have to make to never work again. You dream of fuck-you social success, wher

[Previously: Every Bay Area House Party] Blaise Pascal said all human evil comes from inability to sit alone in a room. Your better nature - your rational soul - tells you that nothing good has ever c

Some of the more unhinged writing on superintelligence pictures AI doing things that seem like magic. Crossing air gaps to escape its data center. Building nanomachines from simple components. Plowing through physical bottlenecks to revolutionize the economy in months. More sober thinkers point out…

dynomight.net/money Examples are good. Let’s start with some examples: We all need kidneys, or at least one kidney. Donating a kidney sucks, but having zero working kidneys really sucks. Paying people for kidneys would increase the number available, but it seems gross to pay people for part of their…

We’re not the only people who get confused by OpenAI model names. There’s actually a simple explanation: most of the models are modifications of each other in a giant dependency graph that OpenAI is keeping secret. In this post we’ll dive into explaining our best guess on how that graph looks.1…

For a lay audience, but I've seen a surprising number of knowledgeable people fretting over depressed-seeming comics from current systems. Either they're missing something or I am. Perhaps you’ve seen images like this self-portrait from ChatGPT, when asked to make a comic about its own experience.…

Our universe is probably a computer simulation created by a paperclip maximizer to map the spectrum of rival resource‑grabbers it may encounter while expanding through the cosmos. The purpose of this simulation is to see what kind of ASI (artificial superintelligence) we humans end up creating. The…

Comp sci in 2017: Student: I get the feeling the compiler is just ignoring all my comments. Teaching assistant: You have failed to understand not just compilers but the concept of computation itself. Comp sci in 2027: Student: I get the feeling the compiler is just ignoring all my comments. TA:…

With an Elo rating of 2706, OpenAI’s o3 model is better than 99.7% of competitive programmers on CodeForces (based on this rating distribution data from 2024). Humans with this performance would excel at any top tech company, whereas o3 is not ready to automate even the median big-tech engineer’s…

With an Elo rating of 2706, OpenAI’s o3 model is better than 99.7% of competitive programmers on CodeForces (based on this rating distribution data from 2024). Humans with this performance would excel at any top tech company, whereas o3 is not ready to automate even the median big-tech engineer’s…

Britain today celebrates the crowning of a new king, but the coronation will be watched and celebrated by millions across the Commonwealth. To an extent that is often not appreciated abroad, the Queen – who was Defender of the Faith – was revered by her subjects of all faiths. In our often sectarian…

If you don’t have time to read this post, these five graphs give most of the argument. Each includes both the energy/water cost of using ChatGPT in the moment and the amortized cost of training GPT-4: Source for original graph. Each bar here represents 1 year of the activity, so the live car-free…

Image: Cursor In a viral video posted on X, an 8-year-old girl happily “codes” away. Over the course of 45 minutes, she has Cursor, an AI coding assistant, build a chatbot that replies in the style of Harry Potter. Using only simple text-based prompts, she asks Cursor to create a website, add a…

The Eve of the Deluge. (John Martin) (Part 1 here) The AGI frog is getting boiled As mentioned in part 1, a brief market correction happened in late 2026. In 2027, OpenAI releases o7 to try to shore up excitement and new investments. It’s much more reliable than o6 and can now do a lot of office…

The Great Day of His Wrath. (John Martin) (Part 1 here, part 2 here) The end of white-collar work and the new job scene By the late 2020s, office jobs in developed countries are now basically about overseeing and providing direction to AI systems, and the last part of that is mostly on-paper rather…

Below is part 1 of an extended scenario describing how the future might go if current trends in AI continue. The scenario is deliberately extremely specific: it’s definite rather than indefinite, and makes concrete guesses instead of settling for banal generalities or abstract descriptions of…

Most people think of AI as a pattern-matching chatbot – good at writing emails, terrible at real thinking. They've missed something huge. In 2024, while many declared AI was reaching a plateau, it was actually entering a new paradigm: learning to reason using reinforcement learning. This approach…

This post should not be taken as a polished recommendation to AI companies and instead should be treated as an informal summary of a worldview. The content is inspired by conversations with a large number of people, so I cannot take credit for any of these ideas. For a summary of this post, see the…

The AI Control Agenda, in its own words: … we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to…

As noted further above, most the advice above is selected from the resources linked below (the list is in no particular order):

habryka Hey Everyone! As part of working on dialogues over the last few weeks I've asked a bunch of people what kind of conversations they would be most interested in reading, and one of the most comm

The morning after had come, and all the students had gathered silently around the four Tables of Hogwarts, Harry James Potter-Evans-Verres among them. He had collapsed in exhaustion last night and been awoken in the infirmary next morning, still muzzy, with the Philosopher's Stone underneath his…

If an unaligned entity invests billions of dollars into an application which you use, where they benefit from wasting your time, and you haven’t at least done a cost-benefit analysis so that your usage minimizes your costs and maximizes your benefits—You are probably getting fucked over. Mistake:…

dynomight.net/theanine-2 Theanine is an amino acid that occurs naturally in tea. Many people take it as a supplement for stress or anxiety. It’s mechanistically plausible, but the scientific literature hasn’t been able to find much of a benefit. So I ran a 16-month blinded self-experiment in the…

And it was evening and it was morning, the last day. June 15th, 1992. The beginning light of morning, the pre-dawn before sunrise, was barely brightening the sky. To the east of Hogwarts, where the Sun would rise, that faintest tinge of grey made barely visible the hilly horizon beyond the Quidditch…

Apr 2025 Comments at lemmy, substack. I don’t know how to internet, but I know you’re supposed to get into beefs. In the nearly five years this blog has existed, the closest I’ve come was once politely asking Slime Mold Time Mold, “Hello, would you like to have a beef?” They said, “That sounds great…

When I started graduate school (for math), I was very interested in big ideas. I had had a couple experiences of having general research intuitions pan out really well and felt like the core of good research is having a brave idea, a gestalt. I went into grad school looking for the “gestalt people”.…

This is a cross-post from https://250bpm.substack.com/p/accountability-sinks Back in the 1990s, ground squirrels were briefly fashionable pets, but their popularity came to an abrupt end after an incident at Schiphol Airport on the outskirts of Amsterdam. In April 1999, a cargo of 440 of the rodents…