The arguments made by AI safety researchers Eliezer Yudkowsky and Nate Soares in If Anyone Builds It, Everyone Dies are superficially appealing but fatally flawed, says Jacob Aron
LLMs and humans are both sentence-producing machines, but they were shaped by different processes to do different work
Except not really. We’re not sentence-producing machines, we’re “machines” (so to speak) that can produce sentences. Not the same thing.
Once this is in place, they say, nations must be prepared to enforce these restrictions by bombing unregistered data centres, even if this risks nuclear war, “because datacenters can kill more people than nuclear weapons” (emphasis theirs).
So the plan is still to kill everyone to death to prevent GPT-5678 …
Except not really. We’re not sentence-producing machines, we’re “machines” (so to speak) that can produce sentences. Not the same thing.
So the plan is still to kill everyone to death to prevent GPT-
5678…