“I’m sort of a complex chaotic systems guy, so I have a low estimate that I actually know what the nonlinear dynamic in the memosphere really was,” he said. (Translation: It’s complicated.)
Why do these people have the urge to talk like this? Does it make themselves feel smarter? Do they think it makes them look smart to other people? Are they so caught up in their field they can’t code switch to normal person talk?
Here: https://glowfic.com/posts/4508
Be warned, the three quarters of the thread don’t have much of a plot and are basically two to three characters talking, then the last quarter time skips ahead and gives massive clunky worldbuilding dumps. (This is basically par for the course with glowfic, the format supports dialogue interaction heavy stories and it’s really easy to just kind of let the plot meander. Planecrash, for all of its bloat and diversions into eugenics lectures, is actually relatively plot heavy for glowfic.)
On the upside, the first three quarters almost read like a sneer on rationalists.
We’ve sneered about dath ilan before on the reddit sneerclub, and occasionally I work references to dath ilan’s lore into sneers, but other than that no.
Where else am I supposed to fine deep analyses of the economic implications of 1st level wizards and clerics on an early modern setting? and analyses of Intelligence score distributions across the nations of Golarion?
You are close! It is a bdsm AU (inspired by an Archive of Our Own Trend of writing alternate universe settings of a particular flavor), i.e. everyone identifies as “Dominant” or “Submissive”, and that identification is more important than gender in most ways. Ironically the dath ilan character is the one freaked out by this.
I mean the aftermath of Buterlian Jihad eventually lead to brutal feudalism that lasted a really long time and halted multiple lines of technological and social development, so I wouldn’t exactly call it a success for the common person.
It would be only light cringey if he kept it as a fiction writing project not strongly linked to his other writings. But as Architeuthis says, he not only links to it frequently, he cites it like it was a philosophy paper or something that obviously everyone should have read. Example from our old reddit discussions: https://www.reddit.com/r/SneerClub/comments/x3pihv/to_be_fair_you_have_to_have_a_very_high_iq_to/
Thanks for the lore, and sorry that you had to ingest all that at some point.
Ironically, one of the biggest lore drops about dath ilan happens in a story I initially thought at the time was a parody of rationalists and the concept of dath ilan (Eliezer used a new penname for the story). The main dath ilan character (isekai’d into an Earth mostly similar to our own but with magic and uh… other worldbuilding conceits I won’t get into here) jumps to absurd wild conclusion throughout basically every moment of the story, and unlike HJPEV is actually wrong about basically every conclusion she jumps to. Of course, she’s a woman, and it comes up towards the ending that she is below average for dath ilan intelligence (but still above the Earth average, obviously), so don’t give Eliezer too much credit for allowing a rationalist character to be mostly wrong for once.
I don’t know how he came up with the name… other fanfic writers in rationalist-adjacent space have complained about his amateurish attempts at conlanging, so there probably isn’t a sophisticated conlang explanation about phonemes involved. You might be on the right track guessing at weird anagrams?
Hey, it was Caroline Ellison who wanted to be part of the Imperial Chinese Harem! Also, judging by Dragon Army and Leverage Research. the communes aren’t anarchist! They are tightly controlled by a single great man type leader.
In Eliezer’s “utopian” worldbuilding fiction concept, dath ilan, they erased their entire history just to cover up the any mention of any concept that might inspire someone to think of “superintelligence” (and as an added bonus purge other wrong-think concepts). The Philosopher Kings Keepers have also discouraged investment and improvement in computers (because somehow, despite now holding any direct power and the massive financial incentives and dath ilan being described as capitalist and libertarian, the Keepers can just sort of say their internal secret prediction market predicts bad vibes from improving computers too much and everyone falls in line). According to several worldbuiding posts, dath ilan has built an entire secret city that gets funded with 2% of the entire world’s GDP to solve AI safety in utter secrecy.
Given the USA legislature’s incompetence, I imagine they would leave some sort of massive loopholes. Depending on the exact wording, you could get around it with technically not GPUs (as you suggest), or subdividing companies so each subdivision can be technically under the limit, or cranking up the size of individual GPUs so 8 GPUs is a massive amount of compute. Of course, I really doubt it would get that far in the first place, look at how they killed California’s attempt at the most moderate AI legislation.
It’s a microcosm of lesswrong’s dysfunction: IQ veneration, elitism, and misunderstanding the problem in the first place. And even overlooking those problems, I think intellect only moderately correlates with an appreciation for science and an ability to understand science. Someone can think certain scientific subjects are really cool but only have a layman’s grasp of the technical details. Someone can do decently in introductory college level physics with just a willingness to work hard and being decent at math. And Eliezer could have avoided tangents about nuclear reactors or whatever to focus on stuff relevant to AI.
There’s also Eliezer’s nihilistic outlook, which is deftly woven into his parables-- his personal philosophy draws heavily from Godel Escher Bach, for instance. The fans understand this stuff; they have the intellectual capacity to truly appreciate the depths of his parables, to realize that they’re not just entertaining- they say something deep about the nature of Intelligence. As a consequence people who dislike IABIED truly ARE idiots- of course they wouldn’t appreciate, for instance, the motivation in Eliezier’s existencial catchphrase “Tsuyoku Naritai!,” which itself is a cryptic reference to Japanese culture. I’m smirking right now just imagining one of those addlepated simpletons scratching their heads in confusion as Nate Soares genius unfolds itself on their copy of IABIED. What fools… how I pity them. 😂 And yes by the way, I DO have a rationalist tattoo. And no, you cannot see it. It’s for the math pet’s eyes only- And even they have to demonstrate that they’re within 5 IQ points of my own (preferably lower) beforehand.
Yeah, in some hypothetical bizarro universe where they get their 8 (cutting edge) GPU limit actually passed, I bet the monitoring scheme would be loose in a way that Microsoft just has a box to tick while strict enough that it imposes untenable costs on private individuals.
It was nice to see the call out that doomers and accelerationists are two sides of the same coin and both serving the interests of the LLM makers.
Side note, (maybe I should make a top level post), but even within the lesswronger/doomer community there are a lot of people unimpressed to outright critical of Eliezer’s book. Apparently it has a lot of rambling, kind of off topic, and/or condescending parables. And key pieces of the AI-doom argument are kind of not explained or even elaborated on.
I hadn’t heard of Black Lotus. Also, the article fails to mention rationalist/lesswrong ties to that AI-doom-focused Zen Buddhism cult that was discussed on Lesswrong recently (looking it up, the name is Maple), so you can add that to the cult count.
That discussion is here: https://www.reddit.com/r/SneerClub/comments/1ka6pvb/unsong_a_study_in_misrepresentation/
The author occasionally posts to slatestarcodex, we kind of tried to explain what was wrong with Scott Alexander and I think she halfway got it?
Poor historical accuracy in favor of meme potential is why our reality is so comically absurd. You can basically use the simulation hypothesis to justify anything you want by proposing some weird motive or goals of the simulators. It almost makes God-of-the-gaps religious arguments seem sane and well-founded by comparison!
Within the world-building of the story, the way the logic is structured makes sense in a ruthless utilitarian way (although Scott’s narration and framing is way too sympathetic to the murderously autistic angel that did it), but taken in the context outside the story of the sort of racism Scott likes to promote, yeah it is really bad.
We had previous discussion of Unsong on the old site. (Kind of cringing about the fact that I liked the story at one point and only gradually noticed all the problematic stuff and poor writing quality stuff.)
I wonder exactly when this was. The attempted oust of Sam Altman was November 17, 2023. So either this warning was timely (but something Sam already had the pieces in place to make a counterplay against), or a bit too late (as Sam had recently just beaten an attempt by the true believers to oust him).
Sam Altman has proved adept at keeping the plates spinning and wheedling his way through various deals, I agree with the common sentiment here that he his underlying product just doesn’t work well enough, in a unique/proprietary enough way for him to actually use that to get profitable company. Pivot-to-AI and Ed Zitron have a guess of 2027 for the plates to come crashing down, but with an IPO on the way to infuse more cash into OpenAI I wouldn’t be that surprised if he delays the bubble pop all the way to 2030, and personally gets away cleanly with no legal liability for it and some stock sales lining his pockets.