• New hack by me, worldpieces.
• New Shadertoy hacks brought into the fold: bestill, bubblecolors, darktransit, downfall, driftclouds, goldenapollian, noxfire, prococean, rigrekt, trainmandala, trizm and universeball.

Shadertoys are so wild; you'll see like 30 lines of code, and when it runs it build an entire environment without there being a polygon or a model apparent anywhere. What witchcraft is this?? Well, here's a really good article that explains the techniques used: Painting With Math: A Gentle Study of Raymarching.

I fixed a bunch of Android bullshit, too (some of which meant needing to reimplement glRotatef etc. from first principles). Android's implementation of GLES is a buggy mess. Also I think Android has again lost the ability to ask for permission to load photos. I can't figure it out, so someone who gives a shit will have to send me a patch. I also still can't figure out why transparency doesn't work on Android. This makes Peepers be particularly horrifying.

Map Scroller: I updated the list of available maps, and made it show the name of the nearest city. One of the new map sets that works is the Google Satellite Map, and if you have two monitors, I highly recommend running that map in "Fully random location" mode. It has a habit of picking two places many thousands of miles apart that look surprisingly similar, like Greenland and Syria.

This one took so much time! Most of my screensavers are pretty quick; generally they'll percolate in my head for a while, then I do the first 90% in half a day, and then the second 90% in another half day to two days. But this one was composed entirely of ratholes.

1. Oh, first I have to find some sane way to get the country outlines. That means learning about GeoJSON and getting the proper data set from Natural Earth Data. Then that data has to be massaged and merged, and while there's probably some command-line way to do that, using mapshaper.org was easier.

2. Now I've got a bunch of polylines with weird "hole" rules (not winding-rule!) and I need to triangulate them. Next rathole: learning how to use triangle.c correctly.

3. Then I notice that while the GeoJSON has population data for countries, it does not for states and provinces. Ok, that should be easy to grab from Wikipedia, right? Next rathole: learning about SparQL, the Wikidata query language. It is one of the nastiest and most baffling query languages I've seen. Good job on that.

4. Then I realize that while the GeoJSON contains the names of the countries translated into various languages, it doesn't tell you which ones are the official languages! Next rathole! Ok, surely we can get that from Wikidata too... Nope. While the Wikipedia pages contain the endonyms in the local character set, the Wikidata items only have them transliterated into Latin characters. To fix that, I had to scrape the Wikipedia page and parse the Wiki markup using regexps, dooming us to inhuman toil, etc. etc. Anyway, I got it working so that Japan can be spelled 日本...

5. ...on macOS, iOS and Android. Why doesn't it work on Linux? Oh ho ho ho, another rathole, this one un-solved! So on Linux it notices when all of the characters came out as square boxes and falls back to the transliterated Latin versions. You're welcome.

6. Ok, it's starting to come together. But since I'm zooming in a lot, the Earth imagery is looking a little grainy, since the image I have is 2048x1024. Well let's just go upgrade that. Next rathole! The old images I had were from the public domain 2002 NASA "Blue Marble" images and 2000 "Visible Earth, City Lights" images none of which still exist on NASA's site (all praise archive.org!)

   They seem to have deprecated those data sets in favor of the 2025 "Blue Marble: Next Generation" images and the 2012 "Black Marble" images. But... they're weird. First, both sets of images completely omit the North polar ice. Second, the 2025 daytime oceans are colored completely flat, whereas the 2002 images contained bathymetry details. And third, and worst of all, is no pairing of 2025 day images and 2016 night images where the pair have the same ice extents. This means that when we blend between the day and night images, it looks very weird. It took me days to finally accept that there was nothing that I could do to work around that.

7. So now that those various ratholes are out of the way, it's trigonometry time. The way the countries pop up and face forward toward whereever the camera is a technique called "billboarding" and it's pretty easy. But what's not easy is doing a smooth transition between the object being billboarded, and the object being back in the scene. Especially since it's position in the scene is spinning and wobbling about willy-nilly. (Remember, you can also spin the globe with the mouse, and it has to keep track of that as well!)

8. So the final rathole was finally coming to terms with quaternions. Quaternions and I have had a passing acquaintance for many years, but I finally had to get down and dirty and really understand them to make all of this work.

   Wikipedia describes quaternions as:

   Quaternions form a four-dimensional associative normed division algebra over the real numbers, and therefore a ring, also a division ring and a domain. It is a special case of a Clifford algebra, classified as Cl₀,₂(ℝ)≅Cl⁺₃,₀(ℝ). According to the Frobenius theorem, the algebra ℍ is one of only two finite-dimensional division rings containing a proper subring isomorphic to the real numbers; the other being the complex numbers.

   "I did not understand a math thing, and then I read the Wikipedia article on it, and now I understand the math thing" -- said literally no one ever. As far as I can tell, every math article on Wikipedia is written with the target audience of "person who wrote their grad school thesis on it" and no one else. Wikipedia's math articles are absolutely fucking useless for purposes of, you know, education.

   Anyway, 3Blue1Brown has a good intro video that is not like that!

   By the way, I propose that "Quaternions" should be pronounced like extruded cornmeal toroid "Funyuns".

Previously, previously, previously.

Surprisingly and happily my last post on version control got picked up by Hacker News and got a lot of views. Thanks everybody who engaged with it, and welcome new subscribers. I put a ridiculous amount of work into these things and it’s nice when there’s something to show for it.

Something I didn’t quite realize when writing that last post is that in addition to supporting ‘safe rebase’ by picking one of the parents to be the ‘primary’ one, it’s possible to support ‘safe squash’ by picking a further back ancestor to be the ‘primary’ one. The advantage of this approach is that it gives you strictly more information than the Git approach. If you make the safe versions of blame/history always follow the primary path then if you perform all the same commands in the same order with both implementations then the outputs can be made to look nearly identical, with some caveats about subtle edge cases where the safe versions are behaving more reasonably. But the safe versions will still remember the full history, which both lets you look into what actually happened and gives you a lot less footguns related to pulling in something which was already squashed or rebased. The unsafe versions of these things literally throw out history and replace it with a fiction that whoever did the final operation wrote everything, or that the original author wrote something possibly very divergent from what they actually wrote.

Git is very simple, reliable, and versatile, but it isn’t very functional. It ‘supports’ squash and rebase the way writing with a pen and paper ‘supports’ inline editing. It implicitly makes humans do a lot of the version control system’s job. It’s held on because the more sophisticated version control systems haven’t had enough new functionality to compensate for their implicit reductions in versatility and reliability. My goal is to provide the foundation of something which tells a compelling enough story to be worth switching to. I think the case is good: At the minor cost of committing to diffs at commit time you can have safer versions of squash and rebase which Just Work, plus a better version of local undo for the occasional times when you encounter that nightmare and a pretty good version of cherry-picking as a bonus.

‘Committing to diffs at commit time’ is a subtle point. From a behavioral standpoint it’s a clear win. But it does create some implementation risk. Any such implementation needs a core of functionality which is is very well tested and audited and only updated extremely conservatively. This is why I made my demo implementation as simple as possible within the constraints of eventual consistency. Other systems which claim to do this I couldn’t understand their technical docs, much less develop intuitions of how they behave. With the ‘conflicts are updates which happen too close together’ approach I can intuitively reason about the behavior.

Some people found the ‘left’ and ‘right’ monikers confusing. Those are entirely advisory and can be replaced with any info you about the branch name it came from and whether it’s local or remote, or even blame information.

The ‘anchoring’ algorithm for CRDTs has been invented multiple times by independent groups in the last few years. I find it heartening that it’s only been in the last few years because I feel I should have come up with it over twenty years ago. Oddly they don’t seem to have figured out the generation counting trick, which is something I did come up with over twenty years ago. Combining the two ideas is what allows for there to be no reference to commit ids in the history and have the entire algorithm be structural.

One implementation detail I didn’t get into is that in my demo implementation it doesn’t flag ‘conflict’ sections which only consist of deletions with no insertions on either side. I’ve discussed the semantics of this with people and it seems more often than not people view flagging those as excessively conservative and are okay with them clean merging to everything gone. This is no doubt something people can get into religious wars over and it would be very interesting for someone to collect actual data about it but in practice it will inevitably be a flag individual users can set to their liking and the argument is really about the default.

One interesting question about version control is whether it can ever merge together two branches which look the same into a version which looks like neither parent. The approach I’ve proposed avoids doing that in a lot of practical cases which blow up in your face pretty badly, but that doesn’t mean it never does it. For example if you start out with XaXbX and one branch goes XaXbX → aXbX → XX while the other branch goes XaXbX → XaXb → XX. Then when you merge them together the first branch has deleted the first X of three while the second branch has deleted the last X of three so they’ll clean merge into a single X. This is a fairly artificial example which it would be hard to have happen in practice even if X is a blank line, especially if you use a diff algorithm which tries hard to keep the place repeated lines get attached to be reliable and consistent. But if it were to happen it feels like the history has ‘earned’ this funny behavior and going down to a single X with no conflict really is the right thing to do, as counterintuitive as that may be before you’ve worked through this example.

There was some discussion about my comment at the bottom of the last post about the code being artisanal but the post not. The risk with AI is that the code is a spaghetti mess which nobody realizes because no human has ever read it. There’s clear benefit to artisanal coding, or at least human auditing. The risks of AI assisted writing are a lot less. Humans understand what the words say. The AI’s writing is a lot breezier but less interesting but for reference materials that’s probably what you want. I did repeatedly go through that entire last post and tell the AI things which were wrong and should be changed. The experience was much better than manually editing because the AI could include whatever I said more quickly, in a more appropriate location, and with better wordsmithing than if I’d done it by hand. I didn’t try to change the tone to not sound like AI because that would have been more work and because I think it’s funny to encourage AI doomsaying by making it look like AI can write posts like that. This post was written entirely artisanally. Hopefully the term ‘artisanal code’ becomes a thing.

Subscribe now

Ad name: DNA Lounge Bruno Mars Party
Your ad wasn't approved for the following reasons:
Reason: Your ad contains only music.
Recommendation: Submit a new ad with a clear voiceover that explains what's being advertised.

Do you want TikTok AI Voice? Because this is how you get TikTok AI Voice.

Previously, previously, previously, previously.

GL_VERSION in the Android simulator is "OpenGL ES-CM 1.1 (4.1 Metal - 88.1)".

This works fine on iOS and Cocoa, so it's not strictly a GLES thing, just Android. GLSL is not involved.

Test case:

Bool lights_p = time(0) & 1; glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); glEnable (GL_BLEND); glDisable (GL_COLOR_MATERIAL); glBlendFunc (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); #define glColor4fv(v) glColor4f (v[0], v[1], v[2], v[3]) GLfloat c1[] = { 1, 0, 0, 0.5 }; GLfloat c2[] = { 0, 1, 0, 0.5 }; GLfloat v[] = { 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, }; glVertexPointer (3, GL_FLOAT, 0, v); glEnableClientState (GL_VERTEX_ARRAY); if (lights_p) { GLfloat amb[] = {0.5, 0.5, 0.5, 1}; glLightfv (GL_LIGHT0, GL_AMBIENT, amb); glEnable (GL_LIGHTING); glEnable (GL_LIGHT0); glColor3f (0, 0, 0); } else { glDisable (GL_LIGHTING); glDisable (GL_LIGHT0); } if (lights_p) glMaterialfv (GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE, c1); else glColor4fv (c1); glDrawArrays (GL_TRIANGLES, 0, 6); glPushMatrix(); glTranslatef (0.5, 0.25, 0); if (lights_p) glMaterialfv (GL_FRONT_AND_BACK, GL_AMBIENT_AND_DIFFUSE, c2); else glColor4fv (c2); glDrawArrays (GL_TRIANGLES, 0, 6); glPopMatrix(); glDisableClientState (GL_VERTEX_ARRAY); glVertexPointer (3, GL_FLOAT, 0, 0);

Now, I'm old enough to remember when "remastered" meant "we re-scanned the 35mm negative". Let's take a look at the upper left quadrant of the first 15 seconds of that video at half speed. On the left: a 702x576 rip I made of the video in 2003 from the SD DVD release, Directors Series Volume 3: The Work of Michel Gondry; on the right, the 1392x1062 "Official Full HD Remaster".

Pay special attention to the text on the white sign at about 8 seconds in. Full-screen it for the true zalgo-level, AI-upscale, edge-detect horrorshow.

This is why I have trust issues. Kylie.

Previously, previously, previously, previously.

Oberlin Luddites Reject "Year of AI Exploration" Adpoted by School Dear President Ambar, We are writing to you on a typewriter that is over 70 years old; this is a machine that we all know well. With it, we misspell words without the crutch of spell check or generative AI and we think intently about every phrase we pound out. As we force ourselves, for once, to slow down, we engage in a cognitive dialogue with ourselves. We do not seek perfection because we know that education is about the growing and challenging of our young minds' potential, not the chasing of inititurtonal 'gold-star' approval. We do not believe that your so-c alled 'Year of AI Exploration; providing enterprise ChatGPT and Google Gemini sub scriptions to every Oberlin student aligns with our college's founding principles. You claim that this year will be one of experimentation, not adoption. But even just one semester of accepted (encouraged even) chat bot use will jettison our student body down a lazy and irredeem able tunnel of intellctual destruction. We are a college grounded in learning and labor, which now risks straying from these rooted ideals. With ChatGPT at the helm, our emails, essays,and discussion posts will be generated for us, not by us. And let's not fool ourselves. This is precisely what these platforms will be used for by our busy, anxious student body. We see your vision for this year as advancing the college's 'businessification'--an alarming trend also seen in the takeover of our beloved library cafe by a 'bookstore' With no bools in stock (just shiny merch) and an app replacing customer service. In one instance, the college assumes we want efficiency at all cost through automated rather than hand-pulled coffee. In the other lies the false belief that we simply desire to turn in an essay, regardless of how little we've written of it. We need not stand by and witness the further atrophy of our liberal arts education. As you embark on your year of AI, we'll embark on our own year of self-actualization, of realizing the fruits of our labor and embracing human imperfection and raw inquiry. We will boldly reject information technologies operating out of data centers ghat guzzle water and energy sources(that simply must contradict our campus's carbon-neutral devotion). We will turn instead t o ourselves and embrace our natural intelligence, intentional hard work, and generative sweat rather than generative AI. We urge anyone who feels similarly to risk disapproval, perceived backwardness and the outward appearance of naivete and join us. Let us not ask what AI can do for Oberlin students, but what Oberlin students can do ourselves, while we still have the brain capacity to think on our own. Yours truly, The Luddite Club of Oberlin College, Logan Lane, CharLie McLaughlin, Mary Claire McGreivey, Sawyer Van Dyck, Simon Puchner-Noel,and Marlowe Blantz

Previously, previously, previously, previously.

Back in 2021 we finally received some federal grant money to keep us alive and cover the enormous costs of having been completely closed for 18 months starting in early 2020... This was the COVID-19 Shuttered Venue Operations Grant (SVOG) which I think is the Payroll Protection Program (PPP) wearing a trenchcoat. The money was incredibly helpful! And it was incredibly difficult to get. The amount of paperwork needed to apply for and receive the grant was insane. We just about had to allocate someone to babysitting that paperwork full-time for six months. The effort and complexity was far beyond the reach of most small businesses, which is why most of the money for these grants went to billion-dollar corporate grifters instead of to the people it was supposed to help, notable examples being Ruth's Chris Steak House, Veritas (SF's biggest landlord), Shake Shack, Nestea, RealNetworks, Lil Wayne, Chris Brown, Marshmello, Steve Aoki and Alice in Chains. And of course multinational superpredator LiveNation who managed to steal millions in Federal aid meant for independent venues.

Anyway, it took more than a year from when we applied to when we got our first check. A year with zero customers. And then in late 2024 -- nearly four years later -- they did some kind of rule-changing rug-pull on us. They were "reviewing our file" and demanded that we submit an "audit report consistent with 2 CFR 200 Subpart F".

Just more dumb bureaucracy, right? Well.

Hiring someone to do that audit report correctly cost us $20,000. TWENTY THOUSAND DOLLARS. And had we not done it, they would have demanded we pay back all of the money we got in 2021.

In 2021, they literally told us, "You have to spend all this money. If you don't spend it all within a year, on these things, then you can't keep it." So we didn't even have the option to sit on twenty grand to pay for the eventual audit, even if they had told us that was going to happen, which they did not.

Couldn't they have just given us $20k less and paid for the audit with that? No, apparently not.

Anyway, the auditors decided that we had done something slightly wrong. Let's say we had expenses A through Z, and we said, "We are using the grant to pay for A, B and C." The auditors said, "Oh, you can't spend it on C. But you are allowed to spend it on D through F, so all you have to do is file an amendment, and it all comes out the same."

So we do that... and... crickets.

See, in between when this audit crap started, and the auditors finished, DOGE had dismantled the Small Business Administration, and they just... stopped answering their phones, or email. Even the auditors could not get hold of anybody and that's their entire business.

So between October 2024 and March 2026 we were unable to get anyone at SBA to respond to our request to amend the grant! They didn't say "no" (if they said "no" they would have demanded that we pay back hundreds of thousands of dollars) but rather than saying "yes" they just kept sending us monthly automated emails from a "noreply" address noting that "your Audit Report is still outstanding" and that "Your timely response is anticipated and appreciated in order to resolve any compliance issues."

For two years.

And then today -- the rains have come and the crops are saved!

The Small Business Administration's Shuttered Venue Operators Grant Program (SVOG) has completed its review of your audit submission package. SBA has determined that your organization has fully and materially complied with Federal grant audit requirements for fiscal year 2022.

Our accountant and the auditors wasted so many (billable) hours just trying to get someone on the phone to respond in any way, so the total wasn't just the $20,000 we had to pay the audit firm. It also included all the work our accountant did. Fortunately accountants are boss-level note-takers:

• 43 emails with SBA;
• 286 emails with the audit firm;
• 4+ multi-hour zoom calls with the audit firm;
• Calling SBA every Wednesday to inquire on the status of our amendments, for at least a year -- 50+ calls lasting at least 30 minutes and getting nowhere, so that's 25 hours right there.

So besides the audiors themselves, babysitting this also cost us somewhere north of 250 hours of labor from our own accountant.

For nothing.

Nothing.

To get us right back to zero.

It's impressive how Apartheid Emerald Mine Space Karen and Cheeto Mussolini have made our government so much more "efficient".

This is where I mention that if you would like to help out with the DNA Lounge Accountancy Defense Fund, you can make a one-time donation or join our Patreon, which by the way is a really good deal.

• The Bluff (2026):
  An excellent little pirate movie. Which isn't quite right, because though it's about pirates, there's not a lot of pirating, or even sailing. It's more one of those "I was retired and you had to pull me back in" deal -- you know, a Statham Situation. Bloody Mary's ex wants the gollllllld. Anyway, fun characters, very good -- and parsable -- stunts, and the stunts felt very practical. I saw a number of Texas Switches going on, so I think a lot of it was done old-school.

• We Bury The Dead (2026):
  A decent road-trip zombie movie. The US accidentally nukes Tasmania with a zombie bomb, and people volunteer to clean up the corpses, including a bunch of people handling their grief in totally normal healthy ways, such as our hero who is looking for her husband and either hoping he is dead or hoping he is a zombie, it's not clear.

• Greenland (2020) and Greenland 2 (2025):
  These are both stupid disaster movies with absurd physics and geology, but more than that, they are movies that hate people, and believe that people are fundamentally awful, vicious monsters. Their thesis is that if anything goes wrong, everyone will immediately fuck each other over as quickly and as hard as possible, and there's nothing wrong with stepping on someone's face if it gets you into the boat one second earlier. It does not brook even a second's reflection. It was very hard watching this bullshit during the weeks when the people of Minnesota were presenting us a realtime rebuttal to this cynical, evil worldview. Besides that, it's amazing how any time there are like, 10 people who made it into the boat/car/bus/whatever, the ones who are not the 3 main characters are just ... absent ... in the next scene. Did they vaporize? Did our heroes eat them? Such script writing wow. I hated the first one but I hate-watched the second one because I had to know if it got even worse and yes, it did. I need an intervention.

• Cassandra (2025):
  Instead of haunted house stories, now we get them re-skinned as "smart-home automation turns evil" stories. The twist on this one is that the evil smart home is implausibly-vintage 1970s/80s tech. The show is just ok, and I really only even go that hight because of the retro styling. Which is pretty. Since it is Netflix, there's 90 minutes of plot dragged out to 5 hours over 6 episodes and it does the Episode Seven thing (but in ep 5) in the most predictable way. The writers for this definitely believed that you were half watching it while reading something else on your phone, so they have to repeat everything five times.

• The Damned (2025):
  Trapped in the ice, shipwreck, possible haunting. I dug it.

Then I went on a Ghost in the Shell binge, because it turns out there were like twice as many serieses as I realized and I hadn't seen most of them:

• Ghost in the Shell, Stand Alone Complex (2002):
  On a rewatch, this is still good! The animation is beautiful and the characters are well written. The Laughing Man is an interesting villain, and the signature move of being able to hide his face with a logo because every person and camera has mechanical eyes that are hackable is just epic. The eventual reveal of his identity is kind of weird and dumb, and overall the show suffers from several problems: A) it's a cop show and ACAB; and these C truly are B and have never met someone who told them "no" who was not a "terrorist"; and C) they do a whole lot of straight-up domestic political assassination; and D) it's weirdly racist for a cartoon where every character looks like a big-eyed Japanese pop star. You're all Sneeches! Much of the plot is about "immigrants" and "refugees" and how much of "our" taxes should be spent on "those parasites" and the difference between the "good guys" and "bad guys" seems to mainly be "should we deport them" or "should we murder them". So let's say that it doesn't age well.

• Ghost in the Shell, Stand Alone Complex, Second Gig (2004):
  Also a rewatch. Also still good, but kind of more of the same. Same weird anti-immigrant plot; the big bad "Individual Eleven" starts off as the same kind of maybe-hivemind as Laughing Man but then does a fake-out to something else which isn't as clever as they wanted it to be. Much more fun stuff with the emergent-personality Tachikoma smart tanks, which was a fun sub-plot.

• Ghost in the Shell, Solid State Society (2006):
  This was fine. At least they stopped shitting on immigrants and focused on billionaire vampires kidnapping children instead.

• Ghost in the Shell, Arise (2013):
  I never saw this the first time around, and reviews led me to believe that the story and animation were both vastly inferior, a bullshit cash grab. But actually I think it's quite good, possibly even better than SAC. It's a prequel, "here's how we got the gang together". In SAC Motoko was what, 35 to 50?, so she was wearing a 25 year old big-boobed robot porn star body. In this one she's 25-ish? (how old do you have to be to get promoted to Army Major?) so she's wearing a 15 year old robot body instead. Oh, anime, never change. But since she's not old and jaded she's allowed to have emotions and make mistakes instead of just being an always-right killing machine. And they aren't bottomless-funded cops just yet, so out of self-interest they have to actually care about consequences and accountability. They are still all temperamentally murderers, but without immunity, and that's what passes for ethics I guess. Less immigrant hate. Not none, but less.

  I hate that current events have made it impossible for me to enjoy stories about righteous vigilantes.

• Ghost in the Shell, SAC_2045 (2020):
  I never even heard that this came out, and... it's pretty bad. It's much more "3d" so it just looks like a video game cut scene. Section 9 has been disbanded and are working as vigilantes in Los Angeles, and The Major looks like a teen pop star now. Also the environments and especially the cars look very contemporary and out-of-place in the GitS universe. Someone involved really loves cars. The plot is... maybe not completely terrible but I really dislike the look of this show, and the character writing is just off.

  That said, S02 is better. The art is still jarring, but the plot is less "immigrants, amirite?" and more "super-cyborgs are freaked out by a secret society of even-more-super cyborgs." The plot hooked me, despite. Also there were some quite well choreographed fight scenes. Which feels like a weird thing to say about a cartoon but I assume it was a lot of mocap. But as always the Tachikomas are the real stars. The ending is confusing and incoherent, which is how you know it's a GitS show.

Ok, GitS diversion over:

• Night Patrol (2025):
  What if the (real) gangs inside LAPD were actual monsters. Very strong first act! Bogs down a bit in the third. Mostly black cast and they start off setting it up as a "white savior" plot but (spoiler) then they're like, "LOL no". It's fun. Not as good as Night Teeth, which I keep recommending.

• Primal S03:
  RAAAAAAAAAAAHHHHHHHHH!!!!!!!!!!

• Shelter (2026):
  I have simple needs. "Jason Statham was in hiding but then the commandos found him" -- shit man that's all you had to say. This time he's in a lighthouse.

• Monarch Legacy of Monsters S02:
  I continue to love everything about this show! Previously. Every time I tell someone this they say, "You're kidding." I am not kidding. It's great.

• Whistle (2025):
  Aztec Death Whistle summons your fate. It's basically a Final Destination movie, with the same dumb rules-lawyering. It's alright I guess.

• Umamusume Pretty Derby (2018):
  Ok wait hear me out. We have this recurring event at the club that plays "anime music", which is mostly indistinguishable from happy hardcore, and is some of the worst shit you'll ever hear (and I include "riddim" in that) and at the most recent event, everyone was wearing horse accessories and they left carrots all over the club, so I had to look up what the actual fuck this was all about. Anyway I watched like half of the first two episodes of this and it's some of the worst trash I've ever seen. It makes Sailor Moon look like Shakespeare. And like Rembrandt. Plot: sometimes genetic mutants are born with horse tails and once they grow into big-boobed teenagers they encounter a biological imperative to run in circles, very very fast; and then do karaoke. Oh anime, never change.

• The Beauty (2026):
  I started watching this knowing nothing about it, and the first episode was like: "Oh it's X Files body horror, I'm down", followed by, "This is a straight-up rip-off of The Substance, stop reminding me of better movies that I'd rather be watching instead", and then I soon realized that it's by the American Horror Story guy, and yup, by episode 2 he's back on his usual bullshit: "The only way I know how to write women is as cunty drag queens, aren't they just the worst? Girlfriend!" But, it had body horror and some billionaires get murdered so I kept hate-watching. The season ended with literally nothing resolved.

• Is This Thing On? (2025):
  Will Arnett and Laura Dern get divorced and he works out his trauma by doing standup. It's kinda just a romcom but it's really sweet.

• Young Sherlock (2026):
  This is some bullshit.

  The first thing you need to know about this show, which should have been a damned title screen, is that it is not a prequel, it is an alternate history that in no way lines up with the books or any other Holmes story. And knowing that would have saved me so much frustration and annoyance.

  But it's still some bullshit.

  I appreciate how some people might be here for "will twentysomething Sherlock and Moriarty fuck" but I can't get past "they didn't meet until Sherlock was like 35 and Moriarty was like 70, what even is this."

  I have been a Sherlock fan for a little while now. I still have the copy of Hound of the Baskervilles that I bought on a whim at a flea market when I was like 9 years old, and it imprinted on me hard. I assume I paid 10¢ for it. It is absolutely falling apart to dust but the latest copyright on it is 1904. The thing about Sherlock is, there's only like 1700 pages total, that's barely 5 novels, but it launched a thousand ships. This ship though, this "shipping", this is some crap. I might have enjoyed this show if the characters all had different names like "Steve" and "Brian" instead of "Sherlock" and "Moriarty" but as-is, I JUST. CAN. NOT.

  One of the things about being a lifelong fan of Sherlock is that when you are a child you think "This guy is the greatest guy ever and I love him" and then when you are an adult you think "This guy is an absolute piece of shit and I love him" and if you never made that transition, everyone else in your life has my sympathies.

  Anyway, as movies and TV go,

  • The Cushing movies -- honestly I'm not into them.
  • The Seven Percent Solution -- pretty great and nuts.
  • The Downey movies -- first eh, second hard pass.
  • House -- yes to S01, strongly diminishing returns after. Probably didn't age well.
  • The Cumberbach shows -- mostly fun, overall unsatisfying.
  • Elementary -- middling.
  • Enola Holmes -- Fun and I appreciate it as criticism.

  It's not a great track record.

• Good Luck Have Fun Don't Die (2026):
  A pretty fun low-budget riff on Terminator, but with the dirtbaggy lunacy of 12 Monkeys.

• Scream 7 (2026):
  This was definitely a Scream movie. It was fine. Always good to see [SPOILER] again. Has some AI hate.

• Scarlet (2025):
  King Hamlet's daughter dies and goes to hell, which is somehow the Middle East but underwater, gets a 21st Century paramedic as a sidekick, and goes on a revenge-quest, fighting an endless series of Deadites. And then, I dunno, the planet explodes into love or something. The animation is great; it's a weird mix of photorealistic 3D-rendered backgrounds with cel-style anime characters, but the motion in the sword fights and dance routines (plural) feels rotoscoped, it has a twitchy Bakshi feel. Weird-ass incoherent movie, but pretty.

Previously.

"The U.S. Strategic Oil Reserve is a series of vast, subterranean salt caverns in four different sites in Louisiana and Texas. Many are enormous -- the average cavern holds about 10mn barrels [...] This network of tunnels, grottos, pumps and wells can in total hold about 715mn barrels of oil, or enough to supply the entire U.S. with all the oil it needs for over a month" -- but those salt caverns were only designed to be drained and refilled five times.

The Financial Times calculates that we are already at the cavern's ninth historic drawdown, suggesting that "catastrophic structural damage," including dissolution of the salt caverns, is now a viable risk. This could mean, among other things, that the reserves can no longer be drained in their entirety, as "a minimum level of oil... must be kept in the salt caverns" to avoid this fate, with the result that the reserves' effectiveness in a time of future national emergency will be reduced.

Of course, this could also mean that someday the caverns will simply collapse.

Previously, previously, previously, previously, previously, previously, previously, previously, previously.

I’m releasing Manyana, a project which I believe presents a coherent vision for the future of version control — and a compelling case for building it.

It’s based on the fundamentally sound approach of using CRDTs for version control, which is long overdue but hasn’t happened yet because of subtle UX issues. A CRDT merge always succeeds by definition, so there are no conflicts in the traditional sense — the key insight is that changes should be flagged as conflicting when they touch each other, giving you informative conflict presentation on top of a system which never actually fails. This project works that out.

Better conflict presentation

One immediate benefit is much more informative conflict markers. Two people branch from a file containing a function. One deletes the function. The other adds a line in the middle of it. A traditional VCS gives you this:

<<<<<<< left
=======
def calculate(x):
    a = x * 2
    logger.debug(f"a={a}")
    b = a + 1
    return b
>>>>>>> right

Two opaque blobs. You have to mentally reconstruct what actually happened.

Manyana gives you this:

<<<<<<< begin deleted left
def calculate(x):
    a = x * 2
======= begin added right
    logger.debug(f"a={a}")
======= begin deleted left
    b = a + 1
    return b
>>>>>>> end conflict

Each section tells you what happened and who did it. Left deleted the function. Right added a line in the middle. You can see the structure of the conflict instead of staring at two blobs trying to figure it out.

What CRDTs give you

CRDTs (Conflict-Free Replicated Data Types) give you eventual consistency: merges never fail, and the result is always the same no matter what order branches are merged in — including many branches mashed together by multiple people working independently. That one property turns out to have profound implications for every aspect of version control design.

Line ordering becomes permanent. When two branches insert code at the same point, the CRDT picks an ordering and it sticks. This prevents problems when conflicting sections are both kept but resolved in different orders on different branches.

Conflicts are informative, not blocking. The merge always produces a result. Conflicts are surfaced for review when concurrent edits happen “too near” each other, but they never block the merge itself. And because the algorithm tracks what each side did rather than just showing the two outcomes, the conflict presentation is genuinely useful.

History lives in the structure. The state is a weave — a single structure containing every line which has ever existed in the file, with metadata about when it was added and removed. This means merges don’t need to find a common ancestor or traverse the DAG. Two states go in, one state comes out, and it’s always correct.

Rebase without the nightmare

One idea I’m particularly excited about: rebase doesn’t have to destroy history. Conventional rebase creates a fictional history where your commits happened on top of the latest main. In a CRDT system, you can get the same effect — replaying commits one at a time onto a new base — while keeping the full history. The only addition needed is a “primary ancestor” annotation in the DAG.

This matters because aggressive rebasing quickly produces merge topologies with no single common ancestor, which is exactly where traditional 3-way merge falls apart. CRDTs don’t care — the history is in the weave, not reconstructed from the DAG.

What this is and isn’t

Manyana is a demo, not a full-blown version control system. It’s about 470 lines of Python which operate on individual files. Cherry-picking and local undo aren’t implemented yet, though the README lays out a vision for how those can be done well.

What it is is a proof that CRDT-based version control can handle the hard UX problems and come out with better answers than the tools we’re all using today — and a coherent design for building the real thing.

The code is public domain. The full design document is in the README.

Subscribe now

Four years ago I asked whether "Google Pass" was a thing that I needed to give a shit about and consensus was, "no, nobody uses that." But I have heard anecdotally, recently, that this might no longer be true. Thoughts?

The goal here is, "reduce the amount of time it takes for someone standing in front of my nightclub to wave their QR code at the door staff." On iOS, Apple Wallet supports that goal very well.

Note: I don't use Android and know as little about its ecosystem as possible, so please use small words.

Previously, previously, previously, previously, previously, previously, previously, previously, previously.

Configuration Tiers: Three Levels of Infraction

TIER 0, "The Pamphlet" ~$6
Minimum viable violation. A bootable Linux device with a display, network connectivity, and an app store. No battery, no keyboard -- just proof that this constitutes a regulated device under AB 1043. Good for bulk handout at conferences (50-100 units).

Legal status: Arguable. The 128×64 display introduces fuzziness. The AG could claim it's a dev board. That's fine -- ambiguity is instructive too.

TIER 1, "The Computer", ~$12
An unambiguous general purpose computing device. Color display, keyboard, WiFi, Linux, app store, user setup. The core product. There is no interpretive gap between this device and the law's definitions.

Legal status: Unambiguous. This is a computer with a color display, keyboard, WiFi, Linux, and an app store. It does not collect age data. It is handed to a child. The maximum fine is $7,500.

TIER 2, "The Appliance", ~$18
A self-contained, battery-powered pocket Linux computer. The educational device angle -- a modern descendant of the Acorn BBC Micro and the original Raspberry Pi.

Legal status: Beyond unambiguous. A pocket computer with a color screen, keyboard, battery, WiFi, 8GB storage, and an AI accelerator. It costs less than a large pizza. It fits in a child's hand.

Every tab on this site is gold:

How Distros Are Responding: We track how Linux distributions are responding to age verification mandates, and we provide tools to undo whatever they implement. If a distribution adds an age collection prompt, we will publish a script that removes it. If it ships a D-Bus age verification daemon, we will publish a package that replaces it with silence.

How One Bill Becomes Every Bill: AB 1043 was not written in isolation. It is a template. ICMEC published the model text as a ready-to-introduce statutory draft, and its Global Head of Policy presented it directly to Virginia's Joint Commission on Technology and Science. The same organizations that drafted the model bill are now deploying it in state legislatures across the country. The companies that benefit from the compliance moat fund the advocacy organizations that draft the bills that create the compliance moat. [...]

The Door That Stays Open: AB 1043 requires only self-declared age -- a birthdate field, not government ID or biometrics. Industry analysts have described this as "an initial implementation designed to get the door open." Self-declaration today. Biometric verification tomorrow. The infrastructure is the same; only the input changes. Once every operating system has an age collection interface and a real-time API for transmitting age data to applications, upgrading from a text field to a face scan is a configuration change, not a new law.

Penalty Comparison: Cost of Giving a Child a Computer:

Cost of one Ageless Linux device: $12-18
Maximum combined US penalty for one device given to one child: $46,000
US penalty-to-cost ratio: 3,067:1
Brazil penalty for one violation: up to 522,222:1

Previously, previously, previously, previously.

We’ve all heard of those network effect laws: the value of a network goes up with the square of the number of members. Or the cost of communication goes up with the square of the number of members, or maybe it was n log n, or something like that, depending how you arrange the members. Anyway doubling a team doesn't double its speed; there’s coordination overhead. Exactly how much overhead depends on how badly you botch the org design.

But there’s one rule of thumb that someone showed me decades ago, that has stuck with me ever since, because of how annoyingly true it is. The rule is annoying because it doesn’t seem like it should be true. There’s no theoretical basis for this claim that I’ve ever heard. And yet, every time I look for it, there it is.

Here we go:

Every layer of approval makes a process 10x slower

I know what you're thinking. Come on, 10x? That’s a lot. It’s unfathomable. Surely we’re exaggerating.

Nope.

Just to be clear, we're counting “wall clock time” here rather than effort. Almost all the extra time is spent sitting and waiting.

Look:

• Code a simple bug fix
  30 minutes

• Get it code reviewed by the peer next to you
  300 minutes → 5 hours → half a day

• Get a design doc approved by your architects team first
  50 hours → about a week

• Get it on some other team’s calendar to do all that
  (for example, if a customer requests a feature)
  500 hours → 12 weeks → one fiscal quarter

I wish I could tell you that the next step up — 10 quarters or about 2.5 years — was too crazy to contemplate, but no. That’s the life of an executive sitting above a medium-sized team; I bump into it all the time even at a relatively small company like Tailscale if I want to change product direction. (And execs sitting above large teams can’t actually do work of their own at all. That's another story.)

AI can’t fix this

First of all, this isn’t a post about AI, because AI’s direct impact on this problem is minimal. Okay, so Claude can code it in 3 minutes instead of 30? That’s super, Claude, great work.

Now you either get to spend 27 minutes reviewing the code yourself in a back-and-forth loop with the AI (this is actually kinda fun); or you save 27 minutes and submit unverified code to the code reviewer, who will still take 5 hours like before, but who will now be mad that you’re making them read the slop that you were too lazy to read yourself. Little of value was gained.

Now now, you say, that’s not the value of agentic coding. You don’t use an agent on a 30-minute fix. You use it on a monstrosity week-long project that you and Claude can now do in a couple of hours! Now we’re talking. Except no, because the monstrosity is so big that your reviewer will be extra mad that you didn’t read it yourself, and it’s too big to review in one chunk so you have to slice it into new bite-sized chunks, each with a 5-hour review cycle. And there’s no design doc so there’s no intentional architecture, so eventually someone’s going to push back on that and here we go with the design doc review meeting, and now your monstrosity week-long project that you did in two hours is... oh. A week, again.

I guess I could have called this post Systems Design 4 (or 5, or whatever I’m up to now, who knows, I’m writing this on a plane with no wifi) because yeah, you guessed it. It's Systems Design time again.

The only way to sustainably go faster is fewer reviews

It’s funny, everyone has been predicting the Singularity for decades now. The premise is we build systems that are so smart that they themselves can build the next system that is even smarter, that builds the next smarter one, and so on, and once we get that started, if they keep getting smarter faster enough, then the incremental time (t) to achieve a unit (u) of improvement goes to zero, so (u/t) goes to infinity and foom.

Anyway, I have never believed in this theory for the simple reason we outlined above: the majority of time needed to get anything done is not actually the time doing it. It’s wall clock time. Waiting. Latency.

And you can’t overcome latency with brute force.

I know you want to. I know many of you now work at companies where the business model kinda depends on doing exactly that.

Sorry.

But you can’t just not review things!

Ah, well, no, actually yeah. You really can’t.

There are now many people who have seen the symptom: the start of the pipeline (AI generated code) is so much faster, but all the subsequent stages (reviews) are too slow! And so they intuit the obvious solution: stop reviewing then!

The result might be slop, but if the slop is 100x cheaper, then it only needs to deliver 1% of the value per unit and it's still a fair trade. And if your value per unit is even a mere 2% of what it used to be, you’ve doubled your returns! Amazing.

There are some pretty dumb assumptions underlying that theory; you can imagine them for yourself. Suffice it to say that this produces what I will call the AI Developer’s Descent Into Madness:

1. Whoa, I produced this prototype so fast! I have super powers!

2. This prototype is getting buggy. I’ll tell the AI to fix the bugs.

3. Hmm, every change now causes as many new bugs as it fixes.

4. Aha! But if I have an AI agent also review the code, it can find its own bugs!

5. Wait, why am I personally passing data back and forth between agents

6. I need an agent framework

7. I can have my agent write an agent framework!

8. Return to step 1

It’s actually alarming how many friends and respected peers I’ve lost to this cycle already. Claude Code only got good maybe a few months ago, so this only recenlty started happening, so I assume they will emerge from the spiral eventually. I mean, I hope they will. We have no way of knowing.

Why we review

Anyway we know our symptom: the pipeline gets jammed up because of too much new code spewed into it at step 1. But what's the root cause of the clog? Why doesn’t the pipeline go faster?

I said above that this isn’t an article about AI. Clearly I’m failing at that so far, but let’s bring it back to humans. It goes back to the annoyingly true observation I started with: every layer of review is 10x slower. As a society, we know this. Maybe you haven't seen it before now. But trust me: people who do org design for a living know that layers are expensive... and they still do it.

As companies grow, they all end up with more and more layers of collaboration, review, and management. Why? Because otherwise mistakes get made, and mistakes are increasingly expensive at scale. The average value added by a new feature eventually becomes lower than the average value lost through the new bugs it causes. So, lacking a way to make features produce more value (wouldn't that be nice!), we try to at least reduce the damage.

The more checks and controls we put in place, the slower we go, but the more monotonically the quality increases. And isn’t that the basis of continuous improvement?

Well, sort of. Monotonically increasing quality is on the right track. But “more checks and controls” went off the rails. That’s only one way to improve quality, and it's a fraught one.

“Quality Assurance” reduces quality

I wrote a few years ago about W. E. Deming and the "new" philosophy around quality that he popularized in Japanese auto manufacturing. (Eventually U.S. auto manufacturers more or less got the idea. So far the software industry hasn’t.)

One of the effects he highlighted was the problem of a “QA” pass in a factory: build widgets, have an inspection/QA phase, reject widgets that fail QA. Of course, your inspectors probably miss some of the failures, so when in doubt, add a second QA phase after the first to catch the remaining ones, and so on.

In a simplistic mathematical model this seems to make sense. (For example, if every QA pass catches 90% of defects, then after two QA passes you’ve reduced the number of defects by 100x. How awesome is that?)

But in the reality of agentic humans, it’s not so simple. First of all, the incentives get weird. The second QA team basically serves to evaluate how well the first QA team is doing; if the first QA team keeps missing defects, fire them. Now, that second QA team has little incentive to produce that outcome for their friends. So maybe they don’t look too hard; after all, the first QA team missed the defect, it’s not unreasonable that we might miss it too.

Furthermore, the first QA team knows there is a second QA team to catch any defects; if I don’t work too hard today, surely the second team will pick up the slack. That's why they're there!

Also, the team making the widgets in the first place doesn’t check their work too carefully; that’s what the QA team is for! Why would I slow down the production of every widget by being careful, at a cost of say 20% more time, when there are only 10 defects in 100 and I can just eliminate them at the next step for only a 10% waste overhead? It only makes sense. Plus they'll fire me if I go 20% slower.

To say nothing of a whole engineering redesign to improve quality, that would be super expensive and we could be designing all new widgets instead.

Sound like any engineering departments you know?

Well, this isn’t the right time to rehash Deming, but suffice it to say, he was on to something. And his techniques worked. You get things like the famous Toyota Production System where they eliminated the QA phase entirely, but gave everybody an “oh crap, stop the line, I found a defect!” button.

Famously, US auto manufacturers tried to adopt the same system by installing the same “stop the line” buttons. Of course, nobody pushed those buttons. They were afraid of getting fired.

Trust

The basis of the Japanese system that worked, and the missing part of the American system that didn’t, is trust. Trust among individuals that your boss Really Truly Actually wants to know about every defect, and wants you to stop the line when you find one. Trust among managers that executives were serious about quality. Trust among executives that individuals, given a system that can work and has the right incentives, will produce quality work and spot their own defects, and push the stop button when they need to push it.

But, one more thing: trust that the system actually does work. So first you need a system that will work.

Fallibility

AI coders are fallible; they write bad code, often. In this way, they are just like human programmers.

Deming’s approach to manufacturing didn’t have any magic bullets. Alas, you can’t just follow his ten-step process and immediately get higher quality engineering. The secret is, you have to get your engineers to engineer higher quality into the whole system, from top to bottom, repeatedly. Continuously.

Every time something goes wrong, you have to ask, “How did this happen?” and then do a whole post-mortem and the Five Whys (or however many Whys are in fashion nowadays) and fix the underlying Root Causes so that it doesn’t happen again. “The coder did it wrong” is never a root cause, only a symptom. Why was it possible for the coder to get it wrong?

The job of a code reviewer isn't to review code. It's to figure out how to obsolete their code review comment, that whole class of comment, in all future cases, until you don't need their reviews at all anymore.

(Think of the people who first created "go fmt" and how many stupid code review comments about whitespace are gone forever. Now that's engineering.)

By the time your review catches a mistake, the mistake has already been made. The root cause happened already. You're too late.

Modularity

I wish I could tell you I had all the answers. Actually I don’t have much. If I did, I’d be first in line for the Singularity because it sounds kind of awesome.

I think we’re going to be stuck with these systems pipeline problems for a long time. Review pipelines — layers of QA — don’t work. Instead, they make you slower while hiding root causes. Hiding causes makes them harder to fix.

But, the call of AI coding is strong. That first, fast step in the pipeline is so fast! It really does feel like having super powers. I want more super powers. What are we going to do about it?

Maybe we finally have a compelling enough excuse to fix the 20 years of problems hidden by code review culture, and replace it with a real culture of quality.

I think the optimists have half of the right idea. Reducing review stages, even to an uncomfortable degree, is going to be needed. But you can’t just reduce review stages without something to replace them. That way lies the Ford Pinto or any recent Boeing aircraft.

The complete package, the table flip, was what Deming brought to manufacturing. You can’t half-adopt a “total quality” system. You need to eliminate the reviews and obsolete them, in one step.

How? You can fully adopt the new system, in small bites. What if some components of your system can be built the new way? Imagine an old-school U.S. auto manufacturer buying parts from Japanese suppliers; wow, these parts are so well made! Now I can start removing QA steps elsewhere because I can just assume the parts are going to work, and my job of "assemble a bigger widget from the parts" has a ton of its complexity removed.

I like this view. I’ve always liked small beautiful things, that’s my own bias. But, you can assemble big beautiful things from small beautiful things.

It’s a lot easier to build those individual beautiful things in small teams that trust each other, that know what quality looks like to them. They deliver their things to customer teams who can clearly explain what quality looks like to them. And on we go. Quality starts bottom-up, and spreads.

I think small startups are going to do really well in this new world, probably better than ever. Startups already have fewer layers of review just because they have fewer people. Some startups will figure out how to produce high quality components quickly; others won't and will fail. Quality by natural selection?

Bigger companies are gonna have a harder time, because their slow review systems are baked in, and deleting them would cause complete chaos.

But, it’s not just about company size. I think engineering teams at any company can get smaller, and have better defined interfaces between them.

Maybe you could have multiple teams inside a company competing to deliver the same component. Each one is just a few people and a few coding bots. Try it 100 ways and see who comes up with the best one. Again, quality by evolution. Code is cheap but good ideas are not. But now you can try out new ideas faster than ever.

Maybe we’ll see a new optimal point on the monoliths-microservices continuum. Microservices got a bad name because they were too micro; in the original terminology, a “micro” service was exactly the right size for a “two pizza team” to build and operate on their own. With AI, maybe it's one pizza and some tokens.

What’s fun is you can also use this new, faster coding to experiment with different module boundaries faster. Features are still hard for lots of reasons, but refactoring and automated integration testing are things the AIs excel at. Try splitting out a module you were afraid to split out before. Maybe it'll add some lines of code. But suddenly lines of code are cheap, compared to the coordination overhead of a bigger team maintaining both parts.

Every team has some monoliths that are a little too big, and too many layers of reviews. Maybe we won't get all the way to Singularity. But, we can engineer a much better world. Our problems are solvable.

It just takes trust.

Since nobody reads to the end of my posts I’ll start this one with the actionable experiment:

Deep neural network have a fundamental problem. The thing which makes them able to be trained also makes them susceptible to Manchurian Candidate type attacks where you say the right gibberish to them and it hijacks their brain to do whatever you want. They’re so deeply susceptible to this that it’s a miracle they do anything useful at all, but they clearly do and mostly people just pretend this problem is academic when using them in the wild even though the attacks actually work.

There’s a loophole to this which it might be possible to make reliable: thinking. If an LLM spends time talking to itself then it might be possible for it to react to a Manchurian Candidate attack by initially being hijacked but then going ‘Wait, what am I talking about?’ and pulling itself together before giving its final answer. This is a loophole because the final answer changes chaotically with early word selection so it can’t be back propagated over.

This is something which should explicitly be trained for. During training you can even cheat and directly inject adversarial state without finding a specific adversarial prompt which causes that state. You then get its immediate and post-thinking answers to multiple choice questions and use reinforcement learning to improve its accuracy. Make sure to also train on things where it gets the right answer immediately so you aren’t just training to always change its answer. LLMs are sneaky.

Now on to rambling thoughts.

Some people nitpicked that in my last post I was a little too aggressive not including normalization between layers and residuals, which is fair enough, they are important and possibly necessary details which I elided (although I did mention softmax), but they most definitely play strictly within the rules and the framework given, which was the bigger point. It’s still a circuit you can back propagate over. There’s a problem with online discourse in general, where people act like they’ve debunked an entire thesis if any nitpick can be found, even if it isn’t central to the thesis or the nitpick is over a word fumbling or simplification or the adjustment doesn’t change the accuracy of the thesis at all.

It’s beautifully intuitive how the details of standard LLM circuits fit together: Residuals stop gradient decay. Softmax stops gradient explosion. Transformers cause diffusion. Activation functions add in nonlinearity. There’s another big benefit of residuals which I find important but most people don’t worry about: If you just did a matrix multiplication then all permutations of the outputs would be isomorphic and have valid encodings effectively throwing away log(N!) bits from the weights which is a nontrivial loss. Residuals give an order and make the permutations not at all isomorphic. One quirk of the vernacular is that there isn’t a common term for the reciprocal of the gradient, the size of training adjustments, which is the actual problem. When you have gradient decay you have adjustment explosion and the first layer weights become chaotic noise. When you have gradient explosion you have adjustment decay and the first layer weights are frozen and unchanging. Both are bad for different reasons.

There are clear tradeoffs between fundamental limitations and practical trainability. Simple DNNs get mass quantities of feedback but have slightly mysterious limitations which are terrifying. Thinking has slightly less limitations at the cost of doing the thinking both during running and training where it only gets one unit of feedback per entire session instead of per word. Genetic algorithms have no limitations on the kinds of functions then can handle at all at the cost of being utterly incapable of utilizing back propagation. Simple mutational hill climbing has essentially no benefit over genetic algorithms.

On the subject of activation functions, sometimes now people use Relu^2 which seems directly against the rules and only works by ‘divine benevolence’. There must be a lot of devil in the details in that its non-scale-freeness is leveraged and everything is normed to make the values mostly not go above 1 so there isn’t too much gradient growth. I still maintain trying Reluss is an experiment worth doing.

Some things about the structure of LLMs are bugging me (This is a lot fuzzier and more speculative than the above). In the later layers the residuals make sense but for the first few they’re forcing it to hold onto input information in its brain while it’s trying to form more abstract thoughts so it’s going to have to arbitrarily pick some bits to sacrifice. Of course the actual inputs to an LLM have special handling so this may not matter, at least not for the main part of everything. But that raises some other points which feel off. The input handling being special is a bit weird, but maybe reasonable. It still has the property that in practice the input is completely jamming the first layer for a simply practical reason: The ‘context window’ is basically the size of its brain, and you don’t have to literally overwhelm the whole first layer with it, but if you don’t you’re missing out on potentially useful content, so in practice people overwhelm its brain and figure the training will make it make reasonable tradeoffs on which tokens it starts ignoring, although I suspect in practice it somewhat arbitrarily picks token offsets to just ignore so it has some brain space to think. It also feels extremely weird that it has special weights for all token offset. While the very last word is special and the one before that less so, that goes down quickly and it seems wrong that the weights related the hundredth to hundred and first token back are unrelated to the weights related to the hundred and first and hundred and second token back. Those should be tied together so it’s getting trained as one thing. I suspect that some of that is redundant and inefficient and some of it it is again ignoring parts of the input so it has brain space to think.

Subscribe now

In 1995 someone could have written a paper which went like this (using modern vernacular) and advanced the field of AI by decades:

The central problem with building neural networks is training them when they’re deeper than two layers due to gradient descent and gradient decay. You can get around this problem by building a neural network which has N values at each layer which are then multiplied by an NxN matrix of weights and have Relu applied to them afterwards. This causes the derivative of effects on the last layer to be proportionate with the effects on the first layer no matter how deep the neural network is. This represents a quirky family of functions whose theoretical limitations are mysterious but demonstrably work well for simple problems in practice. As computers get faster it will be necessary to use a sub-quadratic structures for the layers.

History being the quirky thing that it is what actually happened is decades later the seminal paper on those sub-quadratic structures happened to stumble across making everything sublinear and as a result people are confused as to which is actually the core insight. But the structure holds: In a deep neural network, you stick to relu, softmax, sigmoid, sin, and other sublinear functions and magically can train neural networks no matter how deep they are.

There are two big advantages which digital brains have over ours: First, they can be copied perfectly for free, and second, as long as they haven’t diverged too much the results of training them can be copied from one to another. Instead of a million individuals with 20 years experience you get a million copies of one individual with 20 million years of experience. The amount of training data current we humans need to become useful is miniscule compared to current AI but they have the advantage of sheer scale.

Subscribe now

As described previously, the Linux kernel security team does not identify or mark or announce any sort of security fixes that are made to the Linux kernel tree. So how, if the Linux kernel were to become a CVE Numbering Authority (CNA) and responsible for issuing CVEs, would the identification of security fixes happen in a way that can be done by a volunteer staff? This post goes into the process of how kernel fixes are currently automatically assigned to CVEs, and also the other “out of band” ways a CVE can be issued for the Linux kernel project.

When playing melodies the effects of microtonality are a bit disappointing. Tunes are still recognizable when played ‘wrong’. The effects are much more dramatic when you play chords:

You can and should play with an interactive version of this here. It’s based off this and this with labels added by me. The larger gray dots are standard 12EDO (Equal Divisions of the Octave) positions and the smaller dots are 24EDO. There are a lot of benefits of going with 24EDO for microtonality. It builds on 12EDO as a foundation, in the places where it deviates it’s as microtonal as is possible, and it hits a lot of good chords.

Unrelated to that I’d like to report on an experiment of mine which failed. I had this idea that you could balance the volumes of dissonant notes to make dyads consonant in unusual places. It turns out this fails because the second derivative of dissonance curves is negative everywhere except unity. This can’t possibly be a coincidence. If you were to freehand something which looks like dissonance curves it wouldn’t have this property. Apparently the human ear uses positions where the second derivative of dissonance is positive to figure out what points form the components of a sound and looks for patterns in those to find complete sounds.

Subscribe now

Here’s the story of a legendary poker hand:

Our hero decides to play with 72, which is the worst hand in Holdem and theory says he was supposed to have folded but he played it anyway.

Later he bluffed all-in with 7332 on the board and the villain was thinking about whether to call. At this point our hero offered a side bet: For a fee you can look at one of my hole cards of your choice. The villain paid the fee and happened to see the 2, at which point he incorrectly deduced that the hero must have 22 as his hole cards and folded.

What’s going on here is that the villain had a mental model which doesn’t include side bets. It may have been theoretically wrong to play 72, but in a world where side bets are allowed and the opponent’s mental model doesn’t include them it can be profitable. The reveal of information in this case was adversarial. The fee charged for it was misdirection to make the opponent think that it was a tradeoff for value rather than information which the hero wanted to give away.

What the villain should have done was think through this one level deeper. Why is my opponent offering this at all? Under what situations would they come up with it? Even without working through the details there’s a much simpler heuristic for cutting through everything: There’s a general poker tell that if you’re considering what to do and your opponent starts talking about the hand that suggests that they want you to fold. A good rule of thumb is that if you’re thinking and the opponent offers some cockamamie scheme you should just call. That certainly would have worked in this case. This seems like a rule which applies in general in life, not just in Poker.

Subscribe now