LLMs, or models like them, are going to start giving us artifacts that we cannot (easily) comprehend.

We’ve been in this boat for a while, first with stochastic optimization algorithms (the classic nasa evolved antenna), and later with automated theorem proving.

Algorithms optimize for an objective solution and we get a thing that looks like it solves the problem we want, but it’s opaque (strange, large, complex, etc.).

This came to mind because of a tweet the other day (January 20 2025) by George Hotz.

It looks like George has since deleted all tweets, but here’s a screenshot I found of the tweet in question:

He’s the text for the SEO bots:

BREAKING: OpenAI’s o3, a new PhD level math agent, found a polynomial time factoring algorithm, none of your crypto assets are safe, web now insecure, per Axios.

Good satire!

But I believe we will be getting findings/products/etc. soon “discovered” by an LLM driven my humans.

And…we won’t understand them.

For a while we could understand them, because LLMs are infinitely patient at explaining. Academics will ask “but why this and that” until they grok it and write it up for other humans.

This will end once human patience runs out.

What kinds of things?

  • Drugs
  • Software
  • Theorems
  • Laws
  • Proofs

(quick search…)

Yep, we’re on our way:

Clinical trials of the first drugs designed with the help of artificial intelligence could commence this year, Google DeepMind CEO Demis Hassabis suggested Tuesday.

Not just hard science stuff, also things from the humanities:

  • A piece of music that makes you cry
  • A piece of poetry makes you feel bliss
  • A piece of writing that convinces you of something important
  • And on.

Work product here probably has to be bespoke, because emotion-eliciting content doesn’t generalize as well as function or math. But some does, and that will be interesting.

I can feel aspects of this already.

I generate code, and as long as it gives the desired effect/result/ui/etc., I don’t look too close.

Soon, it will be whole software-mediated interactions generated on demand, with no want or need to look under the covers.

Later “artifacts”.

Reminds me of “godvomit” from The Rapture of the Nerds.

A summary of the term from gpt4o (so I don’t have to look it up):

In The Rapture of the Nerds by Cory Doctorow and Charles Stross, the term “godvomit” is a vivid and colorful metaphor used to describe the chaotic, overwhelming, and often incomprehensible technological and cultural detritus created by post-singularity superintelligences. These intelligences produce an excessive amount of inventions, ideas, and artifacts, which are far beyond human understanding and utility, yet are haphazardly strewn across the universe.

The word combines “god” (referencing the near-divine status of these superintelligences) with “vomit” (evoking an unregulated, overwhelming, and often unpleasant excess). It encapsulates the satirical tone of the book, which explores themes of post-humanity, technological excess, and the absurdity of living in a world shaped by beings far beyond human comprehension.

Exactly, except, we will be asking for them.

I recall a similar concept in A Fire Upon the Deep.

An snippet from a summary from our gpt4o friend:

In Vinge’s universe, the Zones of Thought divide space into regions where the laws of physics, intelligence, and technology function at varying levels. The Transcend is the realm where superintelligent entities, akin to gods, exist and operate far beyond the comprehension of lesser beings. The artifacts, knowledge, and influence of these transcendent entities often filter down to lower zones like the Beyond, causing chaos, misunderstandings, or unintended consequences.

Also reminds me of Roadside Picnic, except aliens left behind the artifacts. But same result.

An snippet from a summary from our gpt4o friend:

In Roadside Picnic by Arkady and Boris Strugatsky, the equivalent concept to “godvomit” is the “Zone” and its artifacts. The Zone is an area left behind after an alien “visit” to Earth—an event that humans can neither understand nor explain. The alien visitors left behind a variety of bizarre and incomprehensible objects, phenomena, and environmental changes. These remnants are often dangerous, sometimes useful, and always mysterious.

I guess this motif is used a lot in scifi :)

In these three cases, they are detritus rather than the result of requests.

The idea of a wish-granting AI (LLM) seems too far fetched for scifi! Or plain boring.

This is all to say, it’s coming and I’m okay with it, I guess.

I don’t care to ask how my iphone works, as long as it works.