Home/AlgFormer/HoloFormer, explained

part of AlgFormer · how the holographic transformer works

A transformer writes meaning as dots on a map. This one plays it as chords.

Most AI models turn words into points in space and learn enormous tables to move them around. AlgFormer's holographic transformer turns words into sounds, and lets the physics of sound do the work. Here's how it works, no maths required — and why it isn't just a small transformer.

A transformer: scattered points
This model: a stack of tones
01 — Every word is a chord

Meaning is a sound, not a spot.

Give the model a letter, a word, or a number and it writes it down as a chord: a particular stack of pure tones played together. "Cat" is one chord, "dog" another. Chords that sound alike mean alike, so related ideas cluster on their own — "car" ends up humming right next to "streets."

How many tones can it stack into a chord? That's the size of its instrument. A small instrument has few strings, so every chord blurs into the same muddy sound and the model just repeats itself. A bigger instrument keeps the chords crisp and distinct, and suddenly it can tell everything apart.

In the model: the tones are the dimensions — the strings of the instrument. More strings, crisper meanings.
02 — Two chords, played together

To combine ideas, it plays them at once.

The model has essentially one core move: to join two things, it plays their chords together and they merge into a single new chord. That's how it links a word to its role, a question to its subject.

Here's the trick that makes it special. Numbers are chords tuned so cleverly that playing the chord for 6 together with the chord for 9 literally rings out as the chord for 15. The answer is in the sound itself — the model never learned a times-table, the arithmetic falls out of how the tones combine. An ordinary transformer has no such luck: it has to memorise "6 + 9 = 15" like a flashcard.

In the model: this one move is binding. For numbers, binding is addition and multiplication, for free.
03 — Chord moves

It learns a handful of ways to change a chord.

Stacking chords isn't enough — the model also needs to transform them, to steer a question-chord toward its answer-chord. It does this with a small set of learned moves, each one reaching out and blending in tones from elsewhere on the instrument.

The more moves it has, the richer the relationships it can express — simple harmony with a few, real reasoning with more. And the moves are spread out to reach right across the instrument rather than only nudging neighbouring strings, so it can connect distant ideas, not just adjacent notes.

In the model: the moves are the shifts, spread golden across the tones so a few of them cover the whole range.
04 — A chain of arrangers

The chord passes down a line.

The sound isn't shaped all at once. It flows through several stages in a row, and each stage restyles it with its own set of moves, adding a little more structure before handing it on — the way a melody might pass through a chain of arrangers, each one enriching it.

In the model: the stages are the layers. Depth is how many times the sound gets reworked.
05 — Play it through twice

Thinking, for this model, is replaying.

Here's the surprising part, and it happens inside every one of those arrangers. Each one doesn't play its part just once — it plays it again over the top of what it just laid down, and only then hands the sound on to the next. The first pass sketches a rough sound; the second develops that sketch into the actual answer.

Tested on the real model: with a single pass its output is noise, and only on the second pass does the music — the usable answer — appear. So the two ideas stack: a chain of arrangers, each one replaying its own part, and the model can "think harder" about a short question simply by playing it through more times, with no extra information.

In the model: these repeats are the K iterations. More passes (trained in) means deeper thinking on the same input.
06 — One shared sound you tune into

It remembers like a radio, not a room.

This is where it parts ways most sharply with an ordinary transformer. A transformer makes every word listen to every other word — a crowded room where everyone talks to everyone at once. It works, but the effort explodes as the conversation grows.

This model instead folds everything it has heard into one running sound. To recall something, it "tunes in" to a frequency, and the matching part rings out clear while everything else cancels to silence — exactly like finding one station on a radio dial. One sound to hold it all, one dial to find any piece. Cheaper, and you can actually listen to what it's holding in mind.

In the model: this is holographic memory — retrieval by resonance instead of everyone-compares-to-everyone.
07 — The interference does real work

When many chords pile up, new tones appear.

Play a lot of chords into one sound and the tones start to interfere — some reinforce, some clash, and the overlap conjures faint new tones that nobody actually played: the shimmer and beats you hear inside a rich chord.

The key idea behind this model is that the interference isn't just noise to clean up — it is quietly doing the computation itself, for free, in the overlap. And this is proven, not a hunch. In one test, a model with only five numbers to its name untangled the "double spiral" — a famously hard puzzle where two spiral arms coil tightly around each other and have to be told apart, the kind of thing that normally needs a network hundreds of times larger. Five numbers can't do that on their own; the interference between the packed-together tones did the work.

In the model: this is crosstalk as compute — the overlap between packed-together ideas doing useful work (the double spiral, solved in 5 parameters).

Why it isn't just a small transformer

side by side

An ordinary transformer

  • Meaning is a dot on a vast map.
  • To combine ideas it uses huge learned tables — it has to be taught every combination, including 6 + 9.
  • Its thoughts are silent and unreadable, just numbers.
  • Every word compares to every other word — powerful but costly.

The holographic transformer

  • Meaning is a chord — a sound.
  • To combine ideas it just plays them together, so arithmetic and composition come free from the physics of sound.
  • Its thoughts are audible — you can hear, and read off, what it means.
  • Everything folds into one sound it tunes into — cheaper, and it thinks by replaying.

Because the hard parts — combining, counting, remembering — are built into the medium instead of learned from scratch, this model can reason and do exact arithmetic at a tiny size, where an ordinary model that small could only babble.

An ordinary transformer memorises the music.
This one plays it.

The holographic transformer ships as part of AlgFormer (EvaluatedApplications.AlgFormer on NuGet), built on the Phasor codec. Every metaphor here maps to a real mechanism — phasors truly are the mathematics of sound.

← Back to AlgFormer