AlgFormer: a transformer engine for .NET

AlgFormer is a .NET library for defining, training and running transformer-style language models. Give it a vocabulary and a shape, train it on your own token sequences through a data-parallel pipeline, and run inference as a single prediction, as full logits or as streaming generation with a cache.

It is a real, gradient-trained architecture written entirely in C# on double arrays with SIMD. It runs anywhere .NET runs, with no GPU, no Python and no native binary to ship. Most transformer tooling means binding into a native runtime such as ONNX, PyTorch or CUDA, or hand-rolling matrix code with no attention research behind it. AlgFormer sits between the two.

It also lets you compare two ideas of what attention should cost. Softmax attention scores every token against every other token, which is quadratic in the length. The holographic core composes context by binding, bundling and unbinding instead. The two cores are parameter for parameter comparable, so you can train the same shape on the same data with either.

• Two real cores, not a toy

  AlgFormer (softmax dot-product attention, the well-understood baseline) and HoloFormer (holographic attention) share a model shape, a training loop, a checkpoint format and a serving API. You swap the core, not the workflow.

• No external ML runtime

  Everything is plain double arrays and SIMD, compiled into your app. There is no ONNX Runtime, no libtorch, no CUDA toolkit and no Python interop.

• Attention that scales differently

  HoloFormer reads a prompt in time that grows with its length rather than its square, and serves each further token at a constant cost through an incremental cache. That matters when context windows get long. The chart above measures both cores.

• Everything needed to use it

  A subword tokenizer, a data-parallel trainer, streaming generation with a cache, checkpoints with format versioning, and helpers that grow a trained model to a longer context or a bigger vocabulary without retraining from scratch.

• Built for swarm training

  Relay components let several machines train one shared model over MQTT, exchanging either full batches or lightweight position manifests.

using PrismFormer;

// Define a model. Tokens are ints; bring your own tokenizer or encoding.
var model = new AlgFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 32, frozenPrefix: 0);

// Train on (context tokens, target token) pairs.
var data = new List<(int[] Ctx, int Target)>
{
    (new[] { 1, 2, 3 }, 4),
    (new[] { 2, 3, 4 }, 5),
};
var trainer = new PrismTrainer(model);                 // data-parallel, run through EvalApp
double loss = trainer.TrainEpoch(data, batchSize: 2, lr: 5e-2, shuffleSeed: 1);

// Run inference.
var context = new[] { 1, 2, 3 };
int next        = model.Predict(context);                                    // argmax next token
double[] logits = model.LogitsFor(context);                                  // full logits
int[] generated = model.Generate(context, maxNewTokens: 5, temperature: 0.8, seed: 1);

Console.WriteLine($"loss={loss:F3} next={next} logits.Length={logits.Length} generated=[{string.Join(",", generated)}]");

loss=2.776 next=4 logits.Length=16 generated=[3,1,7,12,10]

This is the complete program, compiled and run against the published package.

The holographic core takes the same constructor and the same calls. This program trains it, serves it through the incremental cache, and reads what the final query resonated with.

using PrismFormer;

// The holographic core has the same constructor shape. Swap the type, keep the workflow.
var holo = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 32, frozenPrefix: 0);

var data = new List<(int[] Ctx, int Target)>
{
    (new[] { 1, 2, 3 }, 4),
    (new[] { 2, 3, 4 }, 5),
};
double loss = holo.TrainEpoch(data, batchSize: 2, lr: 5e-2, shuffleSeed: 1);

// Serve with the incremental cache: one running holographic memory per layer, so each token costs the same
// however long the context has become.
var cache = holo.NewCache();
double[] afterPrompt = holo.Prime(cache, new[] { 1, 2, 3 });
double[] afterNext = holo.Step(cache, 4);

// Read the model's mind: the resonance of the final query against every earlier position, and what it retrieved.
var alpha = new double[holo.Layers]; Array.Fill(alpha, 1.0);
int[] toks = { 1, 2, 3 };
double[][] resonance = holo.InspectAttention(toks, K: 2, alpha);
double[] retrieved = holo.InspectRetrieval(toks, K: 2, alpha);
double[] decoded = holo.DecodeFace(retrieved);          // per-token logits for what the query pulled out

Console.WriteLine($"loss={loss:F3} cache length={cache.Length} logits lengths {afterPrompt.Length},{afterNext.Length}");
Console.WriteLine($"resonance rows={resonance.Length}, positions per row={resonance[0].Length}, decoded logits={decoded.Length}");

loss=2.777 cache length=4 logits lengths 16,16
resonance rows=4, positions per row=3, decoded logits=16

• Two comparable attention cores

  AlgFormer (softmax) and HoloFormer (holographic resonance), built from the same shape parameters.

• Pure managed implementation

  double arrays and SIMD, with no external ML runtime, no native dependency and no GPU required.

• Data-parallel training

  A resource-gated, adaptively tuned pipeline for full epochs over your data, built on EvalApp.

• Streaming inference

  An incremental cache for low-latency generation. HoloFormer's cache holds one running holographic memory per layer, so each token costs the same however long the context is.

• Checkpoints that keep loading

  Save and load with backward-compatible format versioning, so old checkpoints keep loading as the format evolves.

• Growth in place

  Extend the context length, the attention shifts or the vocabulary of a trained model without starting again.

• Sizing from a measured law

  HoloShape chooses model dimensions from measured trade-offs. HoloShape.Derive turns a single width into a starting set of shifts, context length and layers.

• Tokenizer and data sources

  A built-in subword tokenizer and ready-made adapters for raw corpus windows, prompt and answer pairs and source mixing.

• Distributed training

  Relay components over MQTT, from full-model relay to a data-free relay of position manifests.

• Preference training

  Direct preference optimisation on the CPU HoloFormer: sequence log-probability over a chosen span, an exact clone for the frozen reference model, a signed per-sequence weighted all-positions step and a helper that returns the loss and the two weights.

• Gradient-check oracles

  Built-in checks let you verify the training maths instead of assuming it is right.

• ADAM and EVE

  An opt-in, per-layer Adam gate for HoloFormer that nudges a layer's update toward a norm range derived from the codebook. It is off unless you enable it through HoloFormer.EveOptions.

• EvaluatedApplications.AlgFormer.Gpu
• Release 1.12.0

AlgFormer.Gpu adds ILGPU and CUDA backed training and inference for HoloFormer. It builds device kernels straight from a CPU HoloFormer's serialized weights, so there is no separate model format and no retraining to switch devices. The backward pass, a fused Adam step on the device and batched forward inference all run on the GPU when one is present.

• A drop-in, not a rewrite

  Point it at an existing CPU HoloFormer. It trains and serves against the same serialized weights, and the CPU model stays the source of truth.

• One build for both machines

  Device detection runs once per process. On a machine without a CUDA GPU the same kernels run on ILGPU's CPU backend, slower but identical in behaviour, so there is no separate code path to maintain.

• Checked against the CPU engine

  Every kernel is gradient-checked against the CPU HoloFormer, which computes in double precision. GPU results run in single precision for speed, so they are close to the CPU reference rather than bit-identical.

• Adam on the device

  The optimiser step runs next to the forward and backward kernels, which avoids a round trip to the CPU on every batch.

A small HoloFormer wrapped by the GPU trainer. This run fell back to the CPU backend; with a CUDA device the first line names it.

using PrismFormer;
using PrismFormer.Gpu;

// Device auto-detect: a real CUDA GPU if present, otherwise ILGPU's own CPU backend running the SAME kernels.
Console.WriteLine(GpuDevice.Describe);   // "GPU: <name> (<MB> MB)" or "CPU (no CUDA GPU)"

// A small CPU HoloFormer. The GPU trainer wraps it, and the CPU copy stays the source of truth.
var model = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);

using var trainer = new GpuHoloTrainer(model);

var batch = new (int[] Ctx, int Target)[]
{
    (new[] { 1, 2, 3 }, 4),
    (new[] { 5, 4, 3 }, 2),
};
double loss = trainer.TrainBatch(batch, lr: 5e-2);   // forward and backward on the device, one Adam step, updates `model` in place

trainer.SyncToGpu();                                  // the device is one step behind the updated CPU model: push it across
float[] logits = trainer.Logits(new[] { 1, 2, 3 });   // inference on the device
Console.WriteLine($"loss={loss:F3} logits.Length={logits.Length}");

CPU (no CUDA GPU)
loss=2.783 logits.Length=16

Good to know

• Using the GPU path needs an ILGPU-supported CUDA device at run time. Without one, the same build still works on ILGPU's CPU backend.
• The package is optional and needs EvaluatedApplications.AlgFormer alongside it. The core package has no GPU dependency and stays safe to trim and to compile ahead of time.
• The device is one process-wide resource. Build, train and infer serially rather than from several threads at once.
• In fused-Adam mode the GPU holds the live parameters and the CPU model lags. Call SyncToCpu before saving or serving the CPU model.
• A HoloFormer built with a complex-bind feed-forward, with normalisation switched off, or with the unitary option is refused at construction instead of computing the wrong thing on the device.
• The package also contains the GPU twin of the older softmax-attention model. That path is deprecated: it builds and runs, but new work targets HoloFormer.

• The two cores share an API, not a behaviour. HoloFormer's capacity is set by its shape: a bundle holds a limited number of items cleanly, and HoloShape reports how many. A context longer than that reads back less cleanly.
• A shift count of 1 collapses the relation bank to a per-position diagonal and attention breaks. Keep at least two shifts and shrink the width or the layer count instead.
• A single model instance is not safe for concurrent training steps. Serving is read-only and safe to call from several threads on a model that is not changing, but a cache belongs to one generation on one thread.
• HoloFormer's serve-time knobs (Iters, IterAlphaServe, IterClean and IterTau) are not saved in a checkpoint. Set them again after loading.
• Step returns false when it drops a batch because the gradient was not finite. Check the result, or watch SkippedSteps, in a long training run.
• The ADAM and EVE gate is exercised in training only through AlgFormer.Gpu. On the CPU trainer it is covered by checks and has not been run as a training loop.
• The package ships as a binary with no source. It depends on Phasor and EvalApp, and the swarm relay uses MQTTnet.

Every public type is documented with a worked example in the package readme on NuGet. Open the NuGet page.

less time to read a 1,984-token prompt with holographic attention

26 times

less time for each further token, which stays flat as the context grows

50 times

smaller serve cache, constant in the length of the context

663 times

Proprietary. Every package ships as a compiled library. Evaluation and non-commercial development or testing are permitted; commercial, production or revenue-generating use needs a commercial licence from the owner. Read the licence terms.