AlgFormer (softmax)
Classic dot-product attention — the well-understood baseline. Every dense map is an
algebraic relation-bank cell (S·d params, not d²), so it stays compact even
in pure managed code.
AlgFormer defines, trains, and runs transformer-style language models in pure managed .NET —
double[] arrays with SIMD, no ONNX, no libtorch, no CUDA toolkit, no Python interop. Give it a
vocabulary and a shape, train it on your own token sequences through a data-parallel pipeline, and run
inference — single prediction, full logits, or streaming generation with a KV cache. It ships two
interchangeable attention cores so you can compare what "attention" costs.
dotnet add package EvaluatedApplications.AlgFormera real transformer, no native runtime
Most transformer tooling means either binding into a native runtime (ONNX, PyTorch/libtorch, CUDA) or hand-rolling matrix code with no attention research behind it. AlgFormer sits in between: a real, gradient-trained transformer architecture, written entirely in C#, that runs anywhere .NET runs. It also lets you compare two different ideas of what attention should cost. Standard softmax attention scores every token against every other token — quadratic in sequence length. AlgFormer ships a second, holographic attention core alongside it that composes context through bind/bundle/unbind operations instead of pairwise scoring — linear in sequence length, with constant-time-per-token serving. Both cores are parameter-for-parameter comparable, so you can train the same shape on the same data with either and see the difference directly.
swap the core, not the workflow
Classic dot-product attention — the well-understood baseline. Every dense map is an
algebraic relation-bank cell (S·d params, not d²), so it stays compact even
in pure managed code.
Attention is bind(k,v) → causal-bundle → unbind(q) — holographic resonance instead of softmax·V. O(sequence length × model width) to train, effectively O(1) per token to serve via an incremental KV cache, versus O(sequence length²) for standard attention.
AlgFormer and HoloFormer share the same model shape, training loop, checkpoint format,
and serving API — construct one instead of the other with matching dimensions, and train/serve it through
the same code.
everything included to actually use it
Everything is plain double[] math with SIMD, compiled straight into your app. No ONNX Runtime, no libtorch, no CUDA toolkit, no Python interop.
HoloFormer's holographic core is useful when context windows get long — training cost grows linearly, serving cost per token stays effectively constant.
A subword tokenizer, a data-parallel trainer, streaming generation with a KV cache, checkpoint save/load with format versioning, and helpers to grow a trained model to a longer context or bigger vocabulary without retraining from scratch.
Relay components let multiple machines train a shared model cooperatively over MQTT, exchanging either full batches or lightweight position manifests.
what you get
double[] + SIMD — no external ML runtime, no native dependency, no GPU required.
Resource-gated, adaptively tuned, for full epochs over your dataset.
An incremental KV cache for low-latency, constant-work-per-token generation.
Backward-compatible format versioning, so old checkpoints keep loading as the format evolves.
Extend context length, attention shifts, or vocabulary on an already-trained model — HoloShape sizes it with measured, not guessed, tradeoffs.
Built in, so the training math itself can be verified, not just assumed correct.
minimal example
using PrismFormer;
// Define a model (tokens are ints — bring your own tokenizer/encoding)
var model = new AlgFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 32, frozenPrefix: 0);
// Train — data is a list of (context tokens, target token) pairs
var trainer = new PrismTrainer(model); // data-parallel via EvalApp
double loss = trainer.TrainEpoch(data, batchSize: 64, lr: 5e-2, shuffleSeed: 1);
// Run inference
int next = model.Predict(context); // argmax next token
double[] lg = model.LogitsFor(context); // full logits
int[] sample = model.Generate(prompt, maxNewTokens: 20, temperature: 0.8);Swapping in the holographic core is the same shape — construct a HoloFormer instead of an AlgFormer with matching dimensions, and train/serve it through the same API.
Compatibility: .NET 8.0+, pure managed code, runs anywhere .NET runs. Depends on the Phasor codec (model geometry defaults) and the EvalApp pipeline runtime (data-parallel training); swarm relay components use MQTTnet. License: proprietary, compiled library only — every capability is free to use today.