EvalApp governs concurrency for async .NET work. It fans a job out across far more operations than you have threads without launching them all at once, holds every backend at its real limit, and retries and compensates when things fail — as one compiled pipeline, one dependency, in place of a MediatR + Polly + Dataflow + DI stack.
A pipeline runtime for async .NET. You describe the work as a sequence of steps against a data type; EvalApp compiles that description into a single delegate and runs it — governing how much executes at once, holding every backend at its real limit, and recovering when a step fails. The fluent builder is the whole surface; everything below is opt-in and costs nothing on a pipeline that doesn't use it.
ForEach streams a collection through a worker pool at a concurrency Tunable.ForItems() tunes to the data. You keep far more in flight than you have threads, but never all of it — a million-row job runs at a fixed memory ceiling instead of launching a million operations at once.
Declare a real limit for each backend — a DB pool, an API cap, a CPU budget — and every stage that touches it contends for the same gate. One shared ceiling across the whole pipeline, so a flood can't oversubscribe a downstream service.
Nobody has to pick the in-flight number. A hill-climbing or Bayesian tuner watches wall-time across runs, finds the concurrency that maximises goodput, and holds it — persisted between runs.
Chain steps that each have an undo. When a later step fails, EvalApp runs the compensations in reverse order — the payment is refunded, the reservation released — so a half-finished job doesn't leave torn state behind.
Wrap every step with retry, timeout, circuit-breaker, validation, audit, timing, or mutation-tracking — built in and composable. The mutation tracker uses compiled expression trees, so it reads changed fields with zero runtime reflection.
The fluent tree is compiled once into one delegate; the checks a run doesn't need are branched away at build time, not re-tested per step. A pipeline step is cheap enough to stand in for a plain function call.
The tempting way to run an async operation over a big list is Task.WhenAll(list.Select(WorkAsync)). It launches every operation at once — every buffer, every open connection, live together — and on a big enough list that is an OutOfMemoryException or an exhausted pool. Bounding it is the easy part. The hard part is the number: cap it too low and the machine idles, too high and the backend falls over — and the right value shifts with the data, the hardware, and every downstream limit, so a constant you hard-code is wrong the moment any of those change. EvalApp's ForEach takes Tunable.ForItems(): instead of a number you guessed, the adaptive tuner starts hot, measures goodput, and settles on the in-flight count that drains this dataset fastest — then holds it across runs.
// launches all 1,000,000 at once —
// you own the crash
await Task.WhenAll(
rows.Select(ProcessAsync));// concurrency the tuner finds, then holds
task.ForEach(ProcessAsync, Tunable.ForItems());
await pipeline.RunAsync(rows);200,000 async operations, each holding an 8 KB buffer across a 300 ms wait. Every bounded approach here is pinned to the same 10,000 so the memory comparison is fair — peak in flight × buffer = live memory. The naive WhenAll is fastest precisely because it is reckless; at a few million rows or with real connections that speed is the crash. The difference shows in your own code: Parallel.ForEachAsync, a hand-rolled semaphore and TPL Dataflow all make you pick that 10,000 and live with it, while Tunable.ForItems() lets the tuner find it and keep it there (the twelve-drain proof is below) — inside the same governed pipeline that gates your backends and retries your failures.
Bounding the fan-out still leaves the question of how many run at once, and that number isn't a preference — it's the peak of a goodput curve specific to this workload and this hardware. Below it the machine idles; above it the backends oversubscribe and shed. On the load below, the peak sits at 96 in flight — 723 useful/sec, drained in 1.0 s. Guess 8 and you get 82 useful/sec and an 8.6 s drain. Guess 256 and it collapses to 234, requests timing out. The only way to find 96 by hand is to run exactly this sweep — and re-run it every time the load or a backend changes.
Goodput vs a fixed in-flight level, same 700-request backlog through backends that inflate latency and time out when oversubscribed. This is the hand sweep you'd run to find the number — and past ~128 in flight, useful throughput falls off a cliff.
Same workload, but nobody sets the number: Tunable.ForItems() and the adaptive tuner probe the in-flight level across drains. It starts cautious, explores, and locks onto ~690 useful/sec — within a few percent of the 723 the hand sweep found — then holds it. The cost of not sweeping by hand is honest and on the chart: a couple of exploratory drains overshoot into the drop zone before it settles. You pay that once, you never pick the number, and it re-finds the peak when the load shifts.
EvalApp's built-in tuner, 30 consecutive drains, in-flight range [4, 512] starting at 32. The dashed line is the hand-swept peak (723 useful/sec). It converges by drain ~26 and holds there — nobody set the value.
Fan-out solves how much you run at once; the other half is what each stage is allowed to touch. Point a burst of 8,000 requests at a service whose five stages each sit behind a real limit — a DB pool, an API cap, a CPU budget. Fire it all with no governance and the raw throughput looks great, until you count what finished: the backends oversubscribe and time out. The bars below are total throughput; the coloured part completed, the red part was dropped.
Ungoverned code pushes ~5,000–6,000 req/s but times out ~97% of it (it drove the 28-connection DB pool to 7,800). Its useful throughput is ~175 req/s. EvalApp holds every resource at capacity and completes all 8,000 — same throughput as a hand-rolled SemaphoreSlim, Polly, or a MediatR+Polly stack, and zero drops.
Every governed approach lands the same ~1,100 req/s with zero drops. What differs is the allocation each carries per request — the framework overhead that compounds under load. Matching EvalApp's behaviour from libraries means composing MediatR (dispatch + behaviours) with Polly (concurrency + retry + timeout); that stack allocates 2.7× what EvalApp does in one compiled pass.
Bytes allocated per request while draining the backlog. A hand-rolled SemaphoreSlim is leanest — it does one thing in five lines; EvalApp runs the whole service (five resources, retry/timeout/saga/middleware, and auto-tuning) and still comes in at a third of the composed stack.
Discovering a hidden optimum takes exploration (above). When the ceiling is instead a real backend cap — a DB pool, an API limit — there's nothing to hunt for: the same tuner behind Tunable.ForItems() fills straight to it and holds. Here the five gated backends bound the service; drain the same backlog twelve times and throughput stays pinned at 1,097–1,099 req/s from the first run on, without anyone ever picking a value.
What resource-governed, resilient async handling costs you to assemble.
Bounded async fan-out, resource gates, sagas with compensation, middleware, and an adaptive tuner — in a single dependency, one compiled pipeline, ⅓ the allocation.
Every architecture through the same 8,000-request backlog. "DB pool" is the peak concurrent connections it drove against a pool whose real limit is 28 — over that number means it oversubscribed the backend.
Deterministic backlog, identical base latencies; only concurrency management differs. SemaphoreSlim, Polly, Dataflow, MediatR+Polly and EvalApp all hold every resource at capacity with zero errors — they differ in overhead and in how much has to be assembled by hand.
If the whole problem is "cap one resource at N," a hand-rolled SemaphoreSlim wins — 1,099 req/s, 3.0 KB/req, nothing to learn. And for a small list, Task.WhenAll is fine and fastest. EvalApp is not built to beat those on raw speed at small scale, and it doesn't.
Many stages, many resources, retries, timeouts, compensation, and a fan-out too big to run all at once — that is where a stack of libraries becomes glue you own forever. EvalApp is one dependency that bounds the fan-out, holds every backend at its limit, drops nothing, allocates 2.7× less than the composed stack, and tunes and holds the concurrency for you.
Two load tests, not microbenchmarks. The fan-out runs one async operation (an 8 KB buffer held across a 300 ms wait) over 200,000 items, capping in-flight at 10,000, and records peak concurrent operations and their live memory. The soak drains a deterministic backlog of 8,000 requests through a five-stage service (Cache / DbRead / Cpu / Api / DbWrite) where each stage inflates latency and times out at 750 ms once its concurrency limit is exceeded — so oversubscribing a backend costs you, as it does in production.
Measured on an Intel Core i9-11950H (8 cores / 16 threads), 32 GB, .NET 8, ServerGC. Comparables: MediatR 12.4, Polly 8.4, TPL Dataflow 8.0. Hardware moves the numbers; the shape holds.
The shared runtime under the Evaluated Applications libraries, and usable on its own.