[{"data":1,"prerenderedAt":212},["ShallowReactive",2],{"doc-\u002Fdocs\u002Fcompute-efficiency":3},{"id":4,"title":5,"body":6,"description":16,"extension":205,"meta":206,"navigation":207,"path":208,"seo":209,"stem":210,"__hash__":211},"content\u002Fdocs\u002Fcompute-efficiency.md","Shared training and inference execution",{"type":7,"value":8,"toc":197},"minimark",[9,13,17,22,25,28,40,43,46,49,53,64,67,71,74,81,98,117,127,130,133,136,140,182,190,193],[10,11,5],"h1",{"id":12},"shared-training-and-inference-execution",[14,15,16],"p",{},"Linnaeus has one execution policy: differentiable FLA fusion, shared causal prefixes,\na bounded frozen-image cache, and asynchronous input\u002Foutput transfer. These are\nthe implemented defaults, with no public backend, stream-count, or cache switches.",[18,19,21],"h2",{"id":20},"runtime","Runtime",[14,23,24],{},"Qwen uses fused RMSNorm, RMSNorm with output gating, and SwiGLU in both training\nand inference. LoRA projection modules remain in the autograd graph; deployment\nmerges them into the frozen base. Vision patch projection uses a linear map and\ncausal convolution uses the installed Triton kernel. These kernels preserve the\nmathematical operations, but BF16 rounding need not match an unfused execution.\nTrain, development selection, calibration and test all use this runtime.",[14,26,27],{},"When questions share a sufficiently long causal prefix, the model computes that\nprefix once and forks both attention KV and delta-rule convolution\u002Frecurrent\nstates into independent question branches. All candidate positions remain in\nthe suffix. The split is aligned to 64 tokens. A request uses one batched model\ninvocation: internally the language backbone processes a shared prefix and\nthen parallel suffixes. It never generates an autoregressive token stream.\nRequests without a reusable prefix use a single batched language pass.",[14,29,30,31,35,36,39],{},"For B questions of length L sharing P tokens, dense projection work is\nproportional to ",[32,33,34],"code",{},"P + B × (L − P)"," rather than ",[32,37,38],{},"B × L",". Attention, recurrent state\nforking and launch overhead must still be measured; the token ratio is not an\nend-to-end speedup guarantee.",[14,41,42],{},"The prefix cache belongs to the current request and weight snapshot. Tensor\nreplacement preserves the autograd history. Training recomputes the complete\nshared request during activation checkpointing, constructing fresh cache state.\nGradients from all branches accumulate into the shared prefix. No language state\nsurvives an optimizer update or another request.",[14,44,45],{},"Frozen image features use a model-local 128 MiB LRU keyed by decoded RGB content\nand image dimensions under the adapter's fixed preprocessing policy. Batch\nmisses are encoded together, duplicate images once, and the same features serve\ntraining and prediction. They remain valid across LoRA updates because neither\nvision weights nor preprocessing changes. Loading a model creates a fresh cache.",[14,47,48],{},"CPU workers prepare pinned batches. One transfer stream prefetches the next\nbatch while the compute stream executes the current batch. Events preserve\nbatch order and memory lifetime. Evaluation overlaps result transfer and JSON\nserialization with the following forward. The optimizer sees four microbatches\nat the same parameter snapshot, averages their loss, clips once and updates once.",[18,50,52],{"id":51},"training-lifecycle","Training lifecycle",[14,54,55,56,59,60,63],{},"The training mixture and trainer serve base initialization and checkpoint\ninitialization. ",[32,57,58],{},"--initialize-from"," supplies verified unmerged LoRA\u002Fhead weights;\nthe data, RLCD objective, schedule, selection and evaluation stages are identical.\n",[32,61,62],{},"--resume"," restores the current run's optimizer, random state and update count.\nThe development baseline at step zero is an eligible selected checkpoint.",[14,65,66],{},"RLCD optimizes the policy produced by the fused runtime. It does not replace\ncorrect dependency handling or held-out quality measurement. Calibration follows\nLoRA merge on an independent partition. Test and benchmark scores never select\nweights or tune temperatures.",[18,68,70],{"id":69},"hardware-bounds","Hardware bounds",[14,72,73],{},"The upstream analysis uses the RX 7900 XTX; the recorded Linnaeus-0.1.0-2B\nrun trained 3,600 updates in 205.7 minutes on a single RTX 4090 24GB\n(OpenBayes), peaking at 23.9 GiB VRAM with fused kernels and shared-prefix\nbatching. The bounds below remain upstream's reference analysis.",[14,75,76,77,80],{},"The RX 7900 XTX has 96 compute units, a reference boost frequency of 2.5 GHz,\n24 GB GDDR6 and 960 GB\u002Fs external memory bandwidth. AMD documents 512 BF16\nmatrix FLOPs per clock per compute unit. The dense BF16 matrix ceiling is\ntherefore ",[32,78,79],{},"96 × 2.5e9 × 512 = 122.88e12 FLOP\u002Fs",", counting a multiply-add as\ntwo operations. Its 80% target is 98.304 TFLOP\u002Fs. The corresponding bandwidth\ntarget is 768 GB\u002Fs. These are different constraints, not two simultaneously\nrequired utilization readings.",[14,82,83,84,91,92,97],{},"Sources: ",[85,86,90],"a",{"href":87,"rel":88},"https:\u002F\u002Fwww.amd.com\u002Fen\u002Fproducts\u002Fgraphics\u002Fdesktops\u002Fradeon\u002F7000-series\u002Famd-radeon-rx-7900xtx.html",[89],"nofollow","AMD device specifications","\nand ",[85,93,96],{"href":94,"rel":95},"https:\u002F\u002Fgpuopen.com\u002Flearn\u002Fwmma_on_rdna3\u002F",[89],"AMD WMMA architecture",".\nThe advertised cache-adjusted graphics bandwidth is not the GDDR bandwidth\navailable for streaming arbitrary model tensors. Sustained clocks and kernel\nshape efficiency must be measured on the actual machine.",[14,99,100,101,104,105,108,109,112,113,116],{},"For useful arithmetic ",[32,102,103],{},"F",", necessary memory traffic ",[32,106,107],{},"Q",", compute ceiling ",[32,110,111],{},"P","\nand bandwidth ",[32,114,115],{},"W",", an optimistic device lower bound is:",[118,119,125],"pre",{"className":120,"code":122,"language":123,"meta":124},[121],"language-text","T_lower = max(F \u002F P, Q \u002F W)\nthroughput_ceiling = decisions \u002F T_lower\nefficiency = measured_throughput \u002F throughput_ceiling = T_lower \u002F T_measured\n80% throughput target: T_measured \u003C= T_lower \u002F 0.8\n","text","",[32,126,122],{"__ignoreMap":124},[14,128,129],{},"A mixed graph needs separate BF16 matrix, FP32\u002Fvector, recurrent and transfer\nbounds, with dependencies between stages. Summing every device\u002Ftransfer bound\nassumes no overlap; taking just one global maximum assumes ideal overlap. A\ncritical-path model sits between these extremes. CPU preparation, queueing,\nserialization and cold compilation also belong in end-to-end measurements.",[14,131,132],{},"The arithmetic count must exclude unused vocabulary projections: Linnaeus gathers\ncandidate hidden states into a scalar head and never computes token-generation\nlogits. Embedding storage is not a dense matrix multiplication per token. Image\nencoding is counted only when an image is present. Reused prefixes or frozen\nimage features reduce useful work and require a new bound. Padding and repeated\nwork must be reported separately; increasing them cannot improve the useful-work\nefficiency claim.",[14,134,135],{},"Projection-only bounds omit attention, delta-rule recurrence, activations,\nnormalization and vision. They cannot establish whole-model efficiency or an\nexact training ETA. Hardware microbenchmark throughput is not API throughput.",[18,137,139],{"id":138},"measurement","Measurement",[118,141,145],{"className":142,"code":143,"language":144,"meta":124,"style":124},"language-bash shiki shiki-themes github-light github-dark","uv run python scripts\u002Fbenchmark.py linnaeus --checkpoint runs\u002Fv1\u002Fcheckpoint --output runs\u002Fv1\u002Fbenchmarks\u002Flinnaeus.jsonl\n","bash",[32,146,147],{"__ignoreMap":124},[148,149,152,156,160,163,166,169,173,176,179],"span",{"class":150,"line":151},"line",1,[148,153,155],{"class":154},"sScJk","uv",[148,157,159],{"class":158},"sZZnC"," run",[148,161,162],{"class":158}," python",[148,164,165],{"class":158}," scripts\u002Fbenchmark.py",[148,167,168],{"class":158}," linnaeus",[148,170,172],{"class":171},"sj4cs"," --checkpoint",[148,174,175],{"class":158}," runs\u002Fv1\u002Fcheckpoint",[148,177,178],{"class":171}," --output",[148,180,181],{"class":158}," runs\u002Fv1\u002Fbenchmarks\u002Flinnaeus.jsonl\n",[14,183,184,185,189],{},"The benchmark measures complete prediction calls over text and image workloads,\nincluding preprocessing and transfers. It records raw timings, throughput,\nmemory and board telemetry. See the ",[85,186,188],{"href":187},"local-benchmarks","measurement protocol","\nfor warmup, precision, batch sizes and cache conditions.",[14,191,192],{},"Training records loss, development quality and resource samples. Held-out\nevaluation establishes model quality. Eighty percent of the full workload\nroofline remains an engineering target that requires measurement.",[194,195,196],"style",{},"html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":124,"searchDepth":198,"depth":199,"links":200},2,3,[201,202,203,204],{"id":20,"depth":198,"text":21},{"id":51,"depth":198,"text":52},{"id":69,"depth":198,"text":70},{"id":138,"depth":198,"text":139},"md",{},true,"\u002Fdocs\u002Fcompute-efficiency",{"title":5,"description":16},"docs\u002Fcompute-efficiency","zPEgSDhAHWZzGuCCEJH3hBfRopxqRth3WX8YQY3cjok",1790560641967]