[{"data":1,"prerenderedAt":256},["ShallowReactive",2],{"doc-\u002Fdocs\u002Fupstream-alignment":3},{"id":4,"title":5,"body":6,"description":248,"extension":249,"meta":250,"navigation":251,"path":252,"seo":253,"stem":254,"__hash__":255},"content\u002Fdocs\u002Fupstream-alignment.md","Upstream protocols and the mixed experiment",{"type":7,"value":8,"toc":241},"minimark",[9,13,30,35,64,71,74,77,81,214,221,224,227,230,238],[10,11,5],"h1",{"id":12},"upstream-protocols-and-the-mixed-experiment",[14,15,16,17,21,22,25,26,29],"p",{},"References are pinned to Laya ",[18,19,20],"code",{},"d113dca2512fb3eaca313534bc54c7162d87c1d4",",\nLaya Vision ",[18,23,24],{},"86ccec115ef3d72d1851168fc9b7194e8dbed35a",", and Bub main\n",[18,27,28],{},"9bf70488e22a44eeb875a9d55d9a7d352b82e381",". The training mixture includes upstream public tasks and soft teacher\nlabels. Existing Laya weights are benchmark references, not training targets.",[31,32,34],"h2",{"id":33},"publicly-documented-training-coverage","Publicly documented training coverage",[14,36,37,38,41,42,45,46,45,49,52,53,45,56,59,60,63],{},"Laya's pinned README and ",[18,39,40],{},"BENCHMARKS.md"," identify AG News, BoolQ, email spam,\nphishing, RAG relevance and support triage as in its training mixture. Its\napplication builders identify the corresponding public sources as\n",[18,43,44],{},"fancyzhx\u002Fag_news",", ",[18,47,48],{},"google\u002Fboolq",[18,50,51],{},"SetFit\u002Fenron_spam",",\n",[18,54,55],{},"zefang-liu\u002Fphishing-email-dataset",[18,57,58],{},"microsoft\u002Fms_marco",", and\n",[18,61,62],{},"Tobi-Bueck\u002Fcustomer-support-tickets",". This is evidence of declared source\u002Ftask\ncoverage, not a complete per-checkpoint training manifest, split audit or proof\nof identical training records for the English and multilingual checkpoints.",[14,65,66,67,70],{},"The typed-decisions fine-tuning notebook explicitly loads\n",[18,68,69],{},"LocalLLaMA\u002Ftyped-decisions\u002Fall\u002Ftrain",": 1,200 cases, 6,000 original decisions.\nThe published test is 400 cases and 2,000 decisions. Teacher-derived distributions\nmeasure agreement with those teachers rather than independently verified\nbusiness outcomes.",[14,72,73],{},"MASSIVE and XNLI appearing in the multilingual evaluation does not establish\nthat Laya trained on them. DAIR Emotion, SST-5, prompt-injections, toxic-chat\nand domain-based model routing are described as held-out evaluation tasks.\nThe public sources inspected do not provide a complete base-training ID list.",[14,75,76],{},"The support dataset's current card does not substantiate verified real enterprise\nticket provenance and advertises a synthetic ticket generator. Its CC BY-NC 4.0\nterms also differ from the Laya model license. Using a task in an upstream\nbenchmark does not certify its annotation quality or commercial training rights.",[31,78,80],{"id":79},"evaluation-alignment","Evaluation alignment",[82,83,84,100],"table",{},[85,86,87],"thead",{},[88,89,90,94,97],"tr",{},[91,92,93],"th",{},"Reference",[91,95,96],{},"Actual evaluation\u002Ftraining practice",[91,98,99],{},"Linnaeus treatment",[101,102,103,121,136,151,162,173,188,199],"tbody",{},[88,104,105,115,118],{},[106,107,108],"td",{},[109,110,114],"a",{"href":111,"rel":112},"https:\u002F\u002Fgithub.com\u002FNandhaKishorM\u002Flaya\u002Fblob\u002Fd113dca2512fb3eaca313534bc54c7162d87c1d4\u002FBENCHMARKS.md",[113],"nofollow","Laya benchmarks",[106,116,117],{},"MASSIVE uses 20 candidate intents across languages; XNLI uses three entailment labels; public classifiers include AG News, emotion and BANKING77",[106,119,120],{},"Train\u002Fevaluate MASSIVE en-US and zh-CN with all 60 intents; XNLI en\u002Fzh; AG News 4, emotion 6, BANKING77 77. These are not the published 51-language\u002F20-candidate MASSIVE setting.",[88,122,123,130,133],{},[106,124,125],{},[109,126,129],{"href":127,"rel":128},"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FLocalLLaMA\u002Ftyped-decisions",[113],"typed-decisions",[106,131,132],{},"1,200 training cases \u002F 6,000 decisions and 400 test cases \u002F 2,000 decisions; labels average three teacher samples",[106,134,135],{},"Keep state-level groups and soft distributions. Results measure teacher agreement, not independent real-world correctness. Reserve train groups for development and calibration.",[88,137,138,145,148],{},[106,139,140],{},[109,141,144],{"href":142,"rel":143},"https:\u002F\u002Fgithub.com\u002Fr33drichards\u002Flaya-vision\u002Fblob\u002F86ccec115ef3d72d1851168fc9b7194e8dbed35a\u002Fhf_model_card.md",[113],"Laya Vision model card",[106,146,147],{},"A-OKVQA four-choice, ScienceQA image subset, VQAv2 yes\u002Fno soft targets; equal source sampling",[106,149,150],{},"Add all three to the mix. Use frozen vision plus LoRA instead of updating the entire language backbone.",[88,152,153,156,159],{},[106,154,155],{},"Laya Vision VQAv2",[106,157,158],{},"Published run re-splits official validation by image; 50k training \u002F 5k evaluation examples",[106,160,161],{},"Download that labeled validation source, split by image hash and audit COCO overlap with A-OKVQA. Our ID list and counts differ; do not present as an exact published-result reproduction or official VQA score.",[88,163,164,167,170],{},[106,165,166],{},"Laya Vision calibration",[106,168,169],{},"Per-type temperature, top-label accuracy, NLL, 15-bin ECE; training-tail calibration rows",[106,171,172],{},"Use an independent grouped calibration partition. ECE uses maximum probability, not entropy-based API confidence.",[88,174,175,178,185],{},[106,176,177],{},"Laya Vision quality checks",[106,179,180,181,184],{},"Four cyclic candidate rotations and image-dependence tests; published ",[18,182,183],{},"score"," head untrained",[106,186,187],{},"Preserve these audits; add supervised CLEVR count\u002Fattribute\u002Fexistence and ScreenQA region decisions. Counting alone does not validate subjective ordinal judgments.",[88,189,190,193,196],{},[106,191,192],{},"Laya latency",[106,194,195],{},"Batch sizes 1, 5, 10, 50; published T4 numbers",[106,197,198],{},"Measure the local RX 7900 XTX, with model-specific token counts and all timing samples. Published Jev figures are third-party references, not a local Jev benchmark.",[88,200,201,208,211],{},[106,202,203],{},[109,204,207],{"href":205,"rel":206},"https:\u002F\u002Fgithub.com\u002Ffstandhartinger\u002Fjevbench\u002Ftree\u002Fv1.2.2",[113],"JevBench v1.2.2",[106,209,210],{},"534 decisions; intelligence, calibration, speed, and cost carry equal weights",[106,212,213],{},"Run all 231 public tasks with the frozen checkpoint and official harness. Compare reference accuracy on identical IDs. The other 303 tasks and a local provider tariff are unavailable, so no official rank or full composite is reported.",[14,215,216,217,220],{},"Dataset revisions, file sizes, URLs and SHA-256 hashes are stored in\n",[18,218,219],{},"data\u002Fmanifests\u002Fupstream-downloads*.json",". Source conversion uses raw annotations\nand Parquet, not downloaded executable dataset scripts. The official A-OKVQA\nannotation release supplies COCO IDs missing from its HF Parquet conversion.",[14,222,223],{},"Train, development, calibration and final evaluation groups are disjoint after\njoining source IDs, normalized text, pixel hashes, and COCO IDs. If official or\nindependently derived partitions conflict, retain the higher-priority partition\n(test > development > calibration > train) and exclude lower-priority examples.\nReport these exclusions rather than silently moving rows into the test set.",[14,225,226],{},"All 77 BANKING77 labels fit the API's 128-candidate limit. A separate\n2,048-token training budget determines dataset eligibility. The Qwen3.5 adapter\nsupports 4,096-token inference. Eligibility is frozen before training; no\ncandidate or question is truncated to manufacture a passing result. Screening\ncounts and excluded IDs accompany accuracy.",[14,228,229],{},"ScienceQA adds noncommercial\u002Fshare-alike source terms to the research mix.\nThe typed-decisions source explicitly warns that a better model can disagree\nwith its small teacher. Model quality claims must therefore separate human,\nprogrammatic, and teacher-derived targets. Annotation terms alone do not settle\nthe license of a released mixed checkpoint.",[14,231,232,233,237],{},"ScienceQA uses the official ",[234,235,236],"strong",{},"test"," image subset (2,017 rows) for our final\nevaluation. Laya Vision's reported 2,097-row set corresponds to official\nvalidation. Our development partition starts from that validation split, with\ncross-partition image overlap removed; the published accuracy is not a directly\nmatched final-test baseline.",[14,239,240],{},"Local upstream checkpoint references also have a training-exposure limitation:\nour A-OKVQA development rows come from the official training set used by the\npublished Vision model, and our VQAv2 split is independently derived from the\nsame validation pool that upstream re-split for training. These scores are useful\nimplementation checks, but they do not establish an uncontaminated head-to-head\ncomparison. Official held-out partitions and any verifiable upstream training\nID exclusions must be identified before making a competitive quality claim.\nReference CPU evaluation uses the published FP32 weights and temperatures, with\n2048-token total\u002Fhead budgets to retain all candidates; upstream still applies\nits 48-token per-option cap. Linnaeus uses BF16 with its own full-candidate template.",{"title":242,"searchDepth":243,"depth":244,"links":245},"",2,3,[246,247],{"id":33,"depth":243,"text":34},{"id":79,"depth":243,"text":80},"References are pinned to Laya d113dca2512fb3eaca313534bc54c7162d87c1d4,\nLaya Vision 86ccec115ef3d72d1851168fc9b7194e8dbed35a, and Bub main\n9bf70488e22a44eeb875a9d55d9a7d352b82e381. The training mixture includes upstream public tasks and soft teacher\nlabels. Existing Laya weights are benchmark references, not training targets.","md",{},true,"\u002Fdocs\u002Fupstream-alignment",{"title":5,"description":248},"docs\u002Fupstream-alignment","6gyc8l1idnl6ZlTj7WyhQuJPH4r9lE-UjAPtWTvsBZw",1790560641968]