Repository navigation
Conversation
* Convert AA index v5 scores onto the v4.3.2 scale AA's leaderboard moved to Intelligence Index v5, so the fetcher has served only the snapshot. Fit v4.3.2 = slope * v5 + intercept on four releases read on both versions, stored in aa_index.json, and convert 5.x values before normalizing. Other versions still fall back to the snapshot. Map the 31 open v5 rows that had no entry. * Keep uploads scored and clamped scores out of the anchor Lists the five uploads that AA's per-benchmark results used to fill next to their originals in aa_names.json, so they keep a score once the index maps the release. A general score at 0 or 100 no longer joins the anchor that puts those results on the general scale. Adds gpt-oss-120b as a fifth v5 pair, now that AA gives it a v5 value. CHANGELOG entry under Unreleased. Refs #244
llama-modchip is a llama.cpp fork for large MoE models on the CPU. Its own quant types do not load in mainline: LUT-VQ3-64 is VQ3_0 at 3.00 bpw, and Fractal-64 reuses type 42 with a 30-byte block, which mainline sizes at 18 bytes and refuses. Standard GGUFs still load, so the card is a runner and lists VQ3_0 under `gguf.extra` the way ik_llama.cpp lists its own types. The platforms are the builds the fork actually ships: x86-64 AVX2 and AVX-512 prebuilt binaries plus a source build for ARM64 on Linux, a MinGW-w64 UCRT64 build on Windows, and the patched Metal engine on Apple silicon. CUDA is Windows only and experimental, the -Cuda switch of the build script, since install.sh builds no GPU engine and the CUDA and Vulkan binaries the Windows hybrid pack downloads are mainline llama.cpp's. There is no `probe`: the patched server is llama-server, so its /props reply is mainline's and would claim the port for this engine on any llama.cpp server. `detect_binaries` names the two files install.sh leaves in the clone, and the note says the installer does not put them on PATH. Refs #294
…#314) * fix(engines): keep CPU-only builds off rows placed on the GPU engines_for checked that an engine runs on the machine, not that it runs on the GPU. ik_llama.cpp, which has no ROCm or SYCL build and needs CUDA compute capability 7.5, was listed for GPU rows on AMD, Intel and older NVIDIA GPUs, where it runs only on the CPU. A row that puts weights on the GPU now counts only engines with a GPU build here. * test(engines): llama-modchip runs only on the CPU on Linux with an NVIDIA GPU
Thor's GPU is named "NVIDIA Thor" by NVML and nvidia-smi, without the "(nvgpu)" suffix of Orin and Xavier, and both report its memory as unsupported. The name matched no unified-memory marker, so the GPU was dropped. Recognise the name, map the T5000 module (p3834-0008) to "Jetson AGX Thor", and add its bandwidth and compute capability. Co-authored-by: Will Schenk <wschenk@thor.monkey-cardassian.ts.net> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
llama-modchip 3.0 gives its own quant types their own IDs (FRACTAL_64, VQ3_64), Strata 0.2.16 no longer runs DeepSeek V4.1 Flash, ik_llama.cpp loads mainline FP8 GGUFs, vLLM 0.34.0 ships ROCm 10.0 wheels only and Lemonade's CUDA build checks the driver. llama.cpp 1.0.0, TurboFieldfare 0.10.0 and WARP 0.8.2 move only their versions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This brings v0.6.9 into main for the 0.6.9 release. The branch holds #308, #309, #314, #316, #317 and the release prep #319, each squash-merged with CI passing.
#309 reads Artificial Analysis scores live again by converting AA's index v5 onto the v4.3.2 scale, which moves the top of the ranking. #308 adds an engine card for llama-modchip. #314 stops a row on the GPU from listing an engine that runs only on the CPU there. #316 detects a Jetson AGX Thor as a GPU, and #317 re-reads eight engine cards.
I'm merging with a merge commit, not a squash, so main keeps each PR as its own commit. main has one commit v0.6.9 doesn't, the README badge from #318, and the two touch different files, so there is nothing to resolve.
#309 changes how
scripts/build_index.pyreads AA scores, so the index picks it up on its next build from main.