Skip to content

mesh-llm-git: new split package (mesh-llm, skippy, CPU and Vulkan runtimes) - #2

Draft
kleinbem wants to merge 6 commits into
mainfrom
feat/mesh-llm-git
Draft

kleinbem wants to merge 6 commits into
mainfrom
feat/mesh-llm-git

Conversation

@kleinbem

@kleinbem kleinbem commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

-git pkgbases that build Mesh-LLM main. They sit next to the release pkgbases in #9 and provide and conflict with the same names.

pkgbase packages
mesh-llm-git mesh-llm-git, skippy-git, skippy-native-runtime-git (CPU), skippy-native-runtime-vulkan-git
skippy-native-runtime-cuda-git CUDA 13 runtime (cudart + cuBLAS bundled, needs only nvidia-utils)
skippy-native-runtime-rocm-git ROCm runtime

Test results:

  • CI on GitHub runners: all three pkgbases built, with namcap clean apart from the expected !strip warnings.
  • Arch distrobox: mesh-llm runtime list verifies all four bundles with symlinked sonames, and skippy serve answers on the Intel iGPU (Vulkan0).
  • CUDA and ROCm are build-tested only: no NVIDIA or AMD machine running Arch.

@kleinbem
kleinbem force-pushed the feat/mesh-llm-git branch 2 times, most recently from 5cdbc3c to 2344de0 Compare October 23, 2026 01:25
kleinbem added a commit that referenced this pull request Nov 11, 2026
…0.79.0

The release counterparts of the -git pkgbases in #2: one split pkgbase
(mesh-llm, skippy, CPU and Vulkan runtimes) and one per GPU runtime,
built from the v0.79.0 tag tarball and the llama.cpp release it pins
(b12199), with checksums. prepare() fails if the two pins drift apart.

Each runtime package also installs llama.cpp's MIT license, and each
directory carries the 0BSD LICENSE the AUR expects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kleinbem added a commit that referenced this pull request Nov 11, 2026
…0.79.0

The release counterparts of the -git pkgbases in #2: one split pkgbase
(mesh-llm, skippy, CPU and Vulkan runtimes) and one per GPU runtime,
built from the v0.79.0 tag tarball and the llama.cpp release it pins
(b12199), with checksums. prepare() fails if the two pins drift apart.

Each runtime package also installs llama.cpp's MIT license, and each
directory carries the 0BSD LICENSE the AUR expects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kleinbem added a commit that referenced this pull request Nov 11, 2026
…0.79.0

The release counterparts of the -git pkgbases in #2: one split pkgbase
(mesh-llm, skippy, CPU and Vulkan runtimes) and one per GPU runtime,
built from the v0.79.0 tag tarball and the llama.cpp release it pins
(b12199), with checksums. prepare() fails if the two pins drift apart.

Each runtime package also installs llama.cpp's MIT license, and each
directory carries the 0BSD LICENSE the AUR expects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1

  • cr-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Central YAML; Path: .coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e99a8beb-acaf-4f6e-b62e-d03940b4ef9d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

kleinbem added a commit that referenced this pull request Nov 18, 2026
….0 (#9)

Release counterparts of the `-git` pkgbases in #2, pinned to Mesh-LLM
v0.80.0. Opened alongside #2 on main (not stacked): the directories
don't overlap. The AUR push is done by hand after the merge.

## What it replaces

The four conflicting AUR packages by luciddream (`mesh-llm`,
`mesh-llm-cuda`, `mesh-llm-rocm`, `mesh-llm-vulkan`, at 0.79.0) become:

| pkgbase | packages |
|---|---|
| `mesh-llm` | `mesh-llm`, `skippy`, `skippy-native-runtime` (CPU),
`skippy-native-runtime-vulkan` |
| `skippy-native-runtime-cuda` | CUDA 13 runtime (cudart + cuBLAS
bundled, needs only `nvidia-utils`) |
| `skippy-native-runtime-rocm` | ROCm runtime |

Runtimes are built from source into
`/usr/lib/mesh-llm/native-runtimes/`, where mesh-llm finds them without
downloads. `mesh-llm` conflicts with the old
`mesh-llm-{cuda,rocm,vulkan}` (all install `/usr/bin/mesh-llm`).

## Why llama.cpp is built here

mesh-llm needs llama.cpp at the commit it pins plus MeshLLM's patch
queue, so the runtimes are built in these pkgbases instead of using a
generic llama.cpp package. nixpkgs does the same
(`skippy-native-runtime`), and `extra/ollama` builds its vendored ggml
into `ollama-cuda`, `-rocm` and `-vulkan`.

## Sources

- `mesh-llm-0.80.0.tar.gz`, the tag tarball.
- llama.cpp `#tag=b12340` and whisper.cpp at the commit upstream pins,
both with checksums. `prepare()` fails if either drifts from
`skippy/llama_cpp/upstream.txt` or `whisper-upstream.txt` on a later
bump.

Still carried, each with a reason in the PKGBUILD: the cargo config
override (upstream builds through sccache and its own linker drivers),
`env -u SOURCE_DATE_EPOCH` for the runtime build (ggml turns off the x86
SIMD defaults when it is set), `_link_sonames` (Mesh-LLM/mesh-llm#3042)
and `!debug` (linking needs more than 16 GB with debuginfo).

## Checks

`check()` in the mesh-llm pkgbase runs the cargo tests and asks the
built `mesh-llm` to list the CPU and Vulkan runtimes built with it. The
CUDA and ROCm pkgbases check that the programs in the runtime's `tools/`
are executable: mesh-llm starts `mesh-llm-gpu-benchmark` itself, and
Mesh-LLM/mesh-packaging#47 is the same bug in upstream's own packages.
The same checks are in the nixpkgs package.

## Test results

CI run
https://github.com/kleinbem/aur-packages/actions/runs/44327180507, all
green:

| pkgbase | Build | namcap |
|---|---|---|
| mesh-llm | 48 min, cargo tests and runtime listing pass | 0 errors, 17
warnings (13 unstripped ELF from `!strip`, 2 unused libraries, 2
possibly unneeded dependencies) |
| skippy-native-runtime-cuda | 2 h 16 min, tools executable | 0 errors,
25 warnings (12 unstripped ELF, 13 unused system libraries linked by the
bundled cudart/cuBLAS) |
| skippy-native-runtime-rocm | 1 h 37 min, tools executable | 0 errors,
11 warnings (9 unstripped ELF, 2 unused libraries) |

Not tested on GitHub: loading the GPU runtimes (no GPU on the runners).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…CUDA license

The runtime bundle stores libfoo.so, .so.0 and .so.0.27.0 as three full
copies. The manifest check follows symlinks inside the bundle, so the
duplicates become symlinks: the CUDA runtime goes from 1.8 GB to 1.0 GB,
ROCm from 1.7 GB to 0.6 GB.

CUDA: the bundle carries cudart and cuBLAS with RUNPATH $ORIGIN, so it
needs the driver's libcuda.so.1 (nvidia-utils), not the 5.7 GB cuda
package; cuda moves to makedepends. Install NVIDIA's license next to
ours, as namcap asks for LicenseRef-NVIDIA-CUDA.

ROCm: libggml-hip links hipblas and amdhip64 only; drop rocblas.
…s from #9

Same changes as the release pkgbases, for upstream main:

- Override upstream's cargo config with CARGO_BUILD_RUSTC_WRAPPER= and
  CARGO_TARGET_<TRIPLE>_LINKER=cc instead of editing it with sed.
- Build whisper.cpp offline: it is a source now, and upstream's script
  checks out the commit in whisper-upstream.txt from the local clone.
- mesh-llm-git and skippy-git depend on the CPU runtime of the same
  build. The CUDA and ROCm -git runtimes aren't pinned: mesh-llm matches
  runtimes by RUNTIME_VERSION and Skippy ABI, not by app version, and
  they are built from separate pkgbases.
- An install script warns on CPUs without AVX2/FMA/F16C/BMI2
  (Mesh-LLM/mesh-llm#2691); 0BSD LICENSE for the package sources;
  workaround comments link upstream.
…e discovery and tool modes

Same checks as the release pkgbases from #9: mesh-llm-git lists the CPU and
Vulkan runtimes it built, and the CUDA and ROCm runtimes ship executable
tools (Mesh-LLM/mesh-packaging#47 is that bug in upstream's packages).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant