[lpalbou]

AbstractCore 2.31.1 (backlog 0859; framework release-closing QA item 4).

Bug: on a fresh Linux install, /v1/chat/completions (which AbstractGateway serves through this app) built a SECOND full llama.cpp instance per request for a model the gateway's runtime pool already held. It failed with 500 "Failed to create llama_context" on a 16 GB GPU. On Metal: holders 1 -> 2 -> 3 -> 4 and RSS 11.2 -> 43.4 GB over three calls.

Fix

• The chat route lends the process's alive huggingface instance of the same model (process_residency.resident_provider).
  • It never lends with unload_after, request-specific provider settings (explicit key, base_url), a request-scoped instance, MLX or embeddings.
  • A lent call runs off the event loop, and a client disconnect cancels it.
• Every operation on a HuggingFace instance's model runs on one worker thread of that instance: generations, prompt-cache ops, and the unload's free step.
  • A stream is produced into a queue, so it never waits for its consumer.

Reviews: two adversarial reviews (Opus, read-only), plus a verification review of the round-2 fixes. Findings and fixes are in backlog 0859.

Tests

• tests/providers/test_hf_instance_generation_gate.py (11) and tests/server/test_server_reuses_resident_provider.py (2). They go red when the worker or the fixes are removed (see 0859).
• Full suite locally: 6731 passed. 2 failed, both pre-existing on main: the kit theme drift against a newer local abstractuic, and the bloc e2e fixture, which needs local tokenizers.

Metal, real weights (scratch gateway, unsloth/Qwen3.5-4B-GGUF): holders stay at 1 and RSS at 11.1-11.2 GB over 3 sequential and 3 concurrent /v1 calls (one streamed), all HTTP 200.

Also in 2.31.1 (backlog 0860, operator QA): the download catalog adds the LFM2 and LFM2.5 families, including the 24B-A2B models. Each model has GGUF artifacts (LiquidAI official and unsloth) and MLX artifacts (mlx-community, or LiquidAI's own where mlx-community has none). Every repo id and size was read from the Hugging Face API on 2026-10-28. The capabilities registry adds lfm2.5-24b-a2b and the repo aliases. test_the_lfm_families_are_in_the_catalog_with_gguf_and_mlx is red against the previous assets.

[lpalbou]

State (paused 2026-10-28)

Done

• PR AbstractCore 2.31.1: /v1 reuses the resident model (backlog 0859) #7 is merged as 8f3494a.
• Tag v2.31.1 is pushed.
• Release run https://github.com/lpalbou/AbstractCore/actions/runs/40781168440 was still in progress at the pause. abstractcore==2.31.1 was not yet on PyPI.

Contents

• Backlog 0859: /v1 lends the resident huggingface instance. A per-instance worker thread serializes every operation on the model. A lent call runs off the event loop.
• Backlog 0860: the LFM2 and LFM2.5 families in the download catalog.
• Three adversarial review rounds plus an LFM review. Every confirmed finding is fixed.

Tests

• PR CI: all 11 checks passed.
• Local full suite: 6731 passed. The 2 failures are pre-existing and caused by the local environment.
• Metal check on a scratch gateway: holders stay at 1 and RSS at 11.1-11.2 GB. Before the fix: holders 1 -> 4 and RSS 11.2 -> 43.4 GB.

Next

1. Confirm the release run is green.
2. pypi_visibility_gate.py --wait 900 abstractcore==2.31.1.

[lpalbou]

State update: release run https://github.com/lpalbou/AbstractCore/actions/runs/40781168440 succeeded, and abstractcore==2.31.1 is now visible on PyPI. Nothing is left to do on this PR. Downstream: AbstractRuntime PR #19 (0.15.3) has all checks green and is not merged yet (paused), and AbstractGateway draft PR #22 is waiting on it.