From 5d16ebe64602fde12059738c10c2f29343b2b60e Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 15:58:56 +0100 Subject: [PATCH 01/13] docs(2.x): document the search relevance add-on and search pipelines Adds the Search Relevance add-on page and its admin panel section page, a Pipelines section to the search add-on page, a search relevance section to the extending search page, a Make Search Learn subsection to the search guide, and a flight plan entry. Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 370 ++++++++++++++++++++++++++++++++ 2.x/addons/search.mdx | 153 +++++++++++++ 2.x/admin/search-relevance.mdx | 96 +++++++++ 2.x/extending/search.mdx | 175 ++++++++++++++- 2.x/guides/search.mdx | 20 ++ docs.json | 6 +- logs/flight-plan.mdx | 10 + 7 files changed, 828 insertions(+), 2 deletions(-) create mode 100644 2.x/addons/search-relevance.mdx create mode 100644 2.x/admin/search-relevance.mdx diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx new file mode 100644 index 0000000..a8e47f1 --- /dev/null +++ b/2.x/addons/search-relevance.mdx @@ -0,0 +1,370 @@ +--- +title: "Search Relevance" +sidebarTitle: "Search Relevance" +description: "Learned ranking for Lunar search: log what shoppers click, add to cart, and buy, score it nightly, and reorder results without leaving the search engine." +--- + +The Search Relevance add-on (`lunarphp/search-relevance`) makes Lunar search improve with traffic. It logs every search and the clicks, cart additions, and purchases that follow, scores each query and product pair on a schedule, and reorders the results the engine returns so products shoppers actually want rise for the queries they type. It also fixes part-number search on Typesense and Meilisearch, and ships an admin panel section for reading the data. + +The add-on is built on the request and results [pipelines](/2.x/addons/search#pipelines) of the `lunarphp/search` add-on, which must be installed first. + +## What it does, and does not do + +The add-on works on the ordered list of product ids the engine returns. It does not replace retrieval: + +- **The engine still decides which products match.** Keyword matching, typo tolerance, hybrid (vector) search, facets, and filters stay with Typesense, Meilisearch, or the database driver. The add-on only reorders what the engine found, plus a small set of learned products it fetches by id (see [Scoring](#scoring)). +- **Reordering is bucketed.** The ranker only moves products within a bucket of `bucket_size` hits (10 by default), so a weak keyword match can never leap above a strong one. Learned scores refine the engine order rather than overriding it. +- **It learns per query, not per product.** A product that is popular for `running shoes` gets no boost for `socks`. Global popularity is deliberately not a signal. +- **It never ranks sorted or empty queries.** An explicit `sort()` is respected as-is, and browse queries (empty or `*`) pass through untouched. + +## Supported engines + +| Engine | Ranking | Part-number retrieval | Notes | +|---|---|---|---| +| Typesense | Yes | Yes, prefix and infix | Full support. Hybrid search is padded by a distance threshold (see [Vector distance threshold](#vector-distance-threshold)). | +| Meilisearch | Yes | Yes, prefix only | No infix matching: `MB32A` will not match `HAG-MB-32A`. Stores whose customers search for fragments from inside a code need Typesense. Meilisearch also pads a page with partial matches after the full matches (see [Meilisearch page padding](#meilisearch-page-padding)). | +| Database | Yes | No | The Scout database driver has no per-field parameters, so the part-number stage has nothing to apply. Ranking works as normal. | +| Custom engines | Opt in | Opt in | A custom `Lunar\Search\Engines\AbstractEngine` subclass gets ranking once its `get()` calls `pipeRequest()` and `pipeResults()`; see [Pipelines](/2.x/addons/search#pipelines). | + +## Installation + +Require the package: + +```bash +composer require lunarphp/search-relevance +``` + +The service provider, `Lunar\SearchRelevance\SearchRelevanceServiceProvider`, is auto-discovered. It appends its three pipeline stages to `lunar.search.pipelines`, registers the storefront events route, the cart and order listeners, the Artisan commands, and the schedule. + +Run the migrations: + +```bash +php artisan migrate +``` + +This creates four tables, prefixed like every other Lunar table: `search_queries`, `search_events`, `search_query_scores`, and `search_relevance_settings`. It also seeds the `search:manage-relevance` permission used by the panel section. + +Make sure a queue worker and the scheduler are running. Logging, event recording, and attribution are dispatched as queued jobs, and scoring runs from the scheduler: + +```bash +php artisan queue:work +php artisan schedule:work +``` + + +The add-on works with the `sync` queue driver, but then every search writes its log row on the request path. See [Scheduling and queues](#scheduling-and-queues). + + +If the [admin panel](/2.x/admin/introduction) (`lunarphp/panel`) is installed, publish or link the add-on's compiled assets so its pages load: + +```bash +# Production: copies every registered add-on build, including this one +php artisan vendor:publish --tag=panel-all-assets --force + +# Local development: symlinks the build instead +php artisan lunar:panel:link +``` + +The panel section is documented under [Admin Panel: Search Relevance](/2.x/admin/search-relevance). Without the panel, everything else (logging, scoring, ranking, part-number retrieval) still works; the mode is then read from config only. + +Optionally publish the config: + +```bash +php artisan vendor:publish --tag=lunar.search-relevance.config +``` + +This writes `config/lunar/search-relevance.php`, merged under the `lunar.search_relevance` key. + +## Part-number search + +Shoppers who type a partial product code expect the whole family of that code, and nothing else. Under default engine settings a five-character prefix like `HAGMB` returns a few family members and then pads the page with unrelated products, because typo tolerance, token dropping, and semantic search all fill in for a "word" that means nothing. + +The add-on classifies each query before it reaches the engine. A query is a **part number** when it is a single token that mixes letters and digits, with hyphens, dots, and slashes allowed, and is not a bare unit such as `20mm`. Letter-only codes (`HAG-MB`) are treated as text on purpose: widening the rule would send ordinary words like `cable-gland` down the SKU-only path. + +For a part number, the `Lunar\SearchRelevance\Pipelines\PartNumberRetrieval` request stage restricts retrieval to the SKU fields and switches off everything that pads: + +| | Typesense | Meilisearch | +|---|---|---| +| Restrict to SKU fields | `query_by: skus,skus_normalised` | `attributesToSearchOn: ['skus', 'skus_normalised']` | +| Partial match | `prefix: true`, `infix: always` | Prefix on the last word is always on; no infix | +| No typos | `num_typos: 0` | Index setting `typoTolerance.disableOnAttributes`, applied by `lunar:meilisearch:setup` | +| Keep every token | `drop_tokens_threshold: 0` | `matchingStrategy: all` | +| No semantic padding | `vector_query` omitted | `hybrid.semanticRatio: 0`, when an embedder is configured | + +Part-number searches are still logged, but they are not ranked: per-code queries are too sparse to learn from, and the engine order is already the right one. The normaliser also skips stemming for them. + +The stage applies these parameters through `AbstractEngine::withParams()`, so nothing engine-specific lives in the engine itself. + +### The `skus_normalised` field + +`Lunar\Core\Search\ProductIndexer` indexes variant SKUs twice: `skus` as printed, and `skus_normalised` uppercased with every non-alphanumeric character stripped. That is what lets `HAGMB` prefix-match `HAG-MB-32A`. Both fields are part of core, so they are indexed whether or not this add-on is installed; the add-on is what makes retrieval use them. + +### Typesense setup + +Typesense needs both fields declared in the collection schema and listed in `query_by`, together with the position-aligned `infix` and `num_typos` lists. Both live in the host application's `config/scout.php`, under `typesense.model-settings`: + +```php +// config/scout.php +'typesense' => [ + // ... + 'model-settings' => [ + Lunar\Core\Models\Product::class => [ + 'collection-schema' => [ + 'fields' => [ + ['name' => 'id', 'type' => 'string'], + ['name' => 'name_en', 'type' => 'string'], + ['name' => 'description_en', 'type' => 'string', 'optional' => true], + ['name' => 'skus', 'type' => 'string[]', 'infix' => true], + ['name' => 'skus_normalised', 'type' => 'string[]', 'infix' => true], + ['name' => 'brand', 'type' => 'string', 'facet' => true, 'optional' => true], + ['name' => 'status', 'type' => 'string', 'facet' => true], + ['name' => 'created_at', 'type' => 'int64'], + // ... + ], + ], + 'search-parameters' => [ + 'query_by' => 'name_en,description_en,skus,skus_normalised', + 'query_by_weights' => '4,1,3,3', + 'infix' => 'off,off,always,always', + 'num_typos' => '2,2,0,0', + 'prefix' => 'true,true,true,true', + ], + ], + ], +], +``` + +Every entry in `infix`, `num_typos`, `prefix`, and `query_by_weights` is positional and must have exactly as many values as `query_by`, or Typesense rejects the request. Only `infix: true` in the schema enables infix search for a field; `always` in the search parameters then uses it. + +Changing the collection schema requires a reindex: + +```bash +php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh +``` + + +The `--refresh` flag drops and recreates the collection, which is what a schema change needs. Without it, Typesense keeps the old field list and the new fields are silently ignored. + + +### Meilisearch setup + +Meilisearch already prefix-matches the last word of a query, so recall of a SKU family is complete out of the box. Typo tolerance is the defect: under default settings an eight-character code returns several unrelated products per page, and none once typos are disabled on the SKU attributes. That is an index setting, not something a request can change, so the [`lunar:meilisearch:setup`](/2.x/addons/search#meilisearch) command applies it: + +```bash +php artisan lunar:meilisearch:setup +``` + +Re-run it after upgrading to a version of Lunar with `skus_normalised`. The command reads each indexer's `getExactMatchFields()` (`['skus', 'skus_normalised']` for products) and sets `typoTolerance.disableOnAttributes` on the index. Then reindex so the new field is populated: + +```bash +php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh +``` + +#### Meilisearch page padding + +Meilisearch's default `last` matching strategy lists every document matching all query terms first, then fills the rest of the page with documents matching fewer terms. For letter-only prefixes such as `HAG-MB`, which the classifier treats as text because they contain no digit, page one is the whole family followed by partial matches. This is not a ranking error, but a storefront that shows a result count will show an inflated total. Part numbers (with a digit) set `matchingStrategy: all` and are not padded. + +## Vector distance threshold + +Typesense hybrid search runs a `k: 200` vector query alongside the keyword query. Without a threshold, that vector query pads every result set with the 200 nearest neighbours of whatever the query was, so a nonsense query returns 200 products. Lunar sets `distance_threshold` on the vector query from `lunar.search.typesense.vector_distance_threshold` (default `0.6`; `0` disables it). This lives in the `lunarphp/search` config, not this add-on's, and applies whether or not the add-on is installed. See [Search: Typesense](/2.x/addons/search#typesense). + +Meilisearch's counterpart, when an embedder is configured, is `lunar.search.meilisearch.ranking_score_threshold`. + + +A nonsense query with a threshold in place can return zero results. Render a zero-results state in the storefront rather than loosening the threshold to pad the page; see the [search guide](/2.x/guides/search#make-search-learn). + + +## Modes + +The `mode` config key (or the `LUNAR_SEARCH_RELEVANCE_MODE` environment variable) controls how much the add-on does: + +| Mode | Logged | Ranked order computed | Ranked order displayed | +|---|---|---|---| +| `off` | No | No | No | +| `shadow` | Yes | Yes, stored beside the shown order | No, the engine order is shown | +| `on` | Yes | Yes | Yes | + +`shadow` is the default after install. Nothing changes for shoppers, but training data accrues from day one, and the [replay command](#replay) can prove the uplift before the switch to `on`. This is the recommended rollout: + +1. Install in `shadow`. Add the storefront tracking (below) so clicks are recorded. +2. Let the scoring job run for a few weeks. Check the panel's overview or `lunar:search-relevance:replay` for the share of searches where the ranked order placed the purchased product higher. +3. Switch to `on` when the replay shows a consistent improvement. + +The admin panel persists a mode override in the `search_relevance_settings` table. `Lunar\SearchRelevance\Settings::mode()` returns the override when one is set and the config value otherwise; every code path reads the mode through `Settings`, so a panel change takes effect immediately without a deploy. + +## Storefront tracking + +Ranking learns from what shoppers do with the results. A search is logged automatically; clicks need a small addition to the results page. + +### Blade storefronts + +Two lines. Once per results page, render the tracking component, which outputs the beacon script and CSRF wiring: + +```blade + +``` + +On each rendered result, spread the tracking attributes onto the element that wraps the product link: + +```blade +@foreach ($results->hits as $hit) +
+ + {{ $hit->document['name_en'] ?? '' }} + +
+@endforeach +``` + +`lunar_search_attrs()` renders `data-lunar-search-id`, `data-lunar-product-id`, `data-lunar-position`, and `data-lunar-source`. It returns an empty string when the search was not logged (for example, in `off` mode), so the markup is safe to leave in place. The script listens for clicks on any link inside a tracked element and sends the event with `navigator.sendBeacon()`, so a navigation that starts immediately still delivers it. + +### Headless and Inertia storefronts + +The same data is available in the results payload. `SearchResults->meta` carries `search_id`, `ranking_mode`, and `ranking_version`, and each hit's `meta` carries `position`, `original_position`, `boost`, and `source`. Post to the events endpoint when a shopper clicks a result: + +```js +fetch('/lunar/search/events', { + method: 'POST', + headers: { 'Content-Type': 'application/json', 'X-CSRF-TOKEN': csrfToken }, + body: JSON.stringify({ search_id: results.meta.search_id, product_id: hit.document.id, position: hit.meta.position, source: hit.meta.source }), +}); +``` + +The endpoint accepts: + +| Field | Required | Description | +|---|---|---| +| `search_id` | Yes | `results.meta.search_id` | +| `product_id` | Yes | The product id, as indexed under `document.id` | +| `position` | Yes | The 1-based position the product was displayed at (`hit.meta.position`) | +| `source` | No | `organic`, `learned`, or `explore` (`hit.meta.source`); defaults to `organic` | +| `session_id` | No | An explicit shopper identifier for API clients that carry no cart session cookie. Omit it from a browser and the server resolves the shopper from the cart session. | + +The endpoint validates that `search_id` exists, that `product_id` was in that search's shown list, and that `position` is in range, and it is rate limited per shopper and per IP (`guards.events_rate_limit`). Anything that fails validation is answered with an empty `204`, so a bot learns nothing about what was rejected. + +## Attribution + +A click is the start of a chain the add-on follows through to a purchase: + +1. The click stores `attribution.{product_id}` in the shopper's session (`search_id`, `position`, `source`, and an expiry) for `attribution_ttl_minutes` (30 by default). +2. When a `Lunar\Core\Models\CartLine` is created, `Lunar\SearchRelevance\Listeners\AttributeCartLine` resolves the line's product from its purchasable, looks up the attribution, and if one is present writes it to `meta['search_attribution']` on the line and records a `basket` event. +3. When `Lunar\Core\Events\Orders\OrderPlaced` fires, `Lunar\SearchRelevance\Listeners\AttributeOrderLines` records a `purchase` event for every order line whose `meta` carries `search_attribution`. Order creation already copies cart line `meta` to the order line, so no pipeline change is needed. + +A cart or order line attributed to a search carries this in `meta`: + +```php +$line->meta['search_attribution']; +// [ +// 'search_id' => '01J9X4M2K7Q8R1S3T5V7W9Y0Z2', +// 'position' => 3, +// 'source' => 'organic', +// ] +``` + +Shopper identity is the Lunar cart session identifier when `session_key` is `cart` (the default). It survives login, and it is what cart and order lines already relate to. Set `session_key` to `session` to use the Laravel session id instead. + +All three writes are queued jobs; nothing runs on the request path beyond the session write. + +## Scoring + +Scoring turns raw events into a score per query and product, stored in `search_query_scores`. It runs daily as `lunar:search-relevance:score`, and can be run by hand: + +```bash +php artisan lunar:search-relevance:score +``` + +In plain language, for each normalised query and product: + +- **Every event counts by type.** A click is worth 1, a cart addition 3, a purchase 5 (`scoring.weights`). +- **Events at low positions count more.** A click on the product at position 8 says more than a click at position 1, because position 1 gets clicked whatever it is. An event at position `p` is multiplied by `p ^ position_eta`, capped at `max_position_weight`. Without this correction the ranking learns position bias, cementing whatever the engine showed first. Events from the `explore` source skip the correction. +- **Recent events count more.** Each event's weight halves every `half_life_days` (30 by default), so a product that used to convert but no longer does fades out. +- **Sparse data is ignored.** Only events within `window_days` are considered, and a query and product pair needs events from at least `min_sessions` distinct shoppers before it gets a score. Sessions searching faster than `guards.max_searches_per_minute` are excluded as bots. +- **Scores are relative.** Within a query, each product's score is divided by the best score for that query, giving a `relative` value from 0 to 1. At most `max_products_per_query` products are kept per query. + +At search time, `Lunar\SearchRelevance\Signals\QueryAffinitySignal` looks up the `relative` score for each product in the window, `Lunar\SearchRelevance\Signals\SignalCombiner` combines every configured signal by weight and clamps to 0..1, and `Lunar\SearchRelevance\Rankers\BucketedRanker` reorders within each bucket by that score. Learned products the engine did not return (up to `learned_union.max`, each with `relative` of at least `learned_union.min_relative`) are fetched by id and inserted at the head of the second bucket, so they appear early without displacing the strongest keyword matches. The ranked window is cached for `cache_ttl` seconds. + +| Key | Default | Description | +|---|---|---| +| `scoring.weights.click` | `1` | Weight of a click event | +| `scoring.weights.basket` | `3` | Weight of a cart addition | +| `scoring.weights.purchase` | `5` | Weight of a purchase | +| `scoring.position_eta` | `0.7` | Exponent of the position correction; `0` disables it | +| `scoring.max_position_weight` | `5` | Cap on the position multiplier | +| `scoring.half_life_days` | `30` | Days for an event's weight to halve | +| `scoring.window_days` | `180` | Events older than this are ignored | +| `scoring.min_sessions` | `3` | Distinct shoppers needed before a pair is scored | +| `scoring.max_products_per_query` | `50` | Products kept per query | +| `scoring.schedule` | `'02:00'` | Daily run time | + +The aggregation is a single SQL query on MySQL 8 and Postgres (`Lunar\SearchRelevance\Scoring\MySqlScoreAggregator`, `PostgresScoreAggregator`), chosen from the Lunar database connection's driver. Other drivers, including SQLite, use `PhpScoreAggregator`, which does the same in chunked PHP. The `Lunar\SearchRelevance\Contracts\ScoreAggregator` binding can be swapped like any other; see [Extending search](/2.x/extending/search#search-relevance). + +## Versioning + +Every logged search and every score carries a retrieval version, `Lunar\SearchRelevance\RetrievalVersion::current()`, built from three parts: the `normaliser_version` config value, the Scout driver, and whether hybrid search is on. `QueryAffinitySignal` only reads scores whose version matches the current one. + +This matters because scores are keyed by the **normalised** query. If the normaliser changes (a new synonym, a change to how plurals are folded), `running shoe` and `running shoes` may no longer normalise to the same string, and scores learned under the old rules would apply to the wrong queries. Stale scores measurably make results worse, so a version change relearns from scratch: the scoring job writes rows for the new version, ignores events logged under the old one, and deletes rows for old versions. + +Bump `normaliser_version` whenever the normalisation rules change, including when swapping in a custom `QueryNormaliser`. Changing the Scout driver, or turning hybrid search on or off, bumps the version automatically. + +## Scheduling and queues + +The service provider registers two scheduled commands: + +| Command | Schedule | +|---|---| +| `lunar:search-relevance:score` | Daily at `scoring.schedule` (`02:00` by default), without overlapping | +| `lunar:search-relevance:prune` | Weekly; deletes queries and events older than `retention_days` (400 by default) | + +Both need the Laravel scheduler running (`php artisan schedule:work` locally, or the `schedule:run` cron entry in production). + +Logging a search, recording an event, and attributing a cart or order line are queued jobs on the default queue. With the `sync` driver they run inline: correct, but every search then performs an insert on the request path, and every tracked click does a validation query. For anything beyond a development environment, run a queue worker. + +The ranked window is cached on the default cache store as plain arrays, so any cache driver works. + +## Replay + +Replay is the proof that ranking helps, computed from real traffic before any shopper sees a reordered result: + +```bash +php artisan lunar:search-relevance:replay --days=30 +``` + +For every logged search in the period that led to a purchase, it compares the purchased product's position in the shown order against its position in the ranked order the add-on computed in shadow mode: + +``` +Replaying 1,284 purchased searches from the last 30 days + + Mean reciprocal rank (engine order) 0.339 + Mean reciprocal rank (learned order) 0.426 + Searches where learned order ranked the purchase higher 38.2% + Searches where learned order ranked the purchase lower 6.1% +``` + +Mean reciprocal rank averages `1 / position` of the purchased product, so it rewards putting the right product first. The same figures drive the "Uplift" card on the panel's overview page. + +## Configuration reference + +The full config, merged under `lunar.search_relevance`: + +| Key | Default | Description | +|---|---|---| +| `mode` | `env('LUNAR_SEARCH_RELEVANCE_MODE', 'shadow')` | `off`, `shadow`, or `on`; see [Modes](#modes). Overridden by the panel setting when one is saved. | +| `models` | `[Lunar\Core\Models\Product::class]` | Searchable models whose results are logged and ranked | +| `window` | `250` | Candidate window fetched from the engine and reordered. Requests beyond the window (`page * perPage > window`) pass through unranked but are still logged. | +| `bucket_size` | `10` | Hits are reordered only within buckets of this size | +| `cache_ttl` | `300` | Seconds the ranked window is cached | +| `learned_union.max` | `5` | Most learned products inserted that the engine did not return | +| `learned_union.min_relative` | `0.1` | Minimum relative score for a learned product to be inserted | +| `impressions_logged` | `50` | How many shown product ids a logged search records; clicks beyond this cannot be validated | +| `attribution_ttl_minutes` | `30` | How long after a click a cart addition or purchase is attributed to it | +| `session_key` | `'cart'` | `cart` (the Lunar cart session id) or `session` (the Laravel session id) | +| `normaliser` | `Lunar\SearchRelevance\Normalisers\DefaultQueryNormaliser::class` | The `QueryNormaliser` implementation | +| `normaliser_version` | `1` | Bump when normalisation rules change; see [Versioning](#versioning) | +| `ranker` | `Lunar\SearchRelevance\Rankers\BucketedRanker::class` | The `Ranker` implementation | +| `signals` | `[QueryAffinitySignal::class => 1.0]` | Signal class to weight; combined scores are clamped to 0..1 | +| `scoring.*` | | See [Scoring](#scoring) | +| `retention_days` | `400` | Raw queries and events older than this are pruned weekly | +| `guards.events_rate_limit` | `'60,1'` | Events endpoint rate limit as `attempts,minutes`, per shopper and per IP | +| `guards.max_searches_per_minute` | `30` | Sessions searching faster than this are ignored by scoring | + +The three pipeline stages (`Lunar\SearchRelevance\Pipelines\PartNumberRetrieval`, `WidenRequest`, and `RankResults`) are appended to `lunar.search.pipelines.request` and `lunar.search.pipelines.results` by the service provider unless they are already listed. A host that sets those keys explicitly in `config/lunar/search.php` controls their order, and can leave one out; `PartNumberRetrieval` and `WidenRequest` must run in that order, and `WidenRequest` must run before any stage that changes the page. diff --git a/2.x/addons/search.mdx b/2.x/addons/search.mdx index 3e54e72..efed641 100644 --- a/2.x/addons/search.mdx +++ b/2.x/addons/search.mdx @@ -35,6 +35,21 @@ The add-on ships its own `facets` configuration, merged under the same `lunar.se use Lunar\Core\Models\Product; return [ + 'pipelines' => [ + 'request' => [], + 'results' => [], + ], + + 'typesense' => [ + 'vector_distance_threshold' => 0.6, + ], + + 'meilisearch' => [ + 'embedder' => null, + 'semantic_ratio' => 0.5, + 'ranking_score_threshold' => null, + ], + 'facets' => [ Product::class => [ 'brand' => [], @@ -45,6 +60,15 @@ return [ This config is merged, not published — there is no `vendor:publish` tag for it. To override or extend it, create `config/lunar/search.php` in the host application; any keys not present there fall back to the add-on's defaults. The `engine_map` and `models` keys read by the core search config (see the [Search reference](/2.x/reference/search)) live in the same file. +| Key | Default | Description | +|---|---|---| +| `pipelines.request` | `[]` | Stages run before the engine queries; see [Pipelines](#pipelines) | +| `pipelines.results` | `[]` | Stages run after the results are built; see [Pipelines](#pipelines) | +| `typesense.vector_distance_threshold` | `0.6` | Maximum vector distance for hybrid matches; `0` sends the bare `k: 200` vector query. See [Typesense](#typesense). | +| `meilisearch.embedder` | `null` | Name of a configured Meilisearch embedder to enable hybrid search; `null` keeps keyword-only retrieval | +| `meilisearch.semantic_ratio` | `0.5` | Semantic ratio for hybrid search, when an embedder is set | +| `meilisearch.ranking_score_threshold` | `null` | Drops hits below this ranking score (0..1) when hybrid search is on; the counterpart of the Typesense distance threshold | + Each key under a model's `facets` entry corresponds to a field in that model's searchable index. The value is an array of per-facet-value configuration, used to attach extra data (such as a hex color) to a specific value: ```php @@ -206,6 +230,125 @@ $results = Search::query('Hoodies') ->get(); ``` +## Pipelines + +Every engine runs two config-driven pipelines around its query, mirroring the cart and order pipelines in core. Stages are class names listed in `lunar.search.pipelines`, resolved from the container and run top to bottom through `Illuminate\Pipeline\Pipeline`: + +```php +// config/lunar/search.php +return [ + 'pipelines' => [ + // Run before the engine queries. Stages may change page, perPage, + // sort, filters or engine parameters. + 'request' => [ + App\Search\Pipelines\BoostInStock::class, + ], + // Run after SearchResults is built. Stages may reorder, annotate or + // replace hits. + 'results' => [], + ], +]; +``` + +The request pipeline receives a `Lunar\Search\Pipelines\SearchRequest`: + +| Property | Type | Description | +|---|---|---| +| `engine` | `AbstractEngine` | The engine about to query; call its fluent methods to change the request | +| `requestedPage` | `int` | The page the caller asked for, before any stage changed it | +| `requestedPerPage` | `int` | The page size the caller asked for | +| `context` | `array` | A free-form bag for stages to hand state to the results pipeline | + +The results pipeline receives a `Lunar\Search\Pipelines\SearchResponse`: + +| Property | Type | Description | +|---|---|---| +| `request` | `SearchRequest` | The request passable, including anything stages put in `context` | +| `results` | `SearchResults` | The built results; a stage may mutate it or assign a replacement | + +### Page and per-page + +`AbstractEngine::page(int $page)` sets the page the engine fetches, alongside the existing `perPage()`. Both are readable with `getPage()` and `getPerPage()`. A request stage can fetch a wider window than the caller asked for (set `page(1)` and a larger `perPage()`), then a results stage slices back to `requestedPage` and `requestedPerPage` from the request passable. + +```php +use Lunar\Search\Facades\Search; + +$results = Search::query('Hoodies') + ->perPage(24) + ->page(3) + ->get(); +``` + +### Engine parameters + +`withParams(array $params)` merges extra engine-specific request parameters on top of everything the engine builds itself, so a request stage can change retrieval without the engine knowing about it. Keys are the engine's own parameter names: Typesense search parameters (`query_by`, `num_typos`, `infix`, ...) for `TypesenseEngine`, Meilisearch request parameters (`attributesToSearchOn`, `matchingStrategy`, `hybrid`, ...) for `MeilisearchEngine`. A `null` value removes a parameter the engine would otherwise send: + +```php +$request->engine->withParams([ + 'query_by' => 'skus,skus_normalised', + 'num_typos' => 0, + 'vector_query' => null, // drop the hybrid vector query for this request +]); +``` + +`getParams()` returns the merged overrides. `DatabaseEngine` has no request parameters and ignores them. + +### Meta fields + +Both data objects carry a free-form `meta` array for engines and stages to annotate: + +- `SearchHit::$meta` holds the engine's own score under `score` where the engine provides one: Typesense `text_match`, Meilisearch `_rankingScore` (the engine requests it with `showRankingScore`). The database engine leaves it unset. Stages add their own keys. +- `SearchResults::$meta` is empty by default; results stages annotate it, for example with a search id. + +Both default to `[]`, so existing calls to `SearchResults::from()` and `SearchHit::from()` are unaffected. + +### A custom stage + +A stage is any class with a `handle($passable, \Closure $next)` method. This request stage restricts a search to in-stock products when the storefront asks for it through the context bag: + +```php +namespace App\Search\Pipelines; + +use Closure; +use Lunar\Search\Pipelines\SearchRequest; + +class BoostInStock +{ + public function handle(SearchRequest $request, Closure $next): SearchRequest + { + $request->engine->addFilter('in_stock', true); + $request->context['in_stock_only'] = true; + + return $next($request); + } +} +``` + +And this results stage annotates every hit with its display position: + +```php +namespace App\Search\Pipelines; + +use Closure; +use Lunar\Search\Pipelines\SearchResponse; + +class NumberHits +{ + public function handle(SearchResponse $response, Closure $next): SearchResponse + { + foreach ($response->results->hits as $index => $hit) { + $hit->meta['position'] = $index + 1; + } + + return $next($response); + } +} +``` + + +The three built-in engines call `pipeRequest()` at the top of `get()` and wrap their return value in `pipeResults()`. A custom engine extending `Lunar\Search\Engines\AbstractEngine` opts in by doing the same; an engine that does not call them keeps working, but stages never run for it. The [Search Relevance](/2.x/addons/search-relevance) add-on is built entirely on these two pipelines. + + ## Response Format All search engines return a `Lunar\Search\Data\SearchResults` object with a consistent structure: @@ -222,6 +365,7 @@ All search engines return a `Lunar\Search\Data\SearchResults` object with a cons | `links` | `View` | Pagination links (Laravel paginator view) | | `sortField` | `?string` | The field the results are sorted by, if any | | `sortDirection` | `?string` | `asc`, `desc`, or `null` | +| `meta` | `array` | Annotations from results-pipeline stages; empty by default | ### SearchHit @@ -231,6 +375,7 @@ Each hit contains the indexed document data and any highlights (Typesense only): |---|---|---| | `highlights` | `SearchHitHighlight[]` | Matched field highlights | | `document` | `array` | The raw indexed document data | +| `meta` | `array` | The engine score under `score` where available (Typesense and Meilisearch), plus any keys added by pipeline stages | ### SearchHitHighlight @@ -340,6 +485,10 @@ php artisan lunar:meilisearch:setup It reads `config('lunar.search.models')`, creates any missing Meilisearch indexes, and applies each model's filterable and sortable attributes (as defined by its indexer — see the [Search reference](/2.x/reference/search)) to the corresponding index. +It also disables typo tolerance on the fields an indexer returns from `getExactMatchFields()`, by setting `typoTolerance.disableOnAttributes` on the index. `Lunar\Core\Search\ProductIndexer` returns `['skus', 'skus_normalised']`, so product codes only match exactly or by prefix rather than matching random tokens one or two typos away. Re-run the command after changing an indexer's exact-match fields. + +To enable hybrid (semantic) search on Meilisearch, configure an embedder on the index and set its name in `lunar.search.meilisearch.embedder`; `semantic_ratio` and `ranking_score_threshold` then apply. Lunar does not configure the embedder itself. + The Meilisearch and Typesense engines used to query results live in the `lunarphp/search` package installed above. The `lunarphp/meilisearch` add-on only configures Meilisearch's index settings; it is not required to run Meilisearch queries through the `Search` facade. @@ -347,3 +496,7 @@ The Meilisearch and Typesense engines used to query results live in the `lunarph ## Typesense Typesense is available as a driver (`Lunar\Search\Engines\TypesenseEngine`) once `lunarphp/search` is installed, with no separate add-on package required. Configure Typesense's own collection schema and search parameters through Scout's `config/scout.php` (the `typesense.model-settings` key), which controls field types for filtering, highlight and hybrid search settings, and query defaults. + +When the collection schema declares an auto-embedding `embedding` field, the engine adds a hybrid vector query to every search with a term. The query carries `distance_threshold` from `lunar.search.typesense.vector_distance_threshold` (default `0.6`) so that a term with no semantic neighbours cannot pad the result set with its 200 nearest vectors; set it to `0` to send the bare `k: 200` query. A `vector_query` search parameter in `config/scout.php` overrides the generated one entirely. + +For part-number search, declare `skus` and `skus_normalised` as `string[]` fields with `infix: true` and list both in `query_by`; see [Search Relevance: Typesense setup](/2.x/addons/search-relevance#typesense-setup) for the full entries. diff --git a/2.x/admin/search-relevance.mdx b/2.x/admin/search-relevance.mdx new file mode 100644 index 0000000..398fdf8 --- /dev/null +++ b/2.x/admin/search-relevance.mdx @@ -0,0 +1,96 @@ +--- +title: "Search Relevance" +description: "The admin panel section shipped by the Search Relevance add-on: search KPIs, per-query learned rankings, and the mode switch." +--- + +The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-relevance`) registers a section in the admin panel for reading what shoppers search for, what the ranking has learned, and switching the add-on from shadow mode to live. It is an add-on section, built with the same extension API described under [Extending the panel](/2.x/admin/extending/overview), so it appears only when both `lunarphp/panel` and the add-on are installed. + +## Setup + +Install the add-on and run its migrations, then publish (or, in local development, link) its compiled assets so the section's pages load: + +```bash +php artisan vendor:publish --tag=panel-all-assets --force +# or, locally +php artisan lunar:panel:link +``` + +See the [add-on installation](/2.x/addons/search-relevance#installation) for the full steps. + +## Permission + +Every route and navigation item in the section is gated by one permission handle, seeded by the add-on's migration: + +| Handle | Grants access to | +| --- | --- | +| `search:manage-relevance` | The Search relevance pages, the settings page, the dashboard widget, the product slot, and the global search source. | + +Assign it to a role or a staff member like any other handle; see [Access control](/2.x/admin/access-control). Staff with `admin` set to `true` pass the check automatically. + +## Navigation + +The section adds a **Search relevance** item to the sidebar and a **Search relevance** entry to the Settings sidebar. Both carry the permission handle above, so staff without it see neither. + +## Overview page + +Route `panel.search-relevance.index`. The page opens with a date range selector and a row of KPI cards for that range: + +| KPI | Meaning | +| --- | --- | +| Searches | Logged searches in the range | +| Click-through rate | Share of searches with at least one click | +| Search conversion rate | Share of searches that led to a purchase of a clicked product | +| Zero-result rate | Share of searches the engine returned nothing for | +| Mean click position | Average position of clicked results; lower is better | + +Beside the KPIs sits the **Uplift** card, the same figures the [replay command](/2.x/addons/search-relevance#replay) prints: mean reciprocal rank of the purchased product under the engine order and under the learned order, and the share of purchased searches where the learned order placed the product higher. In `shadow` mode this is the evidence for switching on; in `on` mode it keeps reporting on the searches the ranking changed. + +Below are three tables: + +- **Top queries**: the most frequent normalised queries with search counts, click-through, and conversion. Each row links to the [query page](#query-page). +- **Zero-result queries**: queries the engine returned nothing for, ordered by frequency. These are catalog gaps or synonym candidates for a custom normaliser. +- **Queries with no clicks**: queries that return results nobody chooses. Merchandising opportunities: the results are probably wrong for what shoppers meant. + +## Query page + +Route `panel.search-relevance.query`, reached from any table row or from the global search palette. It shows one normalised query and what the ranking has learned for it. + +The main table lists the learned products in score order, with the **explainability breakdown** per product: + +| Column | How to read it | +| --- | --- | +| Relative score | 0 to 1, drawn as a bar. The best product for this query is 1; others are relative to it. This is the value the ranker uses. | +| Clicks, Baskets, Purchases | Raw event counts in the scoring window, before position and recency weighting | +| Sessions | Distinct shoppers behind those events. A pair needs at least `scoring.min_sessions` before it is scored, so a product missing from this table may simply not have enough shoppers yet. | +| Last event | When the product was last chosen for this query; the score decays with `scoring.half_life_days` | +| Engine position | Where the engine typically returns this product for the query. A high relative score at a low engine position is exactly what the ranking corrects; a product at engine position 1 with a low score is one the engine over-ranks. | + +A second panel lists the raw query variants that normalise to this query (`running shoe`, `Running Shoes`, `running-shoes`), with counts. If two variants that should be the same query are listed as separate queries, that is a normaliser change; see [custom query normaliser](/2.x/extending/search#custom-query-normaliser). + +## Settings page + +Route `panel.settings.search-relevance.index`, under **Settings > Search relevance**. It has one control, the mode switch (`off`, `shadow`, `on`), and a read-only summary of the scoring weights, the current retrieval version, and when scoring last ran. + +Switching to `on` asks for confirmation, because it changes what every shopper sees. The chosen mode is persisted in the add-on's `search_relevance_settings` table and takes effect immediately; the config value (`lunar.search_relevance.mode`) is only the fallback when nothing has been saved here. See [Modes](/2.x/addons/search-relevance#modes). + +### When to switch from shadow to on + +Leave the add-on in `shadow` until all of the following hold: + +- Storefront tracking is in place and the overview shows a click-through rate, not zero. +- The scoring job has run (the settings page shows a last run) and the top queries have learned products with more than a handful of sessions. +- The Uplift card shows the learned order placing the purchased product higher in clearly more searches than it places it lower, over a range of at least a few weeks. + +If uplift is flat, the likely causes are too little traffic for `scoring.min_sessions`, tracking missing from part of the storefront, or a normaliser splitting the same query across variants. Switching to `on` without evidence is safe (bucketed reordering cannot move a weak match above a strong one) but pointless. + +## Dashboard widget + +`Lunar\SearchRelevance\Panel\Widgets\SearchConversionWidget` adds a half-width **Search conversion** widget to the dashboard, showing searches and the search conversion rate for the dashboard's date range. Staff can reorder, hide, and re-add it like any first-party widget, and it is hidden from staff without the permission. + +## Product slot + +On the product edit page, the section injects a **Search performance** card after the content section (the `products.edit:content:after` zone). It lists the queries the product wins (has a learned score for), with its clicks and purchases per query, so a merchandiser editing a product can see which searches it is expected to answer. + +## Global search + +Normalised queries are a source in the global search palette (Cmd+K). Typing a shopper query jumps straight to its query page. Results are grouped under the section's own label and gated by the permission. diff --git a/2.x/extending/search.mdx b/2.x/extending/search.mdx index 3e936d2..299b731 100644 --- a/2.x/extending/search.mdx +++ b/2.x/extending/search.mdx @@ -89,8 +89,9 @@ Lunar's own models are mapped to dedicated indexers that extend `ScoutIndexer` t - Any searchable custom attributes - `thumbnail` (the small variant's URL, if a thumbnail is set) - `skus`, an array of the product's variant SKUs +- `skus_normalised`, the same SKUs uppercased with every non-alphanumeric character stripped, so a code typed without its printed hyphens (`HAGMB`) still prefix-matches `HAG-MB-32A` -Its sortable fields are `created_at`, `updated_at`, `skus`, and `status`; its filterable fields are `__soft_deleted`, `skus`, and `status`. +Its sortable fields are `created_at`, `updated_at`, `skus`, and `status`; its filterable fields are `__soft_deleted`, `skus`, and `status`. It also returns `['skus', 'skus_normalised']` from `getExactMatchFields()`: fields that must match exactly or by prefix, never through typo tolerance. `ScoutIndexer::getExactMatchFields()` returns `[]` by default; the `lunar:meilisearch:setup` command disables typo tolerance on whatever a custom indexer returns from it (see the [Search add-on](/2.x/addons/search#meilisearch)). Lunar ships similar dedicated indexers for `BrandIndexer`, `CollectionIndexer`, `CustomerIndexer`, `OrderIndexer`, and `ProductOptionIndexer`. See the [Search reference](/2.x/reference/search) for what each one indexes. @@ -236,6 +237,178 @@ Once registered, map the desired model to the new driver name in `engine_map`: ], ``` +## Search relevance + +The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-relevance`) reorders search results from what shoppers click, add to cart, and buy. It exposes four contracts in `Lunar\SearchRelevance\Contracts`, each bound in its service provider from config, so a host application or a client package can replace any of them by rebinding the interface. + +| Contract | Default | Role | +|---|---|---| +| `QueryNormaliser` | `Lunar\SearchRelevance\Normalisers\DefaultQueryNormaliser` | Folds raw queries into the key scores are stored under, and classifies part numbers | +| `Signal` | `Lunar\SearchRelevance\Signals\QueryAffinitySignal` | Scores product ids 0..1 for a query; several can be combined by weight | +| `Ranker` | `Lunar\SearchRelevance\Rankers\BucketedRanker` | Reorders the candidate window from the combined signal scores | +| `ScoreAggregator` | Chosen per database driver | Turns logged events into `search_query_scores` rows | + +### Custom query normaliser + +The normaliser decides which raw queries count as the same query. The default lowercases, strips punctuation, collapses whitespace, and folds simple plurals, and leaves part numbers unstemmed. A store whose customers use their own vocabulary can fold synonyms into a canonical form so that both spellings learn from each other: + +```php +namespace App\Search; + +use Lunar\SearchRelevance\Contracts\QueryNormaliser; +use Lunar\SearchRelevance\Normalisers\DefaultQueryNormaliser; + +class StoreQueryNormaliser implements QueryNormaliser +{ + protected array $synonyms = [ + 'hoody' => 'hoodie', + 'trainers' => 'sneakers', + 'jumper' => 'sweater', + ]; + + public function __construct(protected DefaultQueryNormaliser $default) {} + + public function normalise(string $query): string + { + $normalised = $this->default->normalise($query); + + return implode(' ', array_map( + fn (string $token) => $this->synonyms[$token] ?? $token, + explode(' ', $normalised), + )); + } + + public function isPartNumber(string $query): bool + { + return $this->default->isPartNumber($query); + } +} +``` + +Point the config at it and bump the version: + +```php +// config/lunar/search-relevance.php +return [ + 'normaliser' => App\Search\StoreQueryNormaliser::class, + 'normaliser_version' => 2, +]; +``` + + +The same normaliser must be used when a search is logged and when it is ranked. Scores are stored under the normalised query, so a search normalised one way at log time and another way at search time never finds its scores. Every time the normalisation rules change, including the first time a custom normaliser is installed, bump `normaliser_version`. The version is part of `Lunar\SearchRelevance\RetrievalVersion::current()`, so the scoring job relearns under the new rules and stops applying scores learned under the old ones. + + +### Custom signal + +A signal scores each product id in the candidate window between 0 and 1 for the current query. Signals are listed in `lunar.search_relevance.signals` with a weight, combined by `Lunar\SearchRelevance\Signals\SignalCombiner`, and clamped to 0..1. This example nudges in-stock products up, so learned popularity does not promote something the shopper cannot buy: + +```php +namespace App\Search; + +use Lunar\Core\Models\ProductVariant; +use Lunar\SearchRelevance\Contracts\Signal; +use Lunar\SearchRelevance\Data\RankingContext; + +class StockLevelSignal implements Signal +{ + /** @return array */ + public function scores(RankingContext $context, array $productIds): array + { + $stock = ProductVariant::query() + ->whereIn('product_id', $productIds) + ->groupBy('product_id') + ->selectRaw('product_id, sum(stock) as stock') + ->pluck('stock', 'product_id'); + + return collect($productIds) + ->mapWithKeys(fn (int $id) => [$id => ($stock[$id] ?? 0) > 0 ? 1.0 : 0.0]) + ->all(); + } +} +``` + +Register it alongside the default signal. Weights are relative; here learned affinity still dominates: + +```php +// config/lunar/search-relevance.php +return [ + 'signals' => [ + Lunar\SearchRelevance\Signals\QueryAffinitySignal::class => 1.0, + App\Search\StockLevelSignal::class => 0.25, + ], +]; +``` + +`RankingContext` carries `modelType`, `normalisedQuery`, `sessionId`, `customerId`, `mode`, `sort`, `filtersHash`, and `version`, so a signal can be customer-aware or query-aware without extra lookups. + +### Swapping the ranker + +The default `BucketedRanker` reorders hits by combined score within buckets of `bucket_size` engine positions, so a weak keyword match never overtakes a strong one. A ranker receives the `RankingContext` and a `Lunar\SearchRelevance\Data\HitCollection` (an `Illuminate\Support\Collection` of `Lunar\SearchRelevance\Data\Hit` objects with `productId`, `originalPosition`, `score`, `document`, `source`, and a mutable `boost`) and returns the collection in display order: + +```php +namespace App\Search; + +use Lunar\SearchRelevance\Contracts\Ranker; +use Lunar\SearchRelevance\Data\HitCollection; +use Lunar\SearchRelevance\Data\RankingContext; +use Lunar\SearchRelevance\Signals\SignalCombiner; + +class TopThreeOnlyRanker implements Ranker +{ + public function __construct(protected SignalCombiner $signals) {} + + public function rank(RankingContext $context, HitCollection $hits): HitCollection + { + $scores = $this->signals->scores($context, $hits->productIds()); + + // Promote at most three learned products to the top; keep engine order for the rest. + $promoted = $hits + ->filter(fn ($hit) => ($scores[$hit->productId] ?? 0) >= 0.5) + ->sortByDesc(fn ($hit) => $scores[$hit->productId]) + ->take(3); + + return new HitCollection([ + ...$promoted->values(), + ...$hits->reject(fn ($hit) => $promoted->contains($hit))->values(), + ]); + } +} +``` + +```php +// config/lunar/search-relevance.php +return [ + 'ranker' => App\Search\TopThreeOnlyRanker::class, +]; +``` + +### Binding in a service provider + +The config keys above are the simplest route. The add-on resolves each contract from the container, so a service provider can also rebind an interface directly, which is how a client package ships its own implementation without touching the host config: + +```php +namespace App\Providers; + +use App\Search\StockLevelSignal; +use App\Search\StoreQueryNormaliser; +use App\Search\TopThreeOnlyRanker; +use Illuminate\Support\ServiceProvider; +use Lunar\SearchRelevance\Contracts\QueryNormaliser; +use Lunar\SearchRelevance\Contracts\Ranker; + +class SearchServiceProvider extends ServiceProvider +{ + public function register(): void + { + $this->app->bind(QueryNormaliser::class, StoreQueryNormaliser::class); + $this->app->bind(Ranker::class, TopThreeOnlyRanker::class); + } +} +``` + +Signals are always read from the `signals` config array, because each carries a weight. + The storefront search package (namespace `Lunar\Search\...`) is a separate package from the indexing pieces described on this page. It provides the storefront-facing query layer: faceted filtering, sorting, and instant search, built on top of the Scout search results that Lunar's core indexers produce. Indexers control *what* gets indexed; `Lunar\Search\...` controls how the storefront *queries* it. See the [Search reference](/2.x/reference/search) for details. diff --git a/2.x/guides/search.mdx b/2.x/guides/search.mdx index ad42758..29a2672 100644 --- a/2.x/guides/search.mdx +++ b/2.x/guides/search.mdx @@ -428,6 +428,26 @@ After changing which attributes are searchable, reimport the search index with ` See the [Search reference](/2.x/reference/search#product-indexer) for the complete list of fields the default product indexer sends to the search engine. +## Make Search Learn + +Out of the box, results appear in the order the engine chooses, and nothing improves with traffic. The [Search Relevance](/2.x/addons/search-relevance) add-on logs each search, follows clicks through to cart additions and purchases, and reorders results for each query from what shoppers actually chose. Install it, leave it in its default `shadow` mode while it collects data, and add two lines to the results page so clicks are recorded: + +```blade +{{-- once per results page --}} + + +{{-- on each result --}} +
+ ... +
+``` + +`lunar_search_attrs()` renders the `data-lunar-*` attributes the tracking script reads, and outputs nothing when the search was not logged, so the markup is safe before the add-on is switched on. Headless storefronts post the same data to the add-on's events endpoint; see [Storefront tracking](/2.x/addons/search-relevance#storefront-tracking). + + +With hybrid (semantic) search a distance threshold stops nonsense queries from returning every product, which means some queries return nothing. Render a clear zero-results state (a message, popular collections, a link back to the catalog) rather than loosening the threshold so the engine pads the page. Padded results are unrelated products, and every click on one teaches the ranking the wrong thing. + + ## Routes ```php diff --git a/docs.json b/docs.json index a15ae1b..b31d4c2 100644 --- a/docs.json +++ b/docs.json @@ -380,6 +380,10 @@ "2.x/admin/access-control" ] }, + { + "group": "Add-on sections", + "pages": ["2.x/admin/search-relevance"] + }, { "group": "Extending", "pages": [ @@ -414,7 +418,7 @@ }, { "group": "General", - "pages": ["2.x/addons/table-rate-shipping", "2.x/addons/search"] + "pages": ["2.x/addons/table-rate-shipping", "2.x/addons/search", "2.x/addons/search-relevance"] }, { "group": "Payments", diff --git a/logs/flight-plan.mdx b/logs/flight-plan.mdx index 82e5b34..8405f01 100644 --- a/logs/flight-plan.mdx +++ b/logs/flight-plan.mdx @@ -84,6 +84,16 @@ The open source checkout will also be the foundation for a future hosted product Alpha July/Aug 2026 +## Search that learns + +Search results in most stores are ordered by whatever the engine thinks matches best, and never get better with traffic. Merchants who want ranking that learns from real behaviour pay a hosted search service for it, at prices that scale with their success. + +v2 adds a search relevance add-on that gives a large part of that value at a fixed cost. It logs searches and the clicks, cart additions and purchases that follow, scores them nightly, and reorders results per query, on top of whichever engine the store already runs. It's engine-agnostic by construction: Typesense, Meilisearch and the database driver all work, and a custom engine opts in through two small pipeline hooks in the search package. Alongside it, part-number search gets fixed on both Typesense and Meilisearch, so a partial product code returns its family and nothing else. + +It ships in shadow mode, learning without changing what shoppers see, with a replay command and a panel section that show the uplift before anything is switched on. + +Alpha with the panel + ## Lunar Eclipse Lunar Eclipse extends the platform with the features serious operators ask for: B2B, trade and ERP capabilities, built on top of the open core. It's the main commercial route for Lunar, and it's how we fund the long-term development of everything else on this page. From 8883cc5c92f3cd464ca9657dc5f5e6e31dc701e6 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 16:02:09 +0100 Subject: [PATCH 02/13] docs(2.x): reconcile search relevance pages with the package source Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 16 +++++----- 2.x/admin/search-relevance.mdx | 53 ++++++++++++++++++++------------- 2.x/extending/search.mdx | 6 ++-- 3 files changed, 43 insertions(+), 32 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index a8e47f1..2a77986 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -85,13 +85,13 @@ For a part number, the `Lunar\SearchRelevance\Pipelines\PartNumberRetrieval` req | | Typesense | Meilisearch | |---|---|---| -| Restrict to SKU fields | `query_by: skus,skus_normalised` | `attributesToSearchOn: ['skus', 'skus_normalised']` | -| Partial match | `prefix: true`, `infix: always` | Prefix on the last word is always on; no infix | -| No typos | `num_typos: 0` | Index setting `typoTolerance.disableOnAttributes`, applied by `lunar:meilisearch:setup` | +| Restrict to SKU fields | `query_by: skus,skus_normalised` (and `query_by_weights` dropped) | `attributesToSearchOn: ['skus', 'skus_normalised']` | +| Partial match | `prefix: true`, `infix: always,always` | Prefix on the last word is always on; no infix | +| No typos | `num_typos: 0,0` | Index setting `typoTolerance.disableOnAttributes`, applied by `lunar:meilisearch:setup` | | Keep every token | `drop_tokens_threshold: 0` | `matchingStrategy: all` | | No semantic padding | `vector_query` omitted | `hybrid.semanticRatio: 0`, when an embedder is configured | -Part-number searches are still logged, but they are not ranked: per-code queries are too sparse to learn from, and the engine order is already the right one. The normaliser also skips stemming for them. +Part-number searches are still logged and tracked, but they are not ranked: per-code queries are too sparse to learn from, and the engine order is already the right one. The normaliser also skips stemming for them. The database engine has no per-field request parameters, so the stage leaves it untouched. The stage applies these parameters through `AbstractEngine::withParams()`, so nothing engine-specific lives in the engine itself. @@ -261,9 +261,9 @@ $line->meta['search_attribution']; // ] ``` -Shopper identity is the Lunar cart session identifier when `session_key` is `cart` (the default). It survives login, and it is what cart and order lines already relate to. Set `session_key` to `session` to use the Laravel session id instead. +Shopper identity is the Lunar cart session identifier when `session_key` is `cart` (the default), stored as `cart:{id}`. It survives login, and it is what cart and order lines already relate to. Set `session_key` to `session` to use the Laravel session id (`session:{id}`) instead; that is also the fallback when no cart exists yet. -All three writes are queued jobs; nothing runs on the request path beyond the session write. +Every event is written by the queued `Lunar\SearchRelevance\Events\RecordEvent` job, which re-validates the search id, the product, and the position before inserting, so nothing runs on the request path beyond the session write. ## Scoring @@ -281,7 +281,7 @@ In plain language, for each normalised query and product: - **Sparse data is ignored.** Only events within `window_days` are considered, and a query and product pair needs events from at least `min_sessions` distinct shoppers before it gets a score. Sessions searching faster than `guards.max_searches_per_minute` are excluded as bots. - **Scores are relative.** Within a query, each product's score is divided by the best score for that query, giving a `relative` value from 0 to 1. At most `max_products_per_query` products are kept per query. -At search time, `Lunar\SearchRelevance\Signals\QueryAffinitySignal` looks up the `relative` score for each product in the window, `Lunar\SearchRelevance\Signals\SignalCombiner` combines every configured signal by weight and clamps to 0..1, and `Lunar\SearchRelevance\Rankers\BucketedRanker` reorders within each bucket by that score. Learned products the engine did not return (up to `learned_union.max`, each with `relative` of at least `learned_union.min_relative`) are fetched by id and inserted at the head of the second bucket, so they appear early without displacing the strongest keyword matches. The ranked window is cached for `cache_ttl` seconds. +At search time, `Lunar\SearchRelevance\Signals\QueryAffinitySignal` looks up the `relative` score for each product in the window, `Lunar\SearchRelevance\Signals\SignalCombiner::combine()` sums every configured signal by weight and clamps to 0..1, and `Lunar\SearchRelevance\Rankers\BucketedRanker` sorts within each bucket of `bucket_size` engine positions by that combined score (ties keep the engine order). Learned products the engine did not return (up to `learned_union.max`, each with `relative` of at least `learned_union.min_relative`) are fetched by id through the same engine and inserted at the head of the second bucket, marked `source: learned`, so they can win a first-page slot without displacing the strongest keyword matches. The ranked window is cached for `cache_ttl` seconds, keyed by model, version, mode, normalised query, filters, and customer. | Key | Default | Description | |---|---|---| @@ -300,7 +300,7 @@ The aggregation is a single SQL query on MySQL 8 and Postgres (`Lunar\SearchRele ## Versioning -Every logged search and every score carries a retrieval version, `Lunar\SearchRelevance\RetrievalVersion::current()`, built from three parts: the `normaliser_version` config value, the Scout driver, and whether hybrid search is on. `QueryAffinitySignal` only reads scores whose version matches the current one. +Every logged search and every score carries a retrieval version, `Lunar\SearchRelevance\RetrievalVersion::current($modelType)`, in the form `n{normaliser_version}:{driver}:{hybrid|keyword}`, for example `n1:typesense:hybrid`. The driver comes from `lunar.search.engine_map` for the model (falling back to `scout.driver`); `hybrid` means the Typesense collection schema declares an `embedding` field, or `lunar.search.meilisearch.embedder` is set. `QueryAffinitySignal` only reads scores whose version matches the current one. This matters because scores are keyed by the **normalised** query. If the normaliser changes (a new synonym, a change to how plurals are folded), `running shoe` and `running shoes` may no longer normalise to the same string, and scores learned under the old rules would apply to the wrong queries. Stale scores measurably make results worse, so a version change relearns from scratch: the scoring job writes rows for the new version, ignores events logged under the old one, and deletes rows for old versions. diff --git a/2.x/admin/search-relevance.mdx b/2.x/admin/search-relevance.mdx index 398fdf8..ca1c414 100644 --- a/2.x/admin/search-relevance.mdx +++ b/2.x/admin/search-relevance.mdx @@ -29,47 +29,58 @@ Assign it to a role or a staff member like any other handle; see [Access control ## Navigation -The section adds a **Search relevance** item to the sidebar and a **Search relevance** entry to the Settings sidebar. Both carry the permission handle above, so staff without it see neither. +The section adds a **Search** group to the sidebar with a **Search relevance** item, and a **Search relevance** entry under the **Store** group of the Settings sidebar. Both carry the permission handle above, so staff without it see neither. + +| Route | Page | +| --- | --- | +| `panel.search-relevance.index` | [Overview](#overview-page) | +| `panel.search-relevance.query` | [Query page](#query-page); the normalised query is the route parameter, with an optional `model` query string for installs that rank more than one model | +| `panel.search-relevance.product` | JSON feed for the [product slot](#product-slot) | +| `panel.settings.search-relevance.index`, `panel.settings.search-relevance.update` | [Settings page](#settings-page) | ## Overview page -Route `panel.search-relevance.index`. The page opens with a date range selector and a row of KPI cards for that range: +The page opens with the same date range selector as the dashboard and a row of KPI cards for that range. Rates are percentages of the searches logged in the range; events are counted against the search that produced them, so a purchase after the range still counts toward a search inside it. | KPI | Meaning | | --- | --- | | Searches | Logged searches in the range | | Click-through rate | Share of searches with at least one click | -| Search conversion rate | Share of searches that led to a purchase of a clicked product | +| Conversion rate | Share of searches that led to a purchase of a clicked product | | Zero-result rate | Share of searches the engine returned nothing for | -| Mean click position | Average position of clicked results; lower is better | +| Mean click position | Average position of clicked results; lower is better. Shown as "No clicks yet" until the first click. | -Beside the KPIs sits the **Uplift** card, the same figures the [replay command](/2.x/addons/search-relevance#replay) prints: mean reciprocal rank of the purchased product under the engine order and under the learned order, and the share of purchased searches where the learned order placed the product higher. In `shadow` mode this is the evidence for switching on; in `on` mode it keeps reporting on the searches the ranking changed. +Beside the KPIs sits the **Uplift** card, the same figures the [replay command](/2.x/addons/search-relevance#replay) prints for the range: the number of purchased searches, mean reciprocal rank of the purchased product under the shown order and under the learned order, and the share of purchased searches where the learned order placed the product higher (Improved) and lower (Worsened). In `shadow` mode this is the evidence for switching on; in `on` mode it keeps reporting on the searches the ranking changed. -Below are three tables: +Below are three tables, each showing the twenty most frequent normalised queries in the range and linking every row to the [query page](#query-page): -- **Top queries**: the most frequent normalised queries with search counts, click-through, and conversion. Each row links to the [query page](#query-page). -- **Zero-result queries**: queries the engine returned nothing for, ordered by frequency. These are catalog gaps or synonym candidates for a custom normaliser. -- **Queries with no clicks**: queries that return results nobody chooses. Merchandising opportunities: the results are probably wrong for what shoppers meant. +- **Top queries**: searches, clicks, conversions (searches with a purchase), and conversion rate per query. +- **Zero-result queries**: queries the engine returned nothing for. These are catalog gaps, redirect candidates, or synonyms for a custom normaliser. +- **Queries with no clicks**: queries that returned results nobody opened. Merchandising opportunities: the results are probably wrong for what shoppers meant. ## Query page -Route `panel.search-relevance.query`, reached from any table row or from the global search palette. It shows one normalised query and what the ranking has learned for it. +Reached from any table row or from the global search palette, the query page shows one normalised query and what the ranking has learned for it. When more than one model is ranked, a model selector switches between them. -The main table lists the learned products in score order, with the **explainability breakdown** per product: +The **Learned products** table lists the products scored for this query under the current retrieval version, best first, with the **explainability breakdown** per product: | Column | How to read it | | --- | --- | -| Relative score | 0 to 1, drawn as a bar. The best product for this query is 1; others are relative to it. This is the value the ranker uses. | -| Clicks, Baskets, Purchases | Raw event counts in the scoring window, before position and recency weighting | -| Sessions | Distinct shoppers behind those events. A pair needs at least `scoring.min_sessions` before it is scored, so a product missing from this table may simply not have enough shoppers yet. | +| Product | The product name, linking to its edit page ("Product #id (deleted)" when it no longer exists) | +| Relative | 0 to 1, drawn as a bar. The strongest product for this query is 1; others are relative to it. This is the value the ranker uses. | +| Score | The absolute weighted score the relative value is derived from | +| Clicks, Baskets, Purchases | Raw event counts for this product and query, across every logged search, before position and recency weighting | +| Sessions | Distinct shoppers behind the score. A pair needs at least `scoring.min_sessions` before it is scored, so a product missing from this table may simply not have enough shoppers yet. | | Last event | When the product was last chosen for this query; the score decays with `scoring.half_life_days` | -| Engine position | Where the engine typically returns this product for the query. A high relative score at a low engine position is exactly what the ranking corrects; a product at engine position 1 with a low score is one the engine over-ranks. | +| Engine position | The average position the product sat at when shoppers engaged with it. A high relative score at a low engine position is exactly what the ranking corrects; a product the engine puts first with a low score is one the engine over-ranks. | + +The table is empty until the scoring run has produced scores for the query; the page says so rather than showing raw counts. -A second panel lists the raw query variants that normalise to this query (`running shoe`, `Running Shoes`, `running-shoes`), with counts. If two variants that should be the same query are listed as separate queries, that is a normaliser change; see [custom query normaliser](/2.x/extending/search#custom-query-normaliser). +A second panel, **Raw variants**, lists the raw queries that normalise to this one (`running shoe`, `Running Shoes`, `running-shoes`) with their search counts. If two variants that should be the same query appear as separate queries in the overview tables, that is a normaliser change; see [custom query normaliser](/2.x/extending/search#custom-query-normaliser). ## Settings page -Route `panel.settings.search-relevance.index`, under **Settings > Search relevance**. It has one control, the mode switch (`off`, `shadow`, `on`), and a read-only summary of the scoring weights, the current retrieval version, and when scoring last ran. +Under **Settings > Store > Search relevance**. It has one control, the mode switch (`off`, `shadow`, `on`, each with a one-line description), and two read-only panels: the event weights from `scoring.weights`, and the current retrieval version per ranked model, so staff can see that a normaliser or engine change has started a fresh version. Switching to `on` asks for confirmation, because it changes what every shopper sees. The chosen mode is persisted in the add-on's `search_relevance_settings` table and takes effect immediately; the config value (`lunar.search_relevance.mode`) is only the fallback when nothing has been saved here. See [Modes](/2.x/addons/search-relevance#modes). @@ -78,19 +89,19 @@ Switching to `on` asks for confirmation, because it changes what every shopper s Leave the add-on in `shadow` until all of the following hold: - Storefront tracking is in place and the overview shows a click-through rate, not zero. -- The scoring job has run (the settings page shows a last run) and the top queries have learned products with more than a handful of sessions. +- The scoring job has run and the top queries have learned products with more than a handful of sessions. - The Uplift card shows the learned order placing the purchased product higher in clearly more searches than it places it lower, over a range of at least a few weeks. If uplift is flat, the likely causes are too little traffic for `scoring.min_sessions`, tracking missing from part of the storefront, or a normaliser splitting the same query across variants. Switching to `on` without evidence is safe (bucketed reordering cannot move a weak match above a strong one) but pointless. ## Dashboard widget -`Lunar\SearchRelevance\Panel\Widgets\SearchConversionWidget` adds a half-width **Search conversion** widget to the dashboard, showing searches and the search conversion rate for the dashboard's date range. Staff can reorder, hide, and re-add it like any first-party widget, and it is hidden from staff without the permission. +`Lunar\SearchRelevance\Panel\Widgets\SearchConversionWidget` adds a half-width **Search conversion** widget to the dashboard, showing searches and the search conversion rate for the dashboard's date range, each with its change against the previous range, and a "View report" link to the overview for the same range. Staff can reorder, hide, and re-add it like any first-party widget, and it is hidden from staff without the permission. ## Product slot -On the product edit page, the section injects a **Search performance** card after the content section (the `products.edit:content:after` zone). It lists the queries the product wins (has a learned score for), with its clicks and purchases per query, so a merchandiser editing a product can see which searches it is expected to answer. +On the product edit page, the section injects a **Search performance** card after the content section (the `products.edit:content:after` zone). It loads on demand from `panel.search-relevance.product` and lists up to twenty queries the product has been clicked, added to cart, or purchased from, with the event counts per query and the product's relative score for that query where one has been learned. Each query links to its query page, so a merchandiser editing a product can see which searches it is expected to answer. ## Global search -Normalised queries are a source in the global search palette (Cmd+K). Typing a shopper query jumps straight to its query page. Results are grouped under the section's own label and gated by the permission. +Logged queries are a **Search queries** source in the global search palette (Cmd+K), listed after the first-party sources. Typing part of a shopper query matches normalised queries by substring, shows how many times each was searched, and jumps straight to its query page. The source is gated by the permission. diff --git a/2.x/extending/search.mdx b/2.x/extending/search.mdx index 299b731..501bb64 100644 --- a/2.x/extending/search.mdx +++ b/2.x/extending/search.mdx @@ -250,7 +250,7 @@ The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-re ### Custom query normaliser -The normaliser decides which raw queries count as the same query. The default lowercases, strips punctuation, collapses whitespace, and folds simple plurals, and leaves part numbers unstemmed. A store whose customers use their own vocabulary can fold synonyms into a canonical form so that both spellings learn from each other: +The normaliser decides which raw queries count as the same query. The default lowercases, replaces punctuation other than `.`, `-`, and `/` with spaces, joins a number to a following unit (`20 mm` becomes `20mm`), collapses whitespace, and folds simple English plurals (`cable ties` and `cable tie` pool their data). A part number is returned as-is, never stemmed. A store whose customers use their own vocabulary can fold synonyms into a canonical form so that both spellings learn from each other: ```php namespace App\Search; @@ -344,7 +344,7 @@ return [ ### Swapping the ranker -The default `BucketedRanker` reorders hits by combined score within buckets of `bucket_size` engine positions, so a weak keyword match never overtakes a strong one. A ranker receives the `RankingContext` and a `Lunar\SearchRelevance\Data\HitCollection` (an `Illuminate\Support\Collection` of `Lunar\SearchRelevance\Data\Hit` objects with `productId`, `originalPosition`, `score`, `document`, `source`, and a mutable `boost`) and returns the collection in display order: +The default `BucketedRanker` reorders hits by combined score within buckets of `bucket_size` engine positions, so a weak keyword match never overtakes a strong one. `Lunar\SearchRelevance\Rankers\NullRanker` returns the window untouched, useful for measuring logging overhead without ranking. A ranker receives the `RankingContext` and a `Lunar\SearchRelevance\Data\HitCollection` (an `Illuminate\Support\Collection` of `Lunar\SearchRelevance\Data\Hit` objects with `productId`, `originalPosition`, `score`, `document`, `source`, and a mutable `boost`) and returns the collection in display order: ```php namespace App\Search; @@ -360,7 +360,7 @@ class TopThreeOnlyRanker implements Ranker public function rank(RankingContext $context, HitCollection $hits): HitCollection { - $scores = $this->signals->scores($context, $hits->productIds()); + $scores = $this->signals->combine($context, $hits->productIds()); // Promote at most three learned products to the top; keep engine order for the rest. $promoted = $hits From b4dea60defb5d0f09cd45c362302bcab91ad7ad9 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 16:03:40 +0100 Subject: [PATCH 03/13] docs(2.x): reconcile search relevance tracking, replay and attribution with the source Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 39 ++++++++++++++++++++------------- 2.x/extending/search.mdx | 2 +- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 2a77986..26d2148 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -216,11 +216,13 @@ On each rendered result, spread the tracking attributes onto the element that wr @endforeach ``` -`lunar_search_attrs()` renders `data-lunar-search-id`, `data-lunar-product-id`, `data-lunar-position`, and `data-lunar-source`. It returns an empty string when the search was not logged (for example, in `off` mode), so the markup is safe to leave in place. The script listens for clicks on any link inside a tracked element and sends the event with `navigator.sendBeacon()`, so a navigation that starts immediately still delivers it. +`lunar_search_attrs()` renders `data-lunar-search-id`, `data-lunar-product-id`, `data-lunar-position`, and `data-lunar-source`. It returns an empty string when the search was not logged (for example, in `off` mode), so the markup is safe to leave in place. The component renders nothing in that case either. + +The script listens for clicks anywhere inside a tracked element and posts the four attributes plus the CSRF token to the events endpoint with `navigator.sendBeacon()` (falling back to a `keepalive` fetch), so a navigation that starts immediately still delivers the event. ### Headless and Inertia storefronts -The same data is available in the results payload. `SearchResults->meta` carries `search_id`, `ranking_mode`, and `ranking_version`, and each hit's `meta` carries `position`, `original_position`, `boost`, and `source`. Post to the events endpoint when a shopper clicks a result: +The same data is available in the results payload. `SearchResults->meta` carries `search_id`, `ranking_mode`, and `ranking_version`, and each hit's `meta` carries `position`, `original_position`, `source`, and (when the search was ranked) `boost`. Post to the events endpoint, `POST /lunar/search/events` (route name `lunar.search-relevance.events`), when a shopper clicks a result: ```js fetch('/lunar/search/events', { @@ -230,24 +232,24 @@ fetch('/lunar/search/events', { }); ``` -The endpoint accepts: +The route runs under the `web` middleware group, so a browser client sends the CSRF token and its session cookie as it would for any other form. The endpoint accepts JSON or form data: | Field | Required | Description | |---|---|---| -| `search_id` | Yes | `results.meta.search_id` | +| `search_id` | Yes | `results.meta.search_id`, a 26-character ULID | | `product_id` | Yes | The product id, as indexed under `document.id` | | `position` | Yes | The 1-based position the product was displayed at (`hit.meta.position`) | | `source` | No | `organic`, `learned`, or `explore` (`hit.meta.source`); defaults to `organic` | | `session_id` | No | An explicit shopper identifier for API clients that carry no cart session cookie. Omit it from a browser and the server resolves the shopper from the cart session. | -The endpoint validates that `search_id` exists, that `product_id` was in that search's shown list, and that `position` is in range, and it is rate limited per shopper and per IP (`guards.events_rate_limit`). Anything that fails validation is answered with an empty `204`, so a bot learns nothing about what was rejected. +The endpoint always answers with an empty `204`, whether or not the event was accepted, so a bot learns nothing about which searches or products exist. It is rate limited per shopper and per IP (`guards.events_rate_limit`, whichever trips first). The queued `RecordEvent` job then checks that `search_id` exists, that `product_id` was in that search's shown list (the first `impressions_logged` results), and that `position` is within it, and silently drops anything else. ## Attribution A click is the start of a chain the add-on follows through to a purchase: 1. The click stores `attribution.{product_id}` in the shopper's session (`search_id`, `position`, `source`, and an expiry) for `attribution_ttl_minutes` (30 by default). -2. When a `Lunar\Core\Models\CartLine` is created, `Lunar\SearchRelevance\Listeners\AttributeCartLine` resolves the line's product from its purchasable, looks up the attribution, and if one is present writes it to `meta['search_attribution']` on the line and records a `basket` event. +2. When a `Lunar\Core\Models\CartLine` is created, `Lunar\SearchRelevance\Listeners\AttributeCartLine` (an Eloquent `created` observer) resolves the line's product from its purchasable (product variants only), looks up the attribution, and if one is present writes it to `meta['search_attribution']` on the line and records a `basket` event. 3. When `Lunar\Core\Events\Orders\OrderPlaced` fires, `Lunar\SearchRelevance\Listeners\AttributeOrderLines` records a `purchase` event for every order line whose `meta` carries `search_attribution`. Order creation already copies cart line `meta` to the order line, so no pipeline change is needed. A cart or order line attributed to a search carries this in `meta`: @@ -258,6 +260,7 @@ $line->meta['search_attribution']; // 'search_id' => '01J9X4M2K7Q8R1S3T5V7W9Y0Z2', // 'position' => 3, // 'source' => 'organic', +// 'session_id' => 'cart:1842', // ] ``` @@ -271,8 +274,11 @@ Scoring turns raw events into a score per query and product, stored in `search_q ```bash php artisan lunar:search-relevance:score +# [n1:typesense:hybrid] wrote 3412 query/product rows. ``` +The command runs the aggregation once per distinct retrieval version across the ranked `models`. + In plain language, for each normalised query and product: - **Every event counts by type.** A click is worth 1, a cart addition 3, a purchase 5 (`scoring.weights`). @@ -313,7 +319,7 @@ The service provider registers two scheduled commands: | Command | Schedule | |---|---| | `lunar:search-relevance:score` | Daily at `scoring.schedule` (`02:00` by default), without overlapping | -| `lunar:search-relevance:prune` | Weekly; deletes queries and events older than `retention_days` (400 by default) | +| `lunar:search-relevance:prune` | Weekly; deletes queries and events older than `retention_days` (400 by default) and prints how many of each it removed | Both need the Laravel scheduler running (`php artisan schedule:work` locally, or the `schedule:run` cron entry in production). @@ -329,18 +335,21 @@ Replay is the proof that ranking helps, computed from real traffic before any sh php artisan lunar:search-relevance:replay --days=30 ``` -For every logged search in the period that led to a purchase, it compares the purchased product's position in the shown order against its position in the ranked order the add-on computed in shadow mode: +For every logged search in the period that led to a purchase, and whose ranked order differed from the shown order, it compares the purchased product's position in the shown order against its position in the ranked order the add-on computed: ``` -Replaying 1,284 purchased searches from the last 30 days - - Mean reciprocal rank (engine order) 0.339 - Mean reciprocal rank (learned order) 0.426 - Searches where learned order ranked the purchase higher 38.2% - Searches where learned order ranked the purchase lower 6.1% ++---------------------------+-------+ +| Metric | Value | ++---------------------------+-------+ +| Searches with a purchase | 1284 | +| MRR (shown order) | 0.339 | +| MRR (reranked order) | 0.426 | +| Improved | 38.2% | +| Worsened | 6.1% | ++---------------------------+-------+ ``` -Mean reciprocal rank averages `1 / position` of the purchased product, so it rewards putting the right product first. The same figures drive the "Uplift" card on the panel's overview page. +Mean reciprocal rank (MRR) averages `1 / position` of the purchased product, so it rewards putting the right product first. "Improved" and "Worsened" are the shares of those searches where the reranked order placed the purchased product higher or lower than the shown order. `--days` defaults to 30. The same figures drive the **Uplift** card on the panel's overview page, computed over the page's date range. ## Configuration reference diff --git a/2.x/extending/search.mdx b/2.x/extending/search.mdx index 501bb64..ad6d291 100644 --- a/2.x/extending/search.mdx +++ b/2.x/extending/search.mdx @@ -318,7 +318,7 @@ class StockLevelSignal implements Signal $stock = ProductVariant::query() ->whereIn('product_id', $productIds) ->groupBy('product_id') - ->selectRaw('product_id, sum(stock) as stock') + ->selectRaw('product_id, sum(stock_on_hand) as stock') ->pluck('stock', 'product_id'); return collect($productIds) From 561a3eafc16f260bc5ace02b3dd912c3bd076869 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 16:10:49 +0100 Subject: [PATCH 04/13] docs(2.x): list skus_normalised in the product indexer reference Co-Authored-By: Claude Fable 5.1 --- 2.x/reference/search.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/2.x/reference/search.mdx b/2.x/reference/search.mdx index ace3e8d..a96a9a4 100644 --- a/2.x/reference/search.mdx +++ b/2.x/reference/search.mdx @@ -125,6 +125,7 @@ Translatable attribute values (`Lunar\Core\FieldTypes\TranslatedText`) are explo | `name`, `description`, `short_description` | Translatable columns, exploded per locale (e.g. `name_en`) | | `thumbnail` | Thumbnail URL (`small` variant), if present | | `skus` | Array of variant SKUs | +| `skus_normalised` | The same SKUs uppercased with separators stripped, so a partial code typed without hyphens still matches | | Attribute handles | Values from searchable custom attributes | **Sortable fields:** `created_at`, `updated_at`, `skus`, `status` From bb4422a30d7ee870b0ad8052e1b730bc4de80fc8 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 16:43:28 +0100 Subject: [PATCH 05/13] docs(2.x): document the @lunarphp/search-relevance storefront client Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 53 +++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 26d2148..a69cb40 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -218,11 +218,60 @@ On each rendered result, spread the tracking attributes onto the element that wr `lunar_search_attrs()` renders `data-lunar-search-id`, `data-lunar-product-id`, `data-lunar-position`, and `data-lunar-source`. It returns an empty string when the search was not logged (for example, in `off` mode), so the markup is safe to leave in place. The component renders nothing in that case either. -The script listens for clicks anywhere inside a tracked element and posts the four attributes plus the CSRF token to the events endpoint with `navigator.sendBeacon()` (falling back to a `keepalive` fetch), so a navigation that starts immediately still delivers the event. +The script is the `@lunarphp/search-relevance` client's self-contained build, inlined into the page. It listens for clicks anywhere inside a tracked element and posts the four attributes plus the CSRF token to the events endpoint with `navigator.sendBeacon()` (falling back to a `keepalive` fetch), so a navigation that starts immediately still delivers the event. ### Headless and Inertia storefronts -The same data is available in the results payload. `SearchResults->meta` carries `search_id`, `ranking_mode`, and `ranking_version`, and each hit's `meta` carries `position`, `original_position`, `source`, and (when the search was ranked) `boost`. Post to the events endpoint, `POST /lunar/search/events` (route name `lunar.search-relevance.events`), when a shopper clicks a result: +The same data is available in the results payload. `SearchResults->meta` carries `search_id`, `ranking_mode`, and `ranking_version`, and each hit's `meta` carries `position`, `original_position`, `source`, and (when the search was ranked) `boost`. Both `meta` fields are typed, so a storefront generating types with Spatie's TypeScript transformer (see the [Search add-on](/2.x/addons/search#typescript-integration)) gets `results.meta.search_id` and `hit.meta.position` in the `Lunar.Search` namespace. + +Install the storefront client, which is the same code the Blade component inlines: + +```bash +npm install @lunarphp/search-relevance +``` + +In a Vue or Inertia storefront, the `useSearchTracking` composable takes the results (a ref, a getter, or a plain object) and returns `track(hit)` to send a click and `attrs(hit)` for the `data-lunar-*` attributes: + +```vue + + + +``` + +Both calls are safe when the search was not logged (for example, in `off` mode): `track()` does nothing and `attrs()` returns an empty object. The `v-lunar-search-hit` directive does both at once for storefronts that prefer it: `v-lunar-search-hit="{ results, hit }"`. + +The composable accepts options as its second argument: `endpoint` (defaults to `/lunar/search/events`), `token` (the CSRF token; read from `` when omitted), and `sessionId`, sent as `session_id` for clients that carry no cart session cookie. + +Any other framework uses the framework-agnostic entry. `attachSearchTracking()` adds delegated click tracking to elements that carry the `data-lunar-*` attributes (rendered with `trackingAttributes(results, hit)`), and `trackHit()` or `sendSearchEvent()` send an event directly: + +```js +import { attachSearchTracking, trackHit } from '@lunarphp/search-relevance'; + +const detach = attachSearchTracking(); + +// or, without markup: +trackHit(results, hit); +``` + +Every sender uses `navigator.sendBeacon()` with a `keepalive` fetch fallback, so a click that starts a navigation still delivers the event. + +Without the client, post to the events endpoint, `POST /lunar/search/events` (route name `lunar.search-relevance.events`), when a shopper clicks a result: ```js fetch('/lunar/search/events', { From 34711fa1e563a493d5eda055b5a2c386cd87ec8b Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 16:57:04 +0100 Subject: [PATCH 06/13] docs(2.x): abuse and manipulation guards, panel exclusions and reset Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 31 ++++++++++++++++++++++++++++++- 2.x/admin/search-relevance.mdx | 9 +++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index a69cb40..269d5d5 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -291,7 +291,7 @@ The route runs under the `web` middleware group, so a browser client sends the C | `source` | No | `organic`, `learned`, or `explore` (`hit.meta.source`); defaults to `organic` | | `session_id` | No | An explicit shopper identifier for API clients that carry no cart session cookie. Omit it from a browser and the server resolves the shopper from the cart session. | -The endpoint always answers with an empty `204`, whether or not the event was accepted, so a bot learns nothing about which searches or products exist. It is rate limited per shopper and per IP (`guards.events_rate_limit`, whichever trips first). The queued `RecordEvent` job then checks that `search_id` exists, that `product_id` was in that search's shown list (the first `impressions_logged` results), and that `position` is within it, and silently drops anything else. +The endpoint always answers with an empty `204`, whether or not the event was accepted, so a bot learns nothing about which searches or products exist. It is rate limited per shopper and per IP (`guards.events_rate_limit`, whichever trips first). The queued `RecordEvent` job then checks that `search_id` exists, that the search is no older than `guards.event_window_minutes`, that `product_id` was in that search's shown list (the first `impressions_logged` results), and that `position` is within it, and silently drops anything else. Each search records at most one event of each type per product, so replaying a click adds nothing. ## Attribution @@ -334,6 +334,8 @@ In plain language, for each normalised query and product: - **Events at low positions count more.** A click on the product at position 8 says more than a click at position 1, because position 1 gets clicked whatever it is. An event at position `p` is multiplied by `p ^ position_eta`, capped at `max_position_weight`. Without this correction the ranking learns position bias, cementing whatever the engine showed first. Events from the `explore` source skip the correction. - **Recent events count more.** Each event's weight halves every `half_life_days` (30 by default), so a product that used to convert but no longer does fades out. - **Sparse data is ignored.** Only events within `window_days` are considered, and a query and product pair needs events from at least `min_sessions` distinct shoppers before it gets a score. Sessions searching faster than `guards.max_searches_per_minute` are excluded as bots. +- **One vote per shopper.** Each session contributes at most one event of each type per query and product, however many times it repeats the search. With `guards.trusted_sessions_only` (the default), only sessions that hold a cart or belong to a known customer count at all. +- **Staff overrides win.** Products excluded from learning for a query, and events before a query's learning was reset, are ignored. See [Abuse and manipulation](#abuse-and-manipulation). - **Scores are relative.** Within a query, each product's score is divided by the best score for that query, giving a `relative` value from 0 to 1. At most `max_products_per_query` products are kept per query. At search time, `Lunar\SearchRelevance\Signals\QueryAffinitySignal` looks up the `relative` score for each product in the window, `Lunar\SearchRelevance\Signals\SignalCombiner::combine()` sums every configured signal by weight and clamps to 0..1, and `Lunar\SearchRelevance\Rankers\BucketedRanker` sorts within each bucket of `bucket_size` engine positions by that combined score (ties keep the engine order). Learned products the engine did not return (up to `learned_union.max`, each with `relative` of at least `learned_union.min_relative`) are fetched by id through the same engine and inserted at the head of the second bucket, marked `source: learned`, so they can win a first-page slot without displacing the strongest keyword matches. The ranked window is cached for `cache_ttl` seconds, keyed by model, version, mode, normalised query, filters, and customer. @@ -353,6 +355,30 @@ At search time, `Lunar\SearchRelevance\Signals\QueryAffinitySignal` looks up the The aggregation is a single SQL query on MySQL 8 and Postgres (`Lunar\SearchRelevance\Scoring\MySqlScoreAggregator`, `PostgresScoreAggregator`), chosen from the Lunar database connection's driver. Other drivers, including SQLite, use `PhpScoreAggregator`, which does the same in chunked PHP. The `Lunar\SearchRelevance\Contracts\ScoreAggregator` binding can be swapped like any other; see [Extending search](/2.x/extending/search#search-relevance). +## Abuse and manipulation + +Learned ranking is a feedback loop, so bots and bad actors can try to feed it. The add-on limits the damage in layers: + +- **Structural.** The ranker only reorders within buckets of `bucket_size` hits, and learned products the engine did not return are inserted at the head of the second bucket. Nothing can be pushed into the first bucket unless the engine already put it there, whatever the event volume. +- **Event validation.** An event needs a real search id, a product from that search's shown list, a position inside it, and must arrive within `guards.event_window_minutes` of the search. Repeats of an event already recorded for the search, product, and type are dropped, and the endpoint is rate limited per shopper and per IP. +- **Scoring.** Each session contributes one event of each type per query and product. With `guards.trusted_sessions_only`, only sessions that hold a cart or belong to a known customer count, so a bot minting fresh sessions gains nothing. Sessions searching faster than `guards.max_searches_per_minute` are excluded. +- **Crawlers.** Searches from user agents matching `guards.ignored_user_agents` are neither logged nor ranked, so reporting stays honest and the tables stay small. Headless API clients without a session are still logged; they identify the shopper explicitly on the events endpoint. +- **Refunds and cancellations.** `Lunar\SearchRelevance\Listeners\ForgetPurchases` removes the purchase events an order's attributed lines produced when `Lunar\Core\Events\Orders\OrderCancelled` or `OrderRefunded` fires. +- **Staff overrides.** From the admin panel's query page, staff can exclude a product from learning for a query (its learned score is removed immediately and future events ignored; reversible) or reset learning for a query (everything learned is discarded and only events after the reset count). Both are stored in the `search_learning_overrides` table and applied through `Lunar\SearchRelevance\Learning\Overrides`, which any code can call: + +```php +use Lunar\Core\Models\Product; +use Lunar\SearchRelevance\Learning\Overrides; + +$overrides = app(Overrides::class); + +$overrides->exclude(Product::class, 'cable gland', productId: 42); +$overrides->include(Product::class, 'cable gland', productId: 42); +$overrides->reset(Product::class, 'cable gland'); +``` + +The add-on does not store IP addresses for abuse analysis. The per-IP rate limit on the events endpoint covers the crude case without the privacy obligations. + ## Versioning Every logged search and every score carries a retrieval version, `Lunar\SearchRelevance\RetrievalVersion::current($modelType)`, in the form `n{normaliser_version}:{driver}:{hybrid|keyword}`, for example `n1:typesense:hybrid`. The driver comes from `lunar.search.engine_map` for the model (falling back to `scout.driver`); `hybrid` means the Typesense collection schema declares an `embedding` field, or `lunar.search.meilisearch.embedder` is set. `QueryAffinitySignal` only reads scores whose version matches the current one. @@ -424,5 +450,8 @@ The full config, merged under `lunar.search_relevance`: | `retention_days` | `400` | Raw queries and events older than this are pruned weekly | | `guards.events_rate_limit` | `'60,1'` | Events endpoint rate limit as `attempts,minutes`, per shopper and per IP | | `guards.max_searches_per_minute` | `30` | Sessions searching faster than this are ignored by scoring | +| `guards.event_window_minutes` | `120` | Events are only accepted this long after the search they belong to | +| `guards.trusted_sessions_only` | `true` | Only sessions holding a cart or belonging to a known customer count towards learning | +| `guards.ignored_user_agents` | bot, crawl, spider, ... | Searches from user agents containing any of these substrings are neither logged nor ranked | The three pipeline stages (`Lunar\SearchRelevance\Pipelines\PartNumberRetrieval`, `WidenRequest`, and `RankResults`) are appended to `lunar.search.pipelines.request` and `lunar.search.pipelines.results` by the service provider unless they are already listed. A host that sets those keys explicitly in `config/lunar/search.php` controls their order, and can leave one out; `PartNumberRetrieval` and `WidenRequest` must run in that order, and `WidenRequest` must run before any stage that changes the page. diff --git a/2.x/admin/search-relevance.mdx b/2.x/admin/search-relevance.mdx index ca1c414..6af7d72 100644 --- a/2.x/admin/search-relevance.mdx +++ b/2.x/admin/search-relevance.mdx @@ -78,6 +78,15 @@ The table is empty until the scoring run has produced scores for the query; the A second panel, **Raw variants**, lists the raw queries that normalise to this one (`running shoe`, `Running Shoes`, `running-shoes`) with their search counts. If two variants that should be the same query appear as separate queries in the overview tables, that is a normaliser change; see [custom query normaliser](/2.x/extending/search#custom-query-normaliser). +### Excluding a product and resetting a query + +Two controls on the query page handle manipulated or embarrassing results: + +- **Exclude from learning**, in each learned row's actions menu. The product's learned score for this query is removed immediately, and scoring ignores its events from then on. Excluded products are listed under the learned table with an **Allow again** button that reverses the exclusion. +- **Reset learning**, in the page header. Everything learned for the query is discarded and only events after the reset count, so the query relearns from clean traffic. The page shows when learning was last reset. + +Both take effect on the next search; neither touches the logged searches or events, so the reporting on the overview page is unchanged. + ## Settings page Under **Settings > Store > Search relevance**. It has one control, the mode switch (`off`, `shadow`, `on`, each with a one-line description), and two read-only panels: the event weights from `scoring.weights`, and the current retrieval version per ranked model, so staff can see that a normaliser or engine change has started a fresh version. From e98940b2c0428b02186ba58542e05b9b0a66c0a8 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 18:13:18 +0100 Subject: [PATCH 07/13] docs(2.x): note that page() is optional and the request page still applies Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2.x/addons/search.mdx b/2.x/addons/search.mdx index efed641..c9858d5 100644 --- a/2.x/addons/search.mdx +++ b/2.x/addons/search.mdx @@ -268,7 +268,7 @@ The results pipeline receives a `Lunar\Search\Pipelines\SearchResponse`: ### Page and per-page -`AbstractEngine::page(int $page)` sets the page the engine fetches, alongside the existing `perPage()`. Both are readable with `getPage()` and `getPerPage()`. A request stage can fetch a wider window than the caller asked for (set `page(1)` and a larger `perPage()`), then a results stage slices back to `requestedPage` and `requestedPerPage` from the request passable. +`AbstractEngine::page(int $page)` sets the page the engine fetches, alongside the existing `perPage()`. Until it is called, the page resolves from the request's `page` query parameter as it always has; `getPage()` returns whichever applies, and `getPerPage()` the page size. A request stage can fetch a wider window than the caller asked for (set `page(1)` and a larger `perPage()`), then a results stage slices back to `requestedPage` and `requestedPerPage` from the request passable. ```php use Lunar\Search\Facades\Search; From 14b53f8663f7dbf28ab03696a8116aa5bd4cd674 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 18:22:32 +0100 Subject: [PATCH 08/13] docs(2.x): the search relevance settings page is read-only; mode is configuration Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 8 ++++---- 2.x/admin/search-relevance.mdx | 12 +++++------- 2 files changed, 9 insertions(+), 11 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 269d5d5..374875e 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -42,7 +42,7 @@ Run the migrations: php artisan migrate ``` -This creates four tables, prefixed like every other Lunar table: `search_queries`, `search_events`, `search_query_scores`, and `search_relevance_settings`. It also seeds the `search:manage-relevance` permission used by the panel section. +This creates four tables, prefixed like every other Lunar table: `search_queries`, `search_events`, `search_query_scores`, and `search_learning_overrides`. It also seeds the `search:manage-relevance` permission used by the panel section. Make sure a queue worker and the scheduler are running. Logging, event recording, and attribution are dispatched as queued jobs, and scoring runs from the scheduler: @@ -65,7 +65,7 @@ php artisan vendor:publish --tag=panel-all-assets --force php artisan lunar:panel:link ``` -The panel section is documented under [Admin Panel: Search Relevance](/2.x/admin/search-relevance). Without the panel, everything else (logging, scoring, ranking, part-number retrieval) still works; the mode is then read from config only. +The panel section is documented under [Admin Panel: Search Relevance](/2.x/admin/search-relevance). Without the panel, everything else (logging, scoring, ranking, part-number retrieval) still works. Optionally publish the config: @@ -190,7 +190,7 @@ The `mode` config key (or the `LUNAR_SEARCH_RELEVANCE_MODE` environment variable 2. Let the scoring job run for a few weeks. Check the panel's overview or `lunar:search-relevance:replay` for the share of searches where the ranked order placed the purchased product higher. 3. Switch to `on` when the replay shows a consistent improvement. -The admin panel persists a mode override in the `search_relevance_settings` table. `Lunar\SearchRelevance\Settings::mode()` returns the override when one is set and the config value otherwise; every code path reads the mode through `Settings`, so a panel change takes effect immediately without a deploy. +The mode is store configuration, set in code or through the `LUNAR_SEARCH_RELEVANCE_MODE` environment variable like every other Lunar setting, so it is versioned and can differ per environment (`shadow` on staging, `on` in production). The admin panel shows the current mode but cannot change it. Before switching a store to `on`, run the [replay command](#replay) or read the uplift card in the panel to confirm the learned order performs better. ## Storefront tracking @@ -432,7 +432,7 @@ The full config, merged under `lunar.search_relevance`: | Key | Default | Description | |---|---|---| -| `mode` | `env('LUNAR_SEARCH_RELEVANCE_MODE', 'shadow')` | `off`, `shadow`, or `on`; see [Modes](#modes). Overridden by the panel setting when one is saved. | +| `mode` | `env('LUNAR_SEARCH_RELEVANCE_MODE', 'shadow')` | `off`, `shadow`, or `on`; see [Modes](#modes). | | `models` | `[Lunar\Core\Models\Product::class]` | Searchable models whose results are logged and ranked | | `window` | `250` | Candidate window fetched from the engine and reordered. Requests beyond the window (`page * perPage > window`) pass through unranked but are still logged. | | `bucket_size` | `10` | Hits are reordered only within buckets of this size | diff --git a/2.x/admin/search-relevance.mdx b/2.x/admin/search-relevance.mdx index 6af7d72..9f494dc 100644 --- a/2.x/admin/search-relevance.mdx +++ b/2.x/admin/search-relevance.mdx @@ -1,9 +1,9 @@ --- title: "Search Relevance" -description: "The admin panel section shipped by the Search Relevance add-on: search KPIs, per-query learned rankings, and the mode switch." +description: "The admin panel section shipped by the Search Relevance add-on: search KPIs, per-query learned rankings, and staff overrides." --- -The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-relevance`) registers a section in the admin panel for reading what shoppers search for, what the ranking has learned, and switching the add-on from shadow mode to live. It is an add-on section, built with the same extension API described under [Extending the panel](/2.x/admin/extending/overview), so it appears only when both `lunarphp/panel` and the add-on are installed. +The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-relevance`) registers a section in the admin panel for reading what shoppers search for, what the ranking has learned, and correcting it where needed. It is an add-on section, built with the same extension API described under [Extending the panel](/2.x/admin/extending/overview), so it appears only when both `lunarphp/panel` and the add-on are installed. ## Setup @@ -36,7 +36,7 @@ The section adds a **Search** group to the sidebar with a **Search relevance** i | `panel.search-relevance.index` | [Overview](#overview-page) | | `panel.search-relevance.query` | [Query page](#query-page); the normalised query is the route parameter, with an optional `model` query string for installs that rank more than one model | | `panel.search-relevance.product` | JSON feed for the [product slot](#product-slot) | -| `panel.settings.search-relevance.index`, `panel.settings.search-relevance.update` | [Settings page](#settings-page) | +| `panel.settings.search-relevance.index` | [Settings page](#settings-page) | ## Overview page @@ -89,9 +89,7 @@ Both take effect on the next search; neither touches the logged searches or even ## Settings page -Under **Settings > Store > Search relevance**. It has one control, the mode switch (`off`, `shadow`, `on`, each with a one-line description), and two read-only panels: the event weights from `scoring.weights`, and the current retrieval version per ranked model, so staff can see that a normaliser or engine change has started a fresh version. - -Switching to `on` asks for confirmation, because it changes what every shopper sees. The chosen mode is persisted in the add-on's `search_relevance_settings` table and takes effect immediately; the config value (`lunar.search_relevance.mode`) is only the fallback when nothing has been saved here. See [Modes](/2.x/addons/search-relevance#modes). +Under **Settings > Store > Search relevance**. The page is read-only: like the rest of Lunar, how the store behaves is configured in code, and this screen exists so staff can see why search behaves as it does. It shows the current mode with the environment variable that sets it (`LUNAR_SEARCH_RELEVANCE_MODE`), the event weights from `scoring.weights`, when scoring last ran and its schedule, and the current retrieval version per ranked model, so staff can see that a normaliser or engine change has started a fresh version. See [Modes](/2.x/addons/search-relevance#modes). ### When to switch from shadow to on @@ -101,7 +99,7 @@ Leave the add-on in `shadow` until all of the following hold: - The scoring job has run and the top queries have learned products with more than a handful of sessions. - The Uplift card shows the learned order placing the purchased product higher in clearly more searches than it places it lower, over a range of at least a few weeks. -If uplift is flat, the likely causes are too little traffic for `scoring.min_sessions`, tracking missing from part of the storefront, or a normaliser splitting the same query across variants. Switching to `on` without evidence is safe (bucketed reordering cannot move a weak match above a strong one) but pointless. +If uplift is flat, the likely causes are too little traffic for `scoring.min_sessions`, tracking missing from part of the storefront, or a normaliser splitting the same query across variants. Switching to `on` without evidence is safe (bucketed reordering cannot move a weak match above a strong one) but pointless. The switch itself is a developer change: set `LUNAR_SEARCH_RELEVANCE_MODE=on` in the environment and deploy. ## Dashboard widget From 5e8e0736f51c8f598208e16015b652696e13da9a Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 18:29:49 +0100 Subject: [PATCH 09/13] docs(2.x): follow the search relevance package restructure (DataObjects, Jobs, Observers) Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 4 ++-- 2.x/extending/search.mdx | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 374875e..3242f50 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -298,7 +298,7 @@ The endpoint always answers with an empty `204`, whether or not the event was ac A click is the start of a chain the add-on follows through to a purchase: 1. The click stores `attribution.{product_id}` in the shopper's session (`search_id`, `position`, `source`, and an expiry) for `attribution_ttl_minutes` (30 by default). -2. When a `Lunar\Core\Models\CartLine` is created, `Lunar\SearchRelevance\Listeners\AttributeCartLine` (an Eloquent `created` observer) resolves the line's product from its purchasable (product variants only), looks up the attribution, and if one is present writes it to `meta['search_attribution']` on the line and records a `basket` event. +2. When a `Lunar\Core\Models\CartLine` is created, `Lunar\SearchRelevance\Observers\CartLineObserver` (an Eloquent `created` observer) resolves the line's product from its purchasable (product variants only), looks up the attribution, and if one is present writes it to `meta['search_attribution']` on the line and records a `basket` event. 3. When `Lunar\Core\Events\Orders\OrderPlaced` fires, `Lunar\SearchRelevance\Listeners\AttributeOrderLines` records a `purchase` event for every order line whose `meta` carries `search_attribution`. Order creation already copies cart line `meta` to the order line, so no pipeline change is needed. A cart or order line attributed to a search carries this in `meta`: @@ -315,7 +315,7 @@ $line->meta['search_attribution']; Shopper identity is the Lunar cart session identifier when `session_key` is `cart` (the default), stored as `cart:{id}`. It survives login, and it is what cart and order lines already relate to. Set `session_key` to `session` to use the Laravel session id (`session:{id}`) instead; that is also the fallback when no cart exists yet. -Every event is written by the queued `Lunar\SearchRelevance\Events\RecordEvent` job, which re-validates the search id, the product, and the position before inserting, so nothing runs on the request path beyond the session write. +Every event is written by the queued `Lunar\SearchRelevance\Jobs\RecordEvent` job, which re-validates the search id, the product, and the position before inserting, so nothing runs on the request path beyond the session write. ## Scoring diff --git a/2.x/extending/search.mdx b/2.x/extending/search.mdx index ad6d291..3dd9402 100644 --- a/2.x/extending/search.mdx +++ b/2.x/extending/search.mdx @@ -308,7 +308,7 @@ namespace App\Search; use Lunar\Core\Models\ProductVariant; use Lunar\SearchRelevance\Contracts\Signal; -use Lunar\SearchRelevance\Data\RankingContext; +use Lunar\SearchRelevance\DataObjects\RankingContext; class StockLevelSignal implements Signal { @@ -344,14 +344,14 @@ return [ ### Swapping the ranker -The default `BucketedRanker` reorders hits by combined score within buckets of `bucket_size` engine positions, so a weak keyword match never overtakes a strong one. `Lunar\SearchRelevance\Rankers\NullRanker` returns the window untouched, useful for measuring logging overhead without ranking. A ranker receives the `RankingContext` and a `Lunar\SearchRelevance\Data\HitCollection` (an `Illuminate\Support\Collection` of `Lunar\SearchRelevance\Data\Hit` objects with `productId`, `originalPosition`, `score`, `document`, `source`, and a mutable `boost`) and returns the collection in display order: +The default `BucketedRanker` reorders hits by combined score within buckets of `bucket_size` engine positions, so a weak keyword match never overtakes a strong one. `Lunar\SearchRelevance\Rankers\NullRanker` returns the window untouched, useful for measuring logging overhead without ranking. A ranker receives the `RankingContext` and a `Lunar\SearchRelevance\DataObjects\HitCollection` (an `Illuminate\Support\Collection` of `Lunar\SearchRelevance\DataObjects\Hit` objects with `productId`, `originalPosition`, `score`, `document`, `source`, and a mutable `boost`) and returns the collection in display order: ```php namespace App\Search; use Lunar\SearchRelevance\Contracts\Ranker; -use Lunar\SearchRelevance\Data\HitCollection; -use Lunar\SearchRelevance\Data\RankingContext; +use Lunar\SearchRelevance\DataObjects\HitCollection; +use Lunar\SearchRelevance\DataObjects\RankingContext; use Lunar\SearchRelevance\Signals\SignalCombiner; class TopThreeOnlyRanker implements Ranker From ee7e667efe2d152eaece96aa9cea4ae5548f2755 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 19:16:38 +0100 Subject: [PATCH 10/13] docs(2.x): drop the Scout database driver from the search relevance pages Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 3 +-- 2.x/addons/search.mdx | 4 ++-- 2.x/extending/search.mdx | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 3242f50..8fe17db 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -23,7 +23,6 @@ The add-on works on the ordered list of product ids the engine returns. It does |---|---|---|---| | Typesense | Yes | Yes, prefix and infix | Full support. Hybrid search is padded by a distance threshold (see [Vector distance threshold](#vector-distance-threshold)). | | Meilisearch | Yes | Yes, prefix only | No infix matching: `MB32A` will not match `HAG-MB-32A`. Stores whose customers search for fragments from inside a code need Typesense. Meilisearch also pads a page with partial matches after the full matches (see [Meilisearch page padding](#meilisearch-page-padding)). | -| Database | Yes | No | The Scout database driver has no per-field parameters, so the part-number stage has nothing to apply. Ranking works as normal. | | Custom engines | Opt in | Opt in | A custom `Lunar\Search\Engines\AbstractEngine` subclass gets ranking once its `get()` calls `pipeRequest()` and `pipeResults()`; see [Pipelines](/2.x/addons/search#pipelines). | ## Installation @@ -91,7 +90,7 @@ For a part number, the `Lunar\SearchRelevance\Pipelines\PartNumberRetrieval` req | Keep every token | `drop_tokens_threshold: 0` | `matchingStrategy: all` | | No semantic padding | `vector_query` omitted | `hybrid.semanticRatio: 0`, when an embedder is configured | -Part-number searches are still logged and tracked, but they are not ranked: per-code queries are too sparse to learn from, and the engine order is already the right one. The normaliser also skips stemming for them. The database engine has no per-field request parameters, so the stage leaves it untouched. +Part-number searches are still logged and tracked, but they are not ranked: per-code queries are too sparse to learn from, and the engine order is already the right one. The normaliser also skips stemming for them. The stage applies these parameters through `AbstractEngine::withParams()`, so nothing engine-specific lives in the engine itself. diff --git a/2.x/addons/search.mdx b/2.x/addons/search.mdx index c9858d5..1a62531 100644 --- a/2.x/addons/search.mdx +++ b/2.x/addons/search.mdx @@ -291,13 +291,13 @@ $request->engine->withParams([ ]); ``` -`getParams()` returns the merged overrides. `DatabaseEngine` has no request parameters and ignores them. +`getParams()` returns the merged overrides. ### Meta fields Both data objects carry a free-form `meta` array for engines and stages to annotate: -- `SearchHit::$meta` holds the engine's own score under `score` where the engine provides one: Typesense `text_match`, Meilisearch `_rankingScore` (the engine requests it with `showRankingScore`). The database engine leaves it unset. Stages add their own keys. +- `SearchHit::$meta` holds the engine's own score under `score` where the engine provides one: Typesense `text_match`, Meilisearch `_rankingScore` (the engine requests it with `showRankingScore`). Stages add their own keys. - `SearchResults::$meta` is empty by default; results stages annotate it, for example with a search id. Both default to `[]`, so existing calls to `SearchResults::from()` and `SearchHit::from()` are unaffected. diff --git a/2.x/extending/search.mdx b/2.x/extending/search.mdx index 3dd9402..02cd919 100644 --- a/2.x/extending/search.mdx +++ b/2.x/extending/search.mdx @@ -246,7 +246,7 @@ The [Search Relevance](/2.x/addons/search-relevance) add-on (`lunarphp/search-re | `QueryNormaliser` | `Lunar\SearchRelevance\Normalisers\DefaultQueryNormaliser` | Folds raw queries into the key scores are stored under, and classifies part numbers | | `Signal` | `Lunar\SearchRelevance\Signals\QueryAffinitySignal` | Scores product ids 0..1 for a query; several can be combined by weight | | `Ranker` | `Lunar\SearchRelevance\Rankers\BucketedRanker` | Reorders the candidate window from the combined signal scores | -| `ScoreAggregator` | Chosen per database driver | Turns logged events into `search_query_scores` rows | +| `ScoreAggregator` | Chosen per database connection (MySQL, Postgres, or PHP for the rest) | Turns logged events into `search_query_scores` rows | ### Custom query normaliser From a9f118a227a4b68c8841ac2ffe1cf2eb377bb2fc Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 19:17:34 +0100 Subject: [PATCH 11/13] docs(2.x): drop the vector distance threshold section from the search relevance page It is a lunarphp/search setting, documented on the Search add-on page. Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 12 +----------- 2.x/addons/search.mdx | 4 ++++ 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 8fe17db..6f28c6e 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -21,7 +21,7 @@ The add-on works on the ordered list of product ids the engine returns. It does | Engine | Ranking | Part-number retrieval | Notes | |---|---|---|---| -| Typesense | Yes | Yes, prefix and infix | Full support. Hybrid search is padded by a distance threshold (see [Vector distance threshold](#vector-distance-threshold)). | +| Typesense | Yes | Yes, prefix and infix | Full support, including hybrid search (see [Search: Typesense](/2.x/addons/search#typesense) for the vector distance threshold). | | Meilisearch | Yes | Yes, prefix only | No infix matching: `MB32A` will not match `HAG-MB-32A`. Stores whose customers search for fragments from inside a code need Typesense. Meilisearch also pads a page with partial matches after the full matches (see [Meilisearch page padding](#meilisearch-page-padding)). | | Custom engines | Opt in | Opt in | A custom `Lunar\Search\Engines\AbstractEngine` subclass gets ranking once its `get()` calls `pipeRequest()` and `pipeResults()`; see [Pipelines](/2.x/addons/search#pipelines). | @@ -163,16 +163,6 @@ php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh Meilisearch's default `last` matching strategy lists every document matching all query terms first, then fills the rest of the page with documents matching fewer terms. For letter-only prefixes such as `HAG-MB`, which the classifier treats as text because they contain no digit, page one is the whole family followed by partial matches. This is not a ranking error, but a storefront that shows a result count will show an inflated total. Part numbers (with a digit) set `matchingStrategy: all` and are not padded. -## Vector distance threshold - -Typesense hybrid search runs a `k: 200` vector query alongside the keyword query. Without a threshold, that vector query pads every result set with the 200 nearest neighbours of whatever the query was, so a nonsense query returns 200 products. Lunar sets `distance_threshold` on the vector query from `lunar.search.typesense.vector_distance_threshold` (default `0.6`; `0` disables it). This lives in the `lunarphp/search` config, not this add-on's, and applies whether or not the add-on is installed. See [Search: Typesense](/2.x/addons/search#typesense). - -Meilisearch's counterpart, when an embedder is configured, is `lunar.search.meilisearch.ranking_score_threshold`. - - -A nonsense query with a threshold in place can return zero results. Render a zero-results state in the storefront rather than loosening the threshold to pad the page; see the [search guide](/2.x/guides/search#make-search-learn). - - ## Modes The `mode` config key (or the `LUNAR_SEARCH_RELEVANCE_MODE` environment variable) controls how much the add-on does: diff --git a/2.x/addons/search.mdx b/2.x/addons/search.mdx index 1a62531..0327944 100644 --- a/2.x/addons/search.mdx +++ b/2.x/addons/search.mdx @@ -499,4 +499,8 @@ Typesense is available as a driver (`Lunar\Search\Engines\TypesenseEngine`) once When the collection schema declares an auto-embedding `embedding` field, the engine adds a hybrid vector query to every search with a term. The query carries `distance_threshold` from `lunar.search.typesense.vector_distance_threshold` (default `0.6`) so that a term with no semantic neighbours cannot pad the result set with its 200 nearest vectors; set it to `0` to send the bare `k: 200` query. A `vector_query` search parameter in `config/scout.php` overrides the generated one entirely. + +With a threshold in place, a nonsense query can return zero results instead of 200 padded ones. Render a zero-results state in the storefront rather than loosening the threshold to fill the page. + + For part-number search, declare `skus` and `skus_normalised` as `string[]` fields with `infix: true` and list both in `query_by`; see [Search Relevance: Typesense setup](/2.x/addons/search-relevance#typesense-setup) for the full entries. From 6355dcd6b44b07fbc347b95f7518ae4eefec8e5e Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 19:19:28 +0100 Subject: [PATCH 12/13] docs(2.x): move product code search setup to the search add-on page The skus_normalised field, Typesense schema entries and Meilisearch typo-tolerance step apply with or without the relevance add-on. Co-Authored-By: Claude Fable 5.1 --- 2.x/addons/search-relevance.mdx | 69 +------------------------------ 2.x/addons/search.mdx | 73 ++++++++++++++++++++++++++++++++- 2 files changed, 73 insertions(+), 69 deletions(-) diff --git a/2.x/addons/search-relevance.mdx b/2.x/addons/search-relevance.mdx index 6f28c6e..5b39382 100644 --- a/2.x/addons/search-relevance.mdx +++ b/2.x/addons/search-relevance.mdx @@ -94,74 +94,7 @@ Part-number searches are still logged and tracked, but they are not ranked: per- The stage applies these parameters through `AbstractEngine::withParams()`, so nothing engine-specific lives in the engine itself. -### The `skus_normalised` field - -`Lunar\Core\Search\ProductIndexer` indexes variant SKUs twice: `skus` as printed, and `skus_normalised` uppercased with every non-alphanumeric character stripped. That is what lets `HAGMB` prefix-match `HAG-MB-32A`. Both fields are part of core, so they are indexed whether or not this add-on is installed; the add-on is what makes retrieval use them. - -### Typesense setup - -Typesense needs both fields declared in the collection schema and listed in `query_by`, together with the position-aligned `infix` and `num_typos` lists. Both live in the host application's `config/scout.php`, under `typesense.model-settings`: - -```php -// config/scout.php -'typesense' => [ - // ... - 'model-settings' => [ - Lunar\Core\Models\Product::class => [ - 'collection-schema' => [ - 'fields' => [ - ['name' => 'id', 'type' => 'string'], - ['name' => 'name_en', 'type' => 'string'], - ['name' => 'description_en', 'type' => 'string', 'optional' => true], - ['name' => 'skus', 'type' => 'string[]', 'infix' => true], - ['name' => 'skus_normalised', 'type' => 'string[]', 'infix' => true], - ['name' => 'brand', 'type' => 'string', 'facet' => true, 'optional' => true], - ['name' => 'status', 'type' => 'string', 'facet' => true], - ['name' => 'created_at', 'type' => 'int64'], - // ... - ], - ], - 'search-parameters' => [ - 'query_by' => 'name_en,description_en,skus,skus_normalised', - 'query_by_weights' => '4,1,3,3', - 'infix' => 'off,off,always,always', - 'num_typos' => '2,2,0,0', - 'prefix' => 'true,true,true,true', - ], - ], - ], -], -``` - -Every entry in `infix`, `num_typos`, `prefix`, and `query_by_weights` is positional and must have exactly as many values as `query_by`, or Typesense rejects the request. Only `infix: true` in the schema enables infix search for a field; `always` in the search parameters then uses it. - -Changing the collection schema requires a reindex: - -```bash -php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh -``` - - -The `--refresh` flag drops and recreates the collection, which is what a schema change needs. Without it, Typesense keeps the old field list and the new fields are silently ignored. - - -### Meilisearch setup - -Meilisearch already prefix-matches the last word of a query, so recall of a SKU family is complete out of the box. Typo tolerance is the defect: under default settings an eight-character code returns several unrelated products per page, and none once typos are disabled on the SKU attributes. That is an index setting, not something a request can change, so the [`lunar:meilisearch:setup`](/2.x/addons/search#meilisearch) command applies it: - -```bash -php artisan lunar:meilisearch:setup -``` - -Re-run it after upgrading to a version of Lunar with `skus_normalised`. The command reads each indexer's `getExactMatchFields()` (`['skus', 'skus_normalised']` for products) and sets `typoTolerance.disableOnAttributes` on the index. Then reindex so the new field is populated: - -```bash -php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh -``` - -#### Meilisearch page padding - -Meilisearch's default `last` matching strategy lists every document matching all query terms first, then fills the rest of the page with documents matching fewer terms. For letter-only prefixes such as `HAG-MB`, which the classifier treats as text because they contain no digit, page one is the whole family followed by partial matches. This is not a ranking error, but a storefront that shows a result count will show an inflated total. Part numbers (with a digit) set `matchingStrategy: all` and are not padded. +The retrieval side of this depends on the `skus` and `skus_normalised` fields that core indexes and on engine settings that belong to `lunarphp/search`: the Typesense schema and `query_by` entries, and the Meilisearch typo-tolerance step. See [Search: Product code search](/2.x/addons/search#product-code-search) for that setup; the add-on only decides when to use it. ## Modes diff --git a/2.x/addons/search.mdx b/2.x/addons/search.mdx index 0327944..1d04a27 100644 --- a/2.x/addons/search.mdx +++ b/2.x/addons/search.mdx @@ -503,4 +503,75 @@ When the collection schema declares an auto-embedding `embedding` field, the eng With a threshold in place, a nonsense query can return zero results instead of 200 padded ones. Render a zero-results state in the storefront rather than loosening the threshold to fill the page. -For part-number search, declare `skus` and `skus_normalised` as `string[]` fields with `infix: true` and list both in `query_by`; see [Search Relevance: Typesense setup](/2.x/addons/search-relevance#typesense-setup) for the full entries. +For product code search, declare `skus` and `skus_normalised` as `string[]` fields with `infix: true` and list both in `query_by`; see [Product code search](#product-code-search) below for the full entries. + +## Product code search + +### The `skus_normalised` field + +`Lunar\Core\Search\ProductIndexer` indexes variant SKUs twice: `skus` as printed, and `skus_normalised` uppercased with every non-alphanumeric character stripped. That is what lets `HAGMB` prefix-match `HAG-MB-32A`. Both fields are part of core and are indexed whatever engine is in use. With the settings below, a shopper typing a partial code gets the whole family of that code; the [Search Relevance](/2.x/addons/search-relevance#part-number-search) add-on goes further by detecting code-like queries and restricting retrieval to these fields. + +### Typesense setup + +Typesense needs both fields declared in the collection schema and listed in `query_by`, together with the position-aligned `infix` and `num_typos` lists. Both live in the host application's `config/scout.php`, under `typesense.model-settings`: + +```php +// config/scout.php +'typesense' => [ + // ... + 'model-settings' => [ + Lunar\Core\Models\Product::class => [ + 'collection-schema' => [ + 'fields' => [ + ['name' => 'id', 'type' => 'string'], + ['name' => 'name_en', 'type' => 'string'], + ['name' => 'description_en', 'type' => 'string', 'optional' => true], + ['name' => 'skus', 'type' => 'string[]', 'infix' => true], + ['name' => 'skus_normalised', 'type' => 'string[]', 'infix' => true], + ['name' => 'brand', 'type' => 'string', 'facet' => true, 'optional' => true], + ['name' => 'status', 'type' => 'string', 'facet' => true], + ['name' => 'created_at', 'type' => 'int64'], + // ... + ], + ], + 'search-parameters' => [ + 'query_by' => 'name_en,description_en,skus,skus_normalised', + 'query_by_weights' => '4,1,3,3', + 'infix' => 'off,off,always,always', + 'num_typos' => '2,2,0,0', + 'prefix' => 'true,true,true,true', + ], + ], + ], +], +``` + +Every entry in `infix`, `num_typos`, `prefix`, and `query_by_weights` is positional and must have exactly as many values as `query_by`, or Typesense rejects the request. Only `infix: true` in the schema enables infix search for a field; `always` in the search parameters then uses it. + +Changing the collection schema requires a reindex: + +```bash +php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh +``` + + +The `--refresh` flag drops and recreates the collection, which is what a schema change needs. Without it, Typesense keeps the old field list and the new fields are silently ignored. + + +### Meilisearch setup + +Meilisearch already prefix-matches the last word of a query, so recall of a SKU family is complete out of the box. Typo tolerance is the defect: under default settings an eight-character code returns several unrelated products per page, and none once typos are disabled on the SKU attributes. That is an index setting, not something a request can change, so the [`lunar:meilisearch:setup`](/2.x/addons/search#meilisearch) command applies it: + +```bash +php artisan lunar:meilisearch:setup +``` + +Re-run it after upgrading to a version of Lunar with `skus_normalised`, then reindex so the new field is populated: + +```bash +php artisan lunar:search:index "Lunar\Core\Models\Product" --refresh +``` + +### Meilisearch page padding + +Meilisearch's default `last` matching strategy lists every document matching all query terms first, then fills the rest of the page with documents matching fewer terms. For a letter-only prefix such as `HAG-MB`, page one is the whole family followed by partial matches. This is not a ranking error, but a storefront that shows a result count will show an inflated total. Sending `matchingStrategy: all` (which the Search Relevance add-on does for queries it classifies as part numbers) avoids the padding. From 4cc56e945a55de4bae1766e9a9537e77b1c17139 Mon Sep 17 00:00:00 2001 From: Glenn Jacobs Date: Mon, 14 Sep 2026 19:46:25 +0100 Subject: [PATCH 13/13] docs(2.x): product report page and sidebar card replace the product edit slot Co-Authored-By: Claude Fable 5.1 --- 2.x/admin/search-relevance.mdx | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/2.x/admin/search-relevance.mdx b/2.x/admin/search-relevance.mdx index 9f494dc..29bedf2 100644 --- a/2.x/admin/search-relevance.mdx +++ b/2.x/admin/search-relevance.mdx @@ -35,7 +35,8 @@ The section adds a **Search** group to the sidebar with a **Search relevance** i | --- | --- | | `panel.search-relevance.index` | [Overview](#overview-page) | | `panel.search-relevance.query` | [Query page](#query-page); the normalised query is the route parameter, with an optional `model` query string for installs that rank more than one model | -| `panel.search-relevance.product` | JSON feed for the [product slot](#product-slot) | +| `panel.search-relevance.product` | [Product report page](#product-report-and-sidebar-card) | +| `panel.search-relevance.product.summary` | JSON feed for the product edit sidebar card | | `panel.settings.search-relevance.index` | [Settings page](#settings-page) | ## Overview page @@ -105,9 +106,11 @@ If uplift is flat, the likely causes are too little traffic for `scoring.min_ses `Lunar\SearchRelevance\Panel\Widgets\SearchConversionWidget` adds a half-width **Search conversion** widget to the dashboard, showing searches and the search conversion rate for the dashboard's date range, each with its change against the previous range, and a "View report" link to the overview for the same range. Staff can reorder, hide, and re-add it like any first-party widget, and it is hidden from staff without the permission. -## Product slot +## Product report and sidebar card -On the product edit page, the section injects a **Search performance** card after the content section (the `products.edit:content:after` zone). It loads on demand from `panel.search-relevance.product` and lists up to twenty queries the product has been clicked, added to cart, or purchased from, with the event counts per query and the product's relative score for that query where one has been learned. Each query links to its query page, so a merchandiser editing a product can see which searches it is expected to answer. +Each product has a report page in the section (`panel.search-relevance.product`) listing the queries it has been clicked, added to cart, or purchased from, with the event counts per query and the product's relative score for that query where one has been learned. Each query links to its query page, and the header links to the product's edit page. Product rows on the query page link here. + +On the product edit page, the section adds a compact **Search** card to the sidebar (the `products.edit:sidebar:after` zone), alongside status, type and organization. It loads on demand and shows the product's top three queries with their relative score and a purchases / clicks count, plus a link to the full report. Products with no search activity get no card at all, so the edit page is unchanged for them. ## Global search