Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
0bc0e90
Add a project inventory built from uploaded SBOMs
simonchrz Oct 27, 2026
895f22a
Skip the project's own packages in uploaded SBOMs
simonchrz Oct 27, 2026
bd05533
Delete projects from the project page
simonchrz Oct 27, 2026
b5eec9c
Drop vulnerability markers from the project inventory
simonchrz Oct 27, 2026
8855c86
Match more SBOM names to cached packages
simonchrz Oct 27, 2026
1c706ff
Review fixes for the project inventory
simonchrz Oct 27, 2026
d01d32e
Merge remote-tracking branch 'origin/main' into feat/project-inventory
simonchrz Oct 30, 2026
9458dc8
Share one PURL-to-handler mapping between the mirror and the inventory
simonchrz Oct 30, 2026
479f26e
Start project pruning with the project API and move enrichment routes…
simonchrz Oct 30, 2026
c6635f3
Merge remote-tracking branch 'origin/main' into feat/project-inventory
simonchrz Oct 31, 2026
7a77baf
Keep namespace/name for Swift package URLs in the handler mapping
simonchrz Oct 31, 2026
1cadc4a
Compare Go main modules with their case
simonchrz Oct 31, 2026
3a0e79d
Fuzz SBOM parsing and test concurrent project uploads
simonchrz Oct 31, 2026
e358666
Merge remote-tracking branch 'origin/main' into feat/project-inventory
simonchrz Nov 4, 2026
88c0ba8
Use sbom's root detection for the project's own packages
simonchrz Nov 4, 2026
6d3410d
Use sbom's ProjectRoots for the project's own packages
simonchrz Nov 4, 2026
7095ab6
Keep the Projects link on an empty projects page
simonchrz Nov 4, 2026
60872fe
Use sbom v0.1.9
simonchrz Nov 5, 2026
1e916bf
Merge remote-tracking branch 'origin/main' into feat/project-inventory
simonchrz Nov 5, 2026
c20f64e
Keep the apk namespace for the mirror's repository lookup
simonchrz Nov 5, 2026
4d70793
Name the apk repository in mirror errors
simonchrz Nov 5, 2026
eb34a1a
Name the apk repository, not its URL, in index errors
simonchrz Nov 5, 2026
903fd94
Test that apk index errors leave out the URL
simonchrz Nov 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1343,6 +1343,17 @@ Registered when `hosted.enabled` is set. Pushing and deleting require `Authoriza
| `GET /api/hosted/{ecosystem}/{name}` | List the archives pushed for a package |
| `DELETE /api/hosted/{ecosystem}/{name}/{version}` | Delete a pushed version |

### Projects API

Registered when `projects.enabled` is set. Uploading and deleting require `Authorization: Bearer <projects.token>`. See [Project inventory](docs/configuration.md#project-inventory).

| Endpoint | Description |
|----------|-------------|
| `PUT /api/projects/{project}/sbom` | Replace a project's recorded package versions with the ones the SBOM in the body lists |
| `GET /api/projects` | List projects, or with `?purl=pkg:npm/lodash@4.17.20` the projects using a package or version |
| `GET /api/projects/{project}` | A project's recorded package versions |
| `DELETE /api/projects/{project}` | Delete a project |

### Cache API

Registered when `cache_delete` is set. It has no authentication of its own.
Expand Down Expand Up @@ -1546,6 +1557,7 @@ The proxy serves a web UI under `/ui`. No separate frontend build is needed -- t
- **Version detail** (`/ui/package/{ecosystem}/{name}/{version}`) -- per-version metadata, integrity hash, artifact cache status, and hit counts.
- **Source browser** (`/ui/package/{ecosystem}/{name}/{version}/browse`) -- browse files inside cached archives with syntax highlighting for text files and image previews.
- **Version diff** (`/ui/package/{ecosystem}/{name}/compare/{v1}...{v2}`) -- side-by-side diff of two cached versions showing added, removed, and changed files.
- **Projects** (`/ui/projects`) -- with the [project inventory](docs/configuration.md#project-inventory) enabled, the package versions each repository's uploaded SBOM lists, marked when outdated. Package pages then show which projects use each version.

## Monitoring

Expand Down
5 changes: 4 additions & 1 deletion cmd/proxy/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
// @securityDefinitions.apikey BearerAuth
// @in header
// @name Authorization
// @description "Bearer " followed by hosted.token
// @description "Bearer " followed by hosted.token, or projects.token for /api/projects
//
// The proxy caches package artifacts from upstream registries (npm, cargo, etc.)
// providing faster, more reliable access for development teams.
Expand Down Expand Up @@ -93,6 +93,9 @@
// PROXY_HOSTED_MAX_UPLOAD_SIZE - Max size of one pushed archive (default "100MB")
// PROXY_HOSTED_NPM_SCOPES - npm scopes served from pushed packages only
// PROXY_HOSTED_COMPOSER_VENDORS - Composer vendors served from pushed packages only
// PROXY_PROJECTS_ENABLED - Enable the project inventory (/api/projects, /ui/projects)
// PROXY_PROJECTS_TOKEN - Bearer token for uploading and deleting project SBOMs
// PROXY_PROJECTS_MAX_AGE - Remove projects not uploaded for this long (e.g. "90d")
// PROXY_URL - Proxy URL for proxy hosted
//
// Example:
Expand Down
9 changes: 9 additions & 0 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,15 @@ gradle:
# npm_scopes: ["@acme"]
# composer_vendors: ["acme"]

# Project inventory (disabled by default). Repositories upload an SBOM to
# PUT /api/projects/{project}/sbom, and the UI shows which project uses
# which package versions. Project names are visible on the unauthenticated UI.
# projects:
# enabled: true
# token: "${PROXY_PROJECTS_TOKEN}"
# # Drop projects that have not uploaded for this long ("" keeps them)
# max_age: "90d"

# Re-mirror container image tags on a schedule, so a moved tag is picked up
# even when nobody pulls it. An unchanged tag costs one HEAD request.
# mirror:
Expand Down
4 changes: 4 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,10 @@ Selective package mirroring for pre-populating the proxy cache. Supports multipl

The package also provides a `MetadataCache` for storing raw upstream metadata blobs so the proxy can serve metadata responses offline. The `JobStore` manages async mirror jobs exposed via the `/api/mirror` endpoints.

### `internal/inventory`

Turns an SBOM uploaded to `/api/projects` into the package versions recorded for the project. Package URLs are mapped to the ecosystem names and unversioned PURLs the handlers record packages under, so a project's components join with `packages` by `purl`. Also compares versions for the outdated marker on the project pages.

### `internal/config`

Configuration loading.
Expand Down
83 changes: 83 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1647,6 +1647,89 @@ Generic files never come from an upstream: anything not pushed answers 404. Whil

Pushed archives are stored like cached ones but marked as local. The cache size limit and storage retention never remove them; only a delete does. Scanning, when enabled, checks pushed archives like fetched ones.

## Project inventory

The project inventory answers "which repository uses which version of a package?". Each repository uploads an SBOM of its default branch, and the proxy records the package versions it lists. The UI then shows:

- `/ui/projects`: every project, and per project its package versions, marked when the proxy knows a newer version.
- On a package page: which projects use each version, including versions the proxy hasn't cached.

It is disabled by default. Uploads and deletes require a token:

```yaml
projects:
enabled: true
token: "${PROXY_PROJECTS_TOKEN}"
max_age: "90d"
```

| Config | Environment | Description |
|--------|-------------|-------------|
| `projects.enabled` | `PROXY_PROJECTS_ENABLED` | Register the `/api/projects` endpoints and the UI pages (default `false`) |
| `projects.token` | `PROXY_PROJECTS_TOKEN` | Bearer token required to upload and delete; required when enabled, supports `${VAR}` |
| `projects.max_age` | `PROXY_PROJECTS_MAX_AGE` | Remove projects that haven't uploaded for this long, e.g. `90d` or `720h`; empty or `0` keeps them (default) |

Removing old projects runs hourly. With several instances on one database, one of them does it.

### Uploading an SBOM

`PUT /api/projects/{project}/sbom` takes a CycloneDX (JSON or XML) or SPDX (JSON or tag-value) document of up to 32 MiB and replaces everything recorded for the project with what it lists. The optional `ref` and `branch` query parameters are stored and shown with the project.

The project name is a path such as `group/sub/repo`: segments of letters, digits, `.`, `_` and `-`, at most 255 characters. It can be sent as path segments (`/api/projects/group/sub/repo/sbom`) or URL-encoded as one segment (`/api/projects/group%2Fsub%2Frepo/sbom`).

```bash
syft dir:. -o cyclonedx-json > sbom.json
curl -X PUT --data-binary @sbom.json \
-H "Authorization: Bearer $PROXY_PROJECTS_TOKEN" \
"https://proxy.example.com/api/projects/acme/shop/web/sbom?ref=$(git rev-parse HEAD)&branch=main"
```

The response counts the recorded package versions and the skipped entries:

```json
{"project":"acme/shop/web","components":9,"skipped":6}
```

Entries are recorded by their package URL, with the package named the way the proxy caches it: NuGet IDs lowercased, PyPI names normalized as in PEP 503 (`zope.interface` becomes `zope-interface`), the distribution dropped from `deb`, `rpm` and `apk` package URLs. Entries without a package URL (files, the operating system), package types the proxy doesn't serve (such as `pkg:github`), entries without a version, and conda entries without a `channel` qualifier (the proxy records conda packages as `channel/name`) are skipped. So is the project's own package: the root of a lock file such as `package-lock.json`, and with syft, the Go module its binaries were built from. Duplicates are recorded once.

A GitLab CI job on the default branch could look like this:

```yaml
sbom:
image: alpine:3.20
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
script:
- apk add --no-cache curl
- curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
- syft dir:. -o cyclonedx-json > sbom.json
- >
curl -sSf -X PUT --data-binary @sbom.json
-H "Authorization: Bearer $PROXY_PROJECTS_TOKEN"
"https://proxy.example.com/api/projects/$CI_PROJECT_PATH/sbom?ref=$CI_COMMIT_SHA&branch=$CI_COMMIT_BRANCH"
```

### Querying

```bash
curl https://proxy.example.com/api/projects # all projects
curl https://proxy.example.com/api/projects/acme/shop/web # one project's package versions
curl 'https://proxy.example.com/api/projects?purl=pkg:npm/lodash' # projects using lodash
curl 'https://proxy.example.com/api/projects?purl=pkg:npm/lodash@4.17.20' # projects using lodash 4.17.20
curl -X DELETE -H "Authorization: Bearer $PROXY_PROJECTS_TOKEN" \
https://proxy.example.com/api/projects/acme/shop/web
```

Reading needs no token, like the rest of `/api`. With `?purl=`, each project carries the `version` it uses and no `component_count`. A delete names the project itself: `DELETE /api/projects/acme/shop/web/sbom` is answered with 400 unless a project `acme/shop/web/sbom` exists.

A project's page in the UI also has a delete button. It asks for the token and sends it to `DELETE /ui/api/projects/{project}`, which works like the API endpoint and also requires the token; the token is kept for the browser session. Like `/ui/api/cache`, it sits under `/ui`, so rules for the UI at an ingress cover it.

### What it shows

The inventory reflects what the projects' SBOMs list, not what was downloaded through the proxy: a project that installs from another registry still shows up, and the proxy may not have cached any of its packages. The outdated marker compares the version a project uses with the latest version the proxy has recorded for the package; a package the proxy has no record of has none. Versions are compared as the SBOM gives them: a version spelled differently from how the proxy caches it, such as Composer's `v6.4.1` for `6.4.1` or a Debian version with or without its epoch, shows as "not cached" on the package page.

Project names and the packages they use are shown on `/ui` and `/api/projects` without authentication. If that is sensitive, restrict both at your ingress or reverse proxy.

## Mirror API

The `/api/mirror` endpoints are disabled by default. Enable them to allow starting mirror jobs via HTTP:
Expand Down
Loading
Loading