Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ A 3-day cooldown means that when `lodash` publishes version `4.18.0`, your build

Resolution order: package override, then ecosystem override, then global default. This lets you set a conservative default and carve out exceptions for packages where you need faster updates. See [docs/configuration.md](docs/configuration.md) for the full config reference.

One instance can offer several cooldown policies through [named profiles](docs/configuration.md#cooldown-profiles). Configure `cooldown.profiles` and point clients at paths such as `/cooldown/3d/npm/` or `/cooldown/7d/cargo/`. Profiles share the package cache; existing URLs keep using the global policy.

## Artifact Scanning

Cooldown only looks at a version's publish timestamp — it never inspects the actual bytes. Artifact scanning closes that gap: when enabled, every artifact is staged into storage and scanned by one or more external services (trivy, ClamAV, Wiz, or anything else that speaks a small HTTP/JSON contract) before it's committed to the cache and served to clients.
Expand Down
7 changes: 7 additions & 0 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,13 @@ health:
# to spot malicious releases before they're pulled into projects.
# Supported durations: "7d" (days), "48h" (hours), "30m" (minutes), "0" (disabled)
cooldown:
# Independent policies selected through /cooldown/{name}/{registry}/.
# profiles:
# 3d:
# default: "3d"
# 7d:
# default: "7d"

# Global default cooldown for all ecosystems
# default: "3d"

Expand Down
31 changes: 31 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1148,6 +1148,37 @@ Uploads are off by default, so a client that can reach the proxy cannot overwrit

`max_age` and `max_size` are independent and can be combined. When both are set, age-based eviction runs first, then size-based eviction trims remaining entries oldest-first.

## Cooldown profiles

Set `cooldown.profiles` to offer several policies from one instance. Clients select a profile through their package manager's registry URL:

```yaml
cooldown:
default: "0"
profiles:
3d:
default: "3d"
7d:
default: "7d"
packages:
"pkg:npm/@example/tool": "0"
```

For the `3d` profile, use these paths after your proxy's base URL:

| Client | Path |
|--------|------|
| npm registry | `/cooldown/3d/npm/` |
| Go `GOPROXY` | `/cooldown/3d/go` |
| Cargo registry | `sparse+https://your-proxy.example/cooldown/3d/cargo/` |
| Bundler RubyGems mirror | `/cooldown/3d/gem` |

The prefix works with all package protocol routes. Cooldown has an effect only for [registries with cooldown support](../README.md#supported-registries). Each profile defines its own `default`, `ecosystems`, `packages` and `package_patterns`; it does not inherit the global cooldown overrides. Existing unprefixed URLs, the UI, APIs and background mirror jobs use the global policy.

Profile names contain lowercase letters, digits, hyphens and underscores, and start with a letter or digit. Profiles must be configured before startup; an unknown profile returns 404. Nested profiles are rejected. Profile paths expose package protocols only, so `/cooldown/3d/ui/` and `/cooldown/3d/api/` return 404.

Database records, artifacts and upstream metadata are shared. Generated download URLs retain the profile prefix, and filtered metadata and its validators are separated by profile. Changing a registry URL does not rewrite artifact URLs already recorded in a lockfile; regenerate those URLs if the selected policy must cover the whole installation.

## Cooldown

The cooldown feature hides package versions published too recently, giving the community time to spot malicious releases before they reach your projects. When a version is within its cooldown period, it's stripped from metadata responses so package managers won't install it.
Expand Down
7 changes: 7 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,9 @@ type Config struct {
// CooldownConfig configures version cooldown periods.
// Versions published more recently than the cooldown are hidden from metadata responses.
type CooldownConfig struct {
// Profiles are independent policies exposed at /cooldown/{name}/.
Profiles map[string]CooldownConfig `json:"profiles,omitempty" yaml:"profiles,omitempty"`

// Default is the global default cooldown (e.g., "3d", "48h", "0" to disable).
Default string `json:"default" yaml:"default"`

Expand Down Expand Up @@ -1665,6 +1668,10 @@ func (c *Config) validateCacheArtifacts() error {
}

func (c *Config) validateComponents() error {
if err := c.Cooldown.ValidateProfiles(); err != nil {
return err
}

if err := validateMetadataRewriteCacheSize(c.MetadataRewriteCacheSize); err != nil {
return err
}
Expand Down
64 changes: 64 additions & 0 deletions internal/config/cooldown_profiles.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
package config

import (
"fmt"
"maps"
"regexp"
"slices"

"github.com/git-pkgs/cooldown"
)

var cooldownProfileName = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]*$`)

// Policy builds the policy for this URL's cooldown settings.
func (c *CooldownConfig) Policy() (*cooldown.Policy, error) {
return cooldown.NewPolicy(&cooldown.Config{
Default: c.Default, Ecosystems: c.Ecosystems, Packages: c.NormalizedPackages(),
}, c.PackagePatterns)
}

// ValidateProfiles checks names and policies before they become routes.
func (c *CooldownConfig) ValidateProfiles() error {
for _, name := range slices.Sorted(maps.Keys(c.Profiles)) {
profile := c.Profiles[name]
if !cooldownProfileName.MatchString(name) {
return fmt.Errorf("invalid cooldown profile name %q: use lowercase letters, digits, hyphens or underscores", name)
}
if len(profile.Profiles) > 0 {
return fmt.Errorf("cooldown.profiles.%s cannot contain profiles", name)
}
if err := profile.validateProfileDurations(); err != nil {
return fmt.Errorf("cooldown.profiles.%s: %w", name, err)
}
if _, err := profile.Policy(); err != nil {
return fmt.Errorf("cooldown.profiles.%s: %w", name, err)
}
}
return nil
}

func (c *CooldownConfig) validateProfileDurations() error {
values := map[string]string{"default": c.Default}
for group, overrides := range map[string]map[string]string{
"ecosystems": c.Ecosystems, "packages": c.Packages, "package_patterns": c.PackagePatterns,
} {
for key, value := range overrides {
values[group+"."+key] = value
}
}
for _, key := range slices.Sorted(maps.Keys(values)) {
value := values[key]
if key == "default" && value == "" {
continue
}
duration, err := cooldown.ParseDuration(value)
if err != nil {
return fmt.Errorf("invalid %s: %w", key, err)
}
if duration < 0 {
return fmt.Errorf("invalid %s: duration must be non-negative", key)
}
}
return nil
}
81 changes: 81 additions & 0 deletions internal/config/cooldown_profiles_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package config

import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)

func TestLoadCooldownProfiles(t *testing.T) {
filename := filepath.Join(t.TempDir(), "config.yaml")
data := `cooldown:
default: 7d
ecosystems:
npm: "0"
profiles:
3d:
default: 3d
packages:
"pkg:pypi/Django": "0"
strict:
default: 14d
package_patterns:
"pkg:npm/@example/*": 1d
`
if err := os.WriteFile(filename, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
cfg, err := Load(filename)
if err != nil {
t.Fatal(err)
}
if err := cfg.Validate(); err != nil {
t.Fatal(err)
}
profile := cfg.Cooldown.Profiles["3d"]
policy, err := profile.Policy()
if err != nil {
t.Fatal(err)
}
if got := policy.For("npm", "pkg:npm/demo"); got != 72*time.Hour {
t.Fatalf("profile inherited global ecosystem override: %v", got)
}
if got := policy.For("pypi", "pkg:pypi/django"); got != 0 {
t.Fatalf("canonical package override = %v", got)
}
strict := cfg.Cooldown.Profiles["strict"]
policy, err = strict.Policy()
if err != nil {
t.Fatal(err)
}
if got := policy.For("npm", "pkg:npm/%40example/tool"); got != 24*time.Hour {
t.Fatalf("profile pattern = %v", got)
}
}

func TestLoadRejectsInvalidCooldownProfiles(t *testing.T) {
for _, tc := range []struct{ name, profile, want string }{
{"path", `"a/b": {default: 3d}`, "profile name"},
{"uppercase", `Strict: {default: 3d}`, "profile name"},
{"empty", `"": {default: 3d}`, "profile name"},
{"duration", `strict: {default: nope}`, "cooldown.profiles.strict"},
{"pattern", `strict: {package_patterns: {"pkg:npm/[": 3d}}`, "cooldown.profiles.strict"},
{"nested", `strict: {profiles: {nested: {default: 3d}}}`, "cannot contain profiles"},
} {
t.Run(tc.name, func(t *testing.T) {
filename := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(filename, []byte("cooldown:\n profiles:\n "+tc.profile+"\n"), 0o600); err != nil {
t.Fatal(err)
}
cfg, err := Load(filename)
if err == nil {
err = cfg.Validate()
}
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("Load error = %v, want %q", err, tc.want)
}
})
}
}
2 changes: 1 addition & 1 deletion internal/handler/download_cooldown.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ type publishIndexEntry struct {

// publishIndex reuses parsed documents by digest, or for a minute without one.
func publishIndex[V any](p *Proxy, key string, load func() metadataResult, build func(metadataResult) (V, error)) (V, error) { //nolint:ireturn // Supports timestamp maps and Helm lookups.
c := &p.publishIndexes
c := &p.sharedProxy().publishIndexes
fresh := func() (V, bool) {
c.mu.Lock()
defer c.mu.Unlock()
Expand Down
61 changes: 34 additions & 27 deletions internal/handler/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ type Proxy struct {
// cooldownWatch holds the rewrites recording which versions cooldown
// holds back; see cooldown_expiry.go.
cooldownWatch cooldownWatch
shared *Proxy
}

// cooldownEcosystems are the ecosystems whose handlers apply cooldown. Others
Expand Down Expand Up @@ -946,18 +947,19 @@ type inflightFetch struct {
// proxy_cache_misses_total and upstream fetch observations is what coalescing
// saved.
func (p *Proxy) coalesceFetch(ctx context.Context, key string, recheck func() (artifacts.Artifact, string, bool), commit func(context.Context) (artifacts.Artifact, string, error)) (*CacheResult, error) {
p.fetchMu.Lock()
if p.inFlight == nil {
p.inFlight = make(map[string]*inflightFetch)
shared := p.sharedProxy()
shared.fetchMu.Lock()
if shared.inFlight == nil {
shared.inFlight = make(map[string]*inflightFetch)
}
f, joined := p.inFlight[key]
f, joined := shared.inFlight[key]
if joined {
f.waiters++
} else {
f = &inflightFetch{done: make(chan struct{})}
p.inFlight[key] = f
shared.inFlight[key] = f
}
p.fetchMu.Unlock()
shared.fetchMu.Unlock()

if !joined {
if streamsCacheMiss(ctx) {
Expand Down Expand Up @@ -985,36 +987,38 @@ func (p *Proxy) coalesceFetch(ctx context.Context, key string, recheck func() (a
// leaveFetch withdraws a waiter that gave up. Once the fetch has finished the
// waiter was counted as holding any unrecorded object, so it releases it.
func (p *Proxy) leaveFetch(f *inflightFetch) {
p.fetchMu.Lock()
shared := p.sharedProxy()
shared.fetchMu.Lock()
select {
case <-f.done:
p.fetchMu.Unlock()
shared.fetchMu.Unlock()
if f.unrecorded != nil {
f.unrecorded.release()
}
default:
f.waiters--
p.fetchMu.Unlock()
shared.fetchMu.Unlock()
}
}

// runSharedFetch performs the fetch that joined callers are waiting on. It is
// never abandoned early, and always releases the key and wakes the waiters.
func (p *Proxy) runSharedFetch(ctx context.Context, key string, f *inflightFetch, recheck func() (artifacts.Artifact, string, bool), commit func(context.Context) (artifacts.Artifact, string, error)) (*CacheResult, error) {
shared := p.sharedProxy()
// Set before running so a panicking commit leaves waiters with an error
// rather than a zero-valued artifact.
f.err = errSharedFetchAbandoned
defer func() {
p.fetchMu.Lock()
delete(p.inFlight, key)
shared.fetchMu.Lock()
delete(shared.inFlight, key)
if f.unrecorded != nil {
// Hand each waiter a hold, then drop the one kept while the
// waiters were still unknown.
f.unrecorded.refs.Add(int32(f.waiters))
defer f.unrecorded.release()
}
close(f.done)
p.fetchMu.Unlock()
shared.fetchMu.Unlock()
}()

// A caller checks the cache before reaching here, so a fetch that finished
Expand Down Expand Up @@ -1162,12 +1166,13 @@ func upstreamFetchDetached(ctx context.Context) bool {
const maxDetachedFetches = 64

func (p *Proxy) acquireDetachSlot() bool {
p.fetchMu.Lock()
if p.detachSlots == nil {
p.detachSlots = make(chan struct{}, maxDetachedFetches)
shared := p.sharedProxy()
shared.fetchMu.Lock()
if shared.detachSlots == nil {
shared.detachSlots = make(chan struct{}, maxDetachedFetches)
}
slots := p.detachSlots
p.fetchMu.Unlock()
slots := shared.detachSlots
shared.fetchMu.Unlock()

select {
case slots <- struct{}{}:
Expand All @@ -1178,7 +1183,8 @@ func (p *Proxy) acquireDetachSlot() bool {
}

func (p *Proxy) releaseDetachSlot() {
<-p.detachSlots
shared := p.sharedProxy()
<-shared.detachSlots
}

// fetchedArtifact is what a shared fetch hands its callers: metadata and a
Expand Down Expand Up @@ -1877,18 +1883,19 @@ func metadataCoalesceKey(ecosystem, cacheKey, upstreamURL, accept, acceptEncodin
// fail when the first of them disconnects. It stays bounded by the HTTP
// client's timeout. Waiters still leave when their own context ends.
func (p *Proxy) coalesceMetadata(ctx context.Context, key string, fetch func(context.Context) metadataResult) (metadataResult, bool) {
p.metaMu.Lock()
if p.inFlightMeta == nil {
p.inFlightMeta = make(map[string]*inflightMetadata)
shared := p.sharedProxy()
shared.metaMu.Lock()
if shared.inFlightMeta == nil {
shared.inFlightMeta = make(map[string]*inflightMetadata)
}
f, joined := p.inFlightMeta[key]
f, joined := shared.inFlightMeta[key]
if joined {
f.waiters++
} else {
f = &inflightMetadata{done: make(chan struct{})}
p.inFlightMeta[key] = f
shared.inFlightMeta[key] = f
}
p.metaMu.Unlock()
shared.metaMu.Unlock()

if joined {
select {
Expand All @@ -1903,9 +1910,9 @@ func (p *Proxy) coalesceMetadata(ctx context.Context, key string, fetch func(con
// rather than an empty body.
f.res = metadataResult{err: errSharedFetchAbandoned}
defer func() {
p.metaMu.Lock()
delete(p.inFlightMeta, key)
p.metaMu.Unlock()
shared.metaMu.Lock()
delete(shared.inFlightMeta, key)
shared.metaMu.Unlock()
close(f.done)
}()
f.res = fetch(context.WithoutCancel(ctx))
Expand Down
Loading
Loading