Skip to content

Nexus OSS Kit

id: nexus · sections: Registries, Images, Cleanup rules, Schedules, Monitoring, Reports, Removed · state: works — plan, clean-up, compaction, list of what was removed.

Image clean-up in Sonatype Nexus Repository OSS over its API. The reason for the kit is one and practical: in the OSS edition image clean-up is bad — not "inconvenient", bad enough that in real life people go there with scripts against the API. That "go there with scripts" is the kit.

This is the only kit whose work cannot be undone. A backup taken wrongly is found out by the next restore check; an image deleted wrongly is found out when a release has to be rolled back. Everything below follows from that.

What Nexus OSS does, and what the kit does about it

Facts about Nexus 3 OSS that the kit is built around; each one changed a decision in the code:

  • A component is a tag, not an image. Two tags on one image are two components with different ids and the same checksum. Deleting "an old tag" can remove the latest of the same image, so the kit groups by digest.
  • Deletion frees nothing. DELETE components/{id} answers 204 and the blob store does not change. The built-in task "Docker — Delete unused manifest and images" frees nothing either. Only "Admin — Compact blob store" gives space back — and on a registry nobody has ever compacted, a large share of the blob store is rubbish from past deletions.
  • Tasks can be run over the API (POST /service/rest/v1/tasks/{id}/run → 204), so the kit can finish the job and say how much came back.
  • Anonymous read is on by default. GET /components without a password returned the full list; DELETE without rights returned 403 — and DELETE of a component that no longer exists returned 404 even without rights.
  • Nexus does not stop you deleting the last image of a repository, so the kit has to.
  • Tag sort order means nothing: real services are tagged with commit hashes, and sort=version returned nonsense. Only time can be used.
  • The size of an image is its layers, not its manifest. A Docker component has exactly one asset — the manifest — and its fileSize is the size of that JSON: a couple of kilobytes for an image of tens of megabytes. The kit reads the manifest through the Docker API for the real size, caches by digest, and says in the hint that the column does not add up to disk usage because shared layers are counted in every image.

What it does

  • Registries: a Nexus and an account that may delete from it. The check asks what the account may do — version, edition (if Pro, the kit says there are built-in policies and it may not be needed), the right to delete, access to the task list — and lists the Docker repositories with how much each takes.
  • Images, grouped by image name: one row per name, showing the newest image with its tags, size, build date, last pull; a click opens every version behind it with a size-over-versions chart. Sortable by size, build date and last pull; "never pulled" sorts as oldest. "pulls noticed" is counted by the kit and is a lower bound — Nexus OSS has no download counter.
  • Cleanup rules: keep N newest (by time, never by tag name), older than N days, unused for N days, a name pattern matched against the full image name (angry.team/*, not angry.team), and protected tags (latest, stable, whatever is listed). "Show what it sees" answers from the form before the rule is saved, creating and deleting nothing; an empty plan explains itself — the pattern matched none of the N images and here is what names look like here, or everything matched was spared.
  • Two buttons, never one: show the plan, then run the cleanup. The plan is two tables, one under the other — what would be deleted (red) and what would be kept (green) — each row with its reason, and the reason column sorts.
  • Clean-up is three steps: delete components, run blobstore.compact, measure the blob store before and after — and "after" is read every five seconds until the number stops falling, because Nexus answers OK when it has accepted the compaction, not when it has finished, and a figure read at that moment is a fraction of the real one. The report says how long the store kept shrinking after the task said it was done.
  • Two numbers in the report, never one: the images weighed (measured while they still existed) and freed (the difference of two blob-store readings). The report explains the gap either way — more, because a compaction releases everything unreferenced in the store, not only what this run deleted; less, because layers shared with images that stayed are still in use.
  • Removed: every tag the kit deleted, with the reason and who asked. "The list rather than a count: 'where did that image go' has to be answerable."
  • Reports: one line per run, in words. "would delete 47 images (50 tags), keeping 60"; "deleted N images (M tags) weighing W; the compaction gave back F"; "the credentials work; N docker repositories, compaction available".
  • Schedules belong to the platform; the useful pair is a clean-up every night and a compaction once a week, because deletion frees nothing by itself. Clean-up and plan are tied to a rule; compaction and check to a registry, since compaction is about the whole blob store.

What it needs

  • Nexus Repository 3 OSS with its REST API and Docker API reachable.
  • HTTPS. "http:// sends the credentials in the clear, and these credentials can delete every image in the registry." A self-signed certificate by pinned fingerprint; no "do not verify" flag.
  • An account that may delete, its password in a platform secret: "a server needs an account: reading a Nexus anonymously often works, deleting never does"; "a password kept here would be the second place it lives."
  • Access to the task list, or "this kit cannot run the blob store compaction — and compaction is the only thing that actually frees disk space."

What it measures

Metric Help
servers_total Registries configured
blobstore_used_ratio How full that filesystem is — the one number that says whether a cleanup is overdue
blobstore_bytes, blobstore_free_bytes, blobstore_total_bytes What the blob store holds; room left where it lives — the filesystem's, not a quota of its own; the two added up
freed_bytes_30d What the compactions gave back here over thirty days, measured rather than predicted
uncompacted_deletions Tags deleted since the last compaction — until one runs, every one of them is still occupying the disk
cleanup_age_seconds Age of the last cleanup that finished — or, where none ever has, how long the registry has gone without one
cleanup_failures Cleanups that stopped part way, over a day
repositories_without_rules Docker repositories with no cleanup rule — the ones that will fill the disk
images_total, images_in_registry Images in the repository / in the whole registry, as the sampler last saw it
images_pushed_24h, images_asked_24h, images_gone_24h, tags_removed_24h Sliding-day counts the kit derives by sampling every repository every five minutes: a new digest is pushed, a moved lastDownloaded is asked for, a vanished digest is gone; only removed is exact, from the kit's own record

A "free" figure that a blob store on object storage cannot give stays empty rather than zero — "absent is not zero". The fill chart offers two scales, 0–100 % and fitted to the data, and says under the chart which one is drawn: on a 140 GB registry a day of pushes moves the line by hundredths of a point.

What it refuses to do, and why

  • Delete the newest image of a name. "the last one is never deleted, whatever a rule says." A rule that could empty a repository eventually will.
  • Save a rule with no condition. "a rule with no condition would match every image in the repository." An edit that removes the last condition is refused with the same words — the editor must not accept what the creator rejects.
  • Point a rule at another repository. The record of what it deleted is attached to its name; "make a second rule instead."
  • Clear out more than half without being told. "Clearing out more than half is a rule that was written wrongly more often than it is a repository that needs emptying — check it on the plan, and if this is really what is wanted, say so explicitly."
  • Continue on a shrunken repository. "the repository holds {now} images and held {last_good} at the last successful pass. Something other than this kit is removing images, or this is not the same repository — either way, deleting more of it is not the next step." Compared with the last successful pass, not the last one.
  • Delete more than 200 tags in one run. "a list that long is not one anybody reads. Narrow the rule, or run it more often."
  • Read 404 as "already gone". "That is also the answer when the account has no right to delete, so it is not being read as 'already gone'." A 404 counts as success only for a component this run saw in its own list a second earlier.
  • Collect refusals. On the first 403 the run stops: "this account may read but not delete. Nothing further was attempted."
  • Run a rule on a schedule without a key — but it says how to do it by hand: "the plan and the cleanup buttons on each rule."
  • Roll back a failure in the middle. Two hundred deletions that fail at the hundredth leave a hundred deleted; the job says how far it got and the next run continues rather than starts over.

A dry run is not a mode here but a separate job kind (plan), and every kit of this platform that has a dry run treats it the same way: forgetting to tick a box must not be how images get deleted.

Known limitations

  • How long a compaction takes is the server's business, not the kit's: the kit waits for the task up to AG_NEXUS_TASK_TIMEOUT (an hour by default) and then for the blob store to settle, and a run that outlives the wait ends with a verdict rather than a guess.
  • Pull counts are a lower bound: two pulls inside one sampling interval are one event, and nothing before the first look is counted.
  • Image sizes are read lazily in batches (sixty at a time); unread sizes show as "not read" and sort last.
  • "Free space" is the filesystem's, not a quota of the blob store: something else filling that disk shows up here too.

Free level

One registry, one rule and two cadences are free. That is a disk that stops filling up by itself, which is the point of the kit: a rule somebody has to remember to press stops a disk filling up once. More registries, rules or cadences, and protected lists, are what a key is for.

Two cadences and not one, for the reason this whole page is about: deleting frees nothing, and only a compaction gives the disk back. So one repository looking after itself is two cadences — a cleanup and a compaction — and a free level with one would have promised something the product disproves on its own screens.

From the manifest: cleanup and schedules are free features; servers: 1, rules: 1, schedules: 2. The Schedules screen says «1 of 2» before anything is added rather than refusing after the form is filled in.

Configuration

AG_NEXUS_MAX_DELETIONS (200), AG_NEXUS_DELETE_PAUSE_MS (50), AG_NEXUS_TIMEOUT (30), AG_NEXUS_TASK_TIMEOUT (3600), AG_NEXUS_REMOVED_KEEP (20000) — in kits/nexus/app/config.py; AG_NEXUS_SAMPLE (300 — seconds between two samples of every repository) — in kits/nexus/app/main.py. The compose file passes only the port and the shared variables.