Repository navigation
perf(api): load only the releases apps.json lists - #1970
Open
oleksandr-nc wants to merge 1 commit into
Open
oleksandr-nc wants to merge 1 commit into
oleksandr-nc wants to merge 1 commit into
Conversation
Related #1951 apps.json loaded every release of every app, with its translations, dependencies and licenses, and then serialised only the releases a server can pick. Now one query reads the version fields of all releases, the rule picks the ids to keep, and only those releases are loaded in full. The rule moves from the serializer into the view, so it lives in one place. On the local copy of production a build takes about 2.5 s instead of 4.1 to 4.6 s, with a peak of about 300 MB instead of 500 MB, and the same releases are listed. Both queries read one snapshot. At PostgreSQL's default isolation level each query sees what was committed when it started, so a release deleted or replaced in between, as a nightly upload does, would be missing from the list. The view runs in a REPEATABLE READ transaction when it starts the transaction itself. A stable release is now listed before a nightly release of the same version. A server that accepts nightlies installs the first release with the highest version, so it gets the stable one. Before, their order changed from one build to the next. The ids go into one IN clause, 5,913 of them today. Django's PostgreSQL backend binds the values on the client, so there is no parameter limit there, and SQLite allows 32,766 parameters by default since version 3.32. Signed-off-by: Oleksander Piskun <oleksandr2088@icloud.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related #1951
Follow-up to #1969, as suggested by @lukasdotcom.
apps.jsonused to load every release with its translations, dependencies and licenses, and only then drop the ones no server can pick.Now one small query reads the version fields of all releases and picks the ids to keep.
Only those releases are loaded in full.
A build takes about 2.5 s instead of 4.5 s, and peak memory goes from about 500 MB to 300 MB (measured locally).
The list has the same releases as before.
Both queries run in one REPEATABLE READ transaction on PostgreSQL.
Otherwise a release replaced between them, like a nightly upload, could be missing from the list.
A stable release is now listed before a nightly release with the same version.
A server that accepts nightlies (daily, git,
--allow-unstable, and also beta since Nextcloud 24) installs the first one it finds, so it now always gets the stable build.Before, their order changed from one build to the next.
🤖 AI (if applicable)