Repository navigation
Fix per-feed USGS refresh metadata - #5
Conversation
|
Warning Review limit reached
More reviews will be available in 37 minutes and 45 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7607324d10
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| def feed_meta_value(meta, key, feed, default=''): | ||
| scoped=feed_meta_key(key,feed) | ||
| if scoped in meta: return meta[scoped] | ||
| return meta.get(key, default) if meta.get('last_refresh_source') == feed else default |
There was a problem hiding this comment.
When the app starts with the default ENABLE_BOOTSTRAP_DATA=true, seed_bootstrap_if_empty records the refresh timestamp globally and under bootstrap_demo, not under requested feeds like day. This fallback now discards that global timestamp because last_refresh_source is bootstrap_demo, so if USGS is unavailable every /api/earthquakes or /api/summary request keeps attempting a live fetch until timeout instead of serving the bootstrap/cache under CACHE_TTL_SECONDS as before. Please either seed feed-scoped bootstrap metadata or otherwise throttle fallback attempts.
Useful? React with 👍 / 👎.
| def refresh_if_needed(feed='day', force=False): | ||
| db=current_app.config['SQLITE_DB_PATH']; ttl=current_app.config['CACHE_TTL_SECONDS']; last=int(get_meta(db,'last_successful_refresh_epoch','0') or 0) | ||
| db=current_app.config['SQLITE_DB_PATH']; ttl=current_app.config['CACHE_TTL_SECONDS']; meta=all_meta(db) | ||
| last=int(feed_meta_value(meta,'last_successful_refresh_epoch',feed,'0') or 0) |
There was a problem hiding this comment.
The feed value comes directly from the query string, while feed_url falls back unknown names to the default URL. With this per-feed lookup, requests such as feed=foo1 and feed=foo2 all miss the TTL even though they fetch the same default feed, and upsert_earthquakes then creates separate scoped metadata keys for each arbitrary string. A client or typo can therefore force a live USGS request and unbounded metadata writes on every request; validate/reject unknown feeds or map them to the canonical default before using it as the cache key.
Useful? React with 👍 / 👎.
Motivation
last_successful_refresh_epochandlast_refresh_source, which caused one USGS feed refresh (e.g.day) to incorrectly block refreshes for other feeds (e.g.7day,30day,m4.5).Description
feed_meta_key(k, feed)inapp/services/cache.pyand store feed-scoped metadata keys likelast_successful_refresh_epoch:<feed>,last_refresh_source:<feed>,last_record_count:<feed>, andlast_refresh_error:<feed>inupsert_earthquakeswhile still updating the legacy global keys.feed_meta_value(meta, key, feed, default='')inapp/routes.pyand changedrefresh_if_needed()to consult feed-scoped timestamps first with a legacy global-key fallback only when the legacylast_refresh_sourcematches the requested feed./api/earthquakesand/api/summary) to report per-feedsourceandlast_refresh_epochwhen available.tests/test_app.pyproving that refreshing one feed does not prevent other feeds from refreshing, that legacy global metadata only blocks the matching feed, and that the/api/earthquakes?feed=...endpoint refreshes each feed independently.Testing
python -m py_compile app/routes.py app/services/cache.py tests/test_app.pywhich succeeded.pytest -q, but full test execution failed in this environment becauseFlaskwas not installed andpython -m pip install -r requirements.txtfailed due to a network/package-index error (Tunnel connection failed: 403 Forbidden), preventing dependency installation and complete test runs.Codex Task