Skip to content

Fix earthquake source filtering and coordinate normalization - #6

Merged
vrajpatell merged 1 commit into
masterfrom
codex/fix-wrong-location-behavior-in-map/globe
Jun 26, 2026
Merged

vrajpatell merged 1 commit into
masterfrom
codex/fix-wrong-location-behavior-in-map/globe

Conversation

@vrajpatell

Copy link
Copy Markdown
Owner

Motivation

  • Map/globe markers sometimes appeared in wrong places because random bootstrap demo data mixed with real cached feeds, feed-specific cache results were not filtered, bootstrap cache could be treated as fresh, and longitude bounds broke at the antimeridian.
  • The app needs a canonical per-source normalization layer so different providers (USGS, BMKG, GeoNet, etc.) do not leak differing coordinate orders into the frontend.
  • The UI must clearly indicate demo/cached/live data, allow source/feed selection, and ensure invalid coordinates never reach Leaflet or globe.gl.

Description

  • Add a canonical normalizer and source adapter framework under app/services/sources/ including normalizer.py, usgs_geojson.py, usgs_fdsn.py, bmkg.py, geonet.py and registry/stubs for emsc, isc, geofon, auspass, plus base.py and registry.py.
  • Extend SQLite schema and cache logic to include source, raw_source, per-source/feed refresh metadata keys, and is_bootstrap, plus upsert_earthquakes() changes to set source-aware metadata; add antimeridian-safe longitude filtering to query_earthquakes().
  • Keep USGS GeoJSON parsing correct ([longitude, latitude, depth]) and implement canonical_event() that validates and normalizes latitude/longitude, rejecting NaN/infinite/out-of-range values before storage.
  • Update API paths: GET /api/earthquakes supports source, feed, include_demo, dateline metadata and warnings; add GET /api/sources and POST /api/refresh; propagate source/feed filtering to /api/summary, /api/regions, /api/timeseries, and /api/forecast.
  • Frontend changes: add source/feed selectors and demo warning banners on map/globe/dashboard/forecast, mark marker popups with Source/Feed, style bootstrap/demo markers differently, and validate/skips bad coordinates before plotting.
  • Config and docs: add runtime env vars (ENABLE_BOOTSTRAP_DATA=false, ALLOW_BOOTSTRAP_AS_FRESH_CACHE=false, ENABLED_SOURCES, SOURCE_* settings), update .env.example, README, and add lightweight source rate/timeout settings.

Testing

  • Added and extended tests that cover coordinate normalizers/adapters, BMKG/GeoNet parsing, antimeridian-safe bounding queries, bootstrap/demo filtering, per-source/feed refresh metadata, and the new GET /api/sources and filtered APIs, all exercised by pytest.
  • Ran pytest and received all tests passing (15 passed).
  • Ran python -m compileall . and confirmed the codebase compiles without syntax errors.

Codex Task

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@vrajpatell, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 10 minutes and 39 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aaedc3f4-6df4-48fc-9e6e-5b76c283bd5c

📥 Commits

Reviewing files that changed from the base of the PR and between c735f00 and 7490918.

📒 Files selected for processing (29)
  • .env.example
  • .python-version
  • README.md
  • app/__init__.py
  • app/config.py
  • app/routes.py
  • app/services/cache.py
  • app/services/sources/__init__.py
  • app/services/sources/auspass.py
  • app/services/sources/base.py
  • app/services/sources/bmkg.py
  • app/services/sources/emsc.py
  • app/services/sources/geofon.py
  • app/services/sources/geonet.py
  • app/services/sources/isc.py
  • app/services/sources/normalizer.py
  • app/services/sources/registry.py
  • app/services/sources/usgs_fdsn.py
  • app/services/sources/usgs_geojson.py
  • app/services/usgs_client.py
  • static/js/dashboard.js
  • static/js/forecast.js
  • static/js/globe.js
  • static/js/map.js
  • templates/dashboard.html
  • templates/forecast.html
  • templates/globe.html
  • templates/map.html
  • tests/test_app.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-wrong-location-behavior-in-map/globe

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vrajpatell
vrajpatell merged commit df81cc1 into master Jun 26, 2026
4 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7490918ebc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@@ -0,0 +1,3 @@
from .usgs_fdsn import USGSFDSNSource
class EMSCSource(USGSFDSNSource):
def __init__(self): super().__init__(); self.name='emsc'; self.display_name='EMSC / SeismicPortal'; self.coverage='Europe/global'; self.attribution='EMSC / SeismicPortal (CC BY 4.0 where applicable)'; self.homepage_url='https://www.seismicportal.eu/'; self.supported_feeds=['latest','day','7day','30day']

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Point EMSC at its own backend

When users select source=emsc, this class still inherits USGSFDSNSource.fetch() unchanged, so it calls the USGS FDSN URL and rewrites returned rows as source='usgs_fdsn'. The subsequent API query filters for source = 'emsc', leaving EMSC-specific requests empty while source=all misattributes/duplicates USGS data instead of ingesting EMSC events. Override the fetch/normalization path or keep the source disabled until it uses the EMSC endpoint.

Useful? React with 👍 / 👎.

Comment thread templates/dashboard.html
@@ -1 +1 @@
{% extends 'base.html' %}{% block title %}Earthquake Analytics Dashboard{% endblock %}{% block head %}<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>{% endblock %}{% block content %}<section class="hero"><div><h1>Earthquake Monitoring Dashboard</h1><p class="muted">Live USGS feeds with resilient SQLite cache, analytics, and honest probabilistic risk scoring.</p></div><div><a class="btn" href="/map">Open Map</a> <a class="btn" href="/forecast">Experimental Forecast</a></div></section><section class="card controls"><div><label>Feed</label><select id="feed"><option value="day">Past day</option><option value="hour">Past hour</option><option value="7day">Past 7 days</option><option value="30day">Past 30 days</option><option value="significant">Significant</option><option value="m2.5">M2.5+</option><option value="m4.5">M4.5+</option></select></div><div><label>Min magnitude</label><input id="magMin" type="number" value="1" step=".1"></div><div><label>Limit</label><input id="limit" type="number" value="800"></div><button id="apply">Apply</button><button id="refresh">Force refresh</button><button id="export">Export CSV</button><div id="source" class="muted"></div></section><section class="grid"><div class="card kpi"><span>Total events</span><br><b id="total">0</b></div><div class="card kpi"><span>Max magnitude</span><br><b id="maxmag">-</b></div><div class="card kpi"><span>Average magnitude</span><br><b id="avgmag">-</b></div><div class="card kpi"><span>Tsunami flags</span><br><b id="tsunami">0</b></div></section><section class="grid"><div class="card"><h3>Hourly trend</h3><canvas id="trend"></canvas></div><div class="card"><h3>Magnitude distribution</h3><canvas id="magChart"></canvas></div><div class="card"><h3>Depth distribution</h3><canvas id="depthChart"></canvas></div><div class="card warning"><h3>Experimental model forecast</h3><div id="forecastCard">Loading...</div><p class="muted">Not deterministic earthquake prediction.</p></div></section><section class="grid"><div class="card"><h3>Top regions</h3><ul id="regions"></ul></div><div class="card"><h3>Significant events</h3><table class="table"><tbody id="events"></tbody></table></div></section>{% endblock %}{% block scripts %}<script src="{{ url_for('static', filename='js/dashboard.js') }}"></script>{% endblock %}
{% extends 'base.html' %}{% block title %}Earthquake Analytics Dashboard{% endblock %}{% block head %}<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>{% endblock %}{% block content %}<section class="hero"><div><h1>Earthquake Monitoring Dashboard</h1><p class="muted">Live USGS feeds with resilient SQLite cache, analytics, and honest probabilistic risk scoring.</p></div><div><a class="btn" href="/map">Open Map</a> <a class="btn" href="/forecast">Experimental Forecast</a></div></section><section class="card controls"><div><label>Source</label><select id="source"><option value="all">All enabled sources</option><option value="usgs">USGS GeoJSON</option><option value="usgs_fdsn">USGS FDSN</option><option value="emsc">EMSC</option><option value="isc">ISC</option><option value="geofon">GFZ/GEOFON</option><option value="bmkg">BMKG</option><option value="geonet">GeoNet</option><option value="auspass">AusPass</option></select></div><div><label>Feed</label><select id="feed"><option value="day">Past day</option><option value="hour">Past hour</option><option value="7day">Past 7 days</option><option value="30day">Past 30 days</option><option value="significant">Significant</option><option value="m2.5">M2.5+</option><option value="m4.5">M4.5+</option></select></div><div><label>Min magnitude</label><input id="magMin" type="number" value="1" step=".1"></div><div><label>Limit</label><input id="limit" type="number" value="800"></div><button id="apply">Apply</button><button id="refresh">Force refresh</button><button id="export">Export CSV</button><div id="source" class="muted"></div></section><section class="grid"><div class="card kpi"><span>Total events</span><br><b id="total">0</b></div><div class="card kpi"><span>Max magnitude</span><br><b id="maxmag">-</b></div><div class="card kpi"><span>Average magnitude</span><br><b id="avgmag">-</b></div><div class="card kpi"><span>Tsunami flags</span><br><b id="tsunami">0</b></div></section><section class="grid"><div class="card"><h3>Hourly trend</h3><canvas id="trend"></canvas></div><div class="card"><h3>Magnitude distribution</h3><canvas id="magChart"></canvas></div><div class="card"><h3>Depth distribution</h3><canvas id="depthChart"></canvas></div><div class="card warning"><h3>Experimental model forecast</h3><div id="forecastCard">Loading...</div><p class="muted">Not deterministic earthquake prediction.</p></div></section><section class="grid"><div class="card"><h3>Top regions</h3><ul id="regions"></ul></div><div class="card"><h3>Significant events</h3><table class="table"><tbody id="events"></tbody></table></div></section>{% endblock %}{% block scripts %}<script src="{{ url_for('static', filename='js/dashboard.js') }}"></script>{% endblock %}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use a distinct id for dashboard status

When the dashboard loads, dashboard.js uses document.getElementById('source') both to read the source selector and to write the status text. Because this line now contains both the <select id="source"> and a later <div id="source">, the lookup returns the select, and the first load replaces its options with status text, breaking the source picker and subsequent refreshes. Give the status element a separate id.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant