Skip to content

Refactor app into modular Flask project + probabilistic forecast, dashboard, map, and tests - #4

Merged
vrajpatell merged 1 commit into
masterfrom
codex/upgrade-earthquake-monitoring-dashboard
Jun 26, 2026
Merged

vrajpatell merged 1 commit into
masterfrom
codex/upgrade-earthquake-monitoring-dashboard

Conversation

@vrajpatell

@vrajpatell vrajpatell commented Jun 26, 2026 •

Copy link
Copy Markdown
Owner

Motivation

  • Make the earthquake dashboard production-ready and Render-deployable by refactoring the monolithic main.py into a modular Flask app with services, safer SQLite handling, and clearer configuration.
  • Add an honest, experimental probabilistic forecasting capability (regional risk scoring / magnitude-likelihood estimation) that never makes deterministic predictions and always surfaces uncertainty and limitations.
  • Improve frontend UX (dashboard, 2D map, 3D globe fallback handling), API coverage, and developer ergonomics (tests, CI, docs, environment examples).

Description

  • Refactored the backend into an application factory and service modules: app/__init__.py, app/config.py, app/routes.py, app/services/{usgs_client,cache,analytics,model}.py, and app/utils.py, while keeping main.py compatible with gunicorn main:app for Render.
  • Implemented a robust SQLite cache with WAL mode, busy timeout, table/index creation, non-destructive upserts by earthquake id, refresh metadata, bootstrap/demo fallback data, and safe query helpers (app/services/cache.py).
  • Added richer JSON APIs and pages: GET /api/earthquakes, /api/summary, /api/regions, /api/timeseries, /api/forecast, POST /api/model/train (admin-token protected), GET /api/model/status, /api/health/deep, and frontend pages /dashboard, /map, /globe, /forecast, /about with modern UX (Chart.js, Leaflet, globe.gl).
  • Built an ML pipeline with a heuristic baseline and optional scikit-learn training saved via joblib, time-aware feature engineering (rolling counts, mags, depths, aftershock heuristics), calibrated outputs (risk score 0–100, probability bucket, confidence), and safe fallbacks when ML deps or data are insufficient (app/services/model.py).
  • Added project/ops improvements: pinned requirements.txt, .env.example, .python-version, render.yaml updates, Procfile compatibility, README.md updates, CHANGELOG.md, and a GitHub Actions workflow for tests.

Testing

  • Ran unit and integration tests with pytest and all tests passed: PYENV_VERSION=3.14.4 pytest -q → 9 passed.
  • Verified bytecode compilation with python -m compileall . which completed successfully.
  • Performed local smoke tests via the Flask test client to confirm key endpoints return successful responses: /health, /api/earthquakes, /api/summary, /api/forecast, /dashboard, /map, /globe, and /forecast all served correctly.
  • Note: pip install -r requirements.txt could not be validated in this environment due to outbound package index access returning 403 Forbidden, but the requirements.txt file is updated and Render will install dependencies during build.

Codex Task

Summary by CodeRabbit

  • New Features

    • Added new dashboard, map, forecast, and about pages for exploring earthquake activity.
    • Introduced a forecast view with risk scoring, contributing factors, and nearby event details.
    • Added improved health and status endpoints for app availability and model state.
  • Bug Fixes

    • Improved data handling, caching, and fallback behavior when live feeds are unavailable.
    • Added safer input validation and clearer limits for filters and queries.
  • Documentation

    • Expanded the README and changelog with updated setup, deployment, and feature information.
  • Tests

    • Added automated test coverage and continuous integration checks.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bc0cbddb-6b67-42e8-af70-534e504de116

📥 Commits

Reviewing files that changed from the base of the PR and between 03b8af3 and cf90fa6.

📒 Files selected for processing (30)
  • .env.example
  • .github/workflows/tests.yml
  • .gitignore
  • .python-version
  • CHANGELOG.md
  • README.md
  • app/__init__.py
  • app/config.py
  • app/routes.py
  • app/services/analytics.py
  • app/services/cache.py
  • app/services/model.py
  • app/services/usgs_client.py
  • app/utils.py
  • main.py
  • render.yaml
  • requirements.txt
  • static/css/app.css
  • static/js/dashboard.js
  • static/js/forecast.js
  • static/js/globe.js
  • static/js/map.js
  • templates/about.html
  • templates/base.html
  • templates/dashboard.html
  • templates/forecast.html
  • templates/globe.html
  • templates/map.html
  • tests/conftest.py
  • tests/test_app.py

📝 Walkthrough

Walkthrough

The PR refactors the Flask app into an application factory with shared services for USGS ingestion, SQLite caching, analytics, and risk scoring. It adds new routes, dashboard/map/forecast pages, tests, deployment settings, and documentation updates.

Changes

QuakeRisk application refactor

Layer / File(s) Summary
Shared config and utilities
app/utils.py, app/config.py, .env.example, .python-version, requirements.txt
Environment-backed config, shared time/parsing/distance helpers, and default runtime files are added for the new app layout.
App factory and startup wiring
app/__init__.py, main.py, render.yaml
The Flask factory, minimal runner, and deployment settings are updated around the new startup path.
USGS feed and cache storage
app/services/usgs_client.py, app/services/cache.py, tests/conftest.py, tests/test_app.py
USGS GeoJSON rows are normalized into SQLite storage with refresh metadata and query helpers, and parsing/upsert tests cover the cache path.
Analytics and risk model
app/services/analytics.py, app/services/model.py
Aggregation helpers and the heuristic plus trained risk scoring pipeline are added in the analytics and model services.
Routes and HTTP endpoints
app/routes.py, tests/test_app.py
Page routes and the earthquake, summary, forecast, model, and health APIs are wired together, with endpoint tests covering the new responses.
Templates and client UI
templates/*, static/css/app.css, static/js/*.js
The shared base template, page templates, styling, and dashboard, map, forecast, and globe scripts are rebuilt around the new UI.
Docs and CI support
README.md, CHANGELOG.md, .github/workflows/tests.yml, .gitignore
The README, changelog, ignore rules, and CI workflow are updated for the reorganized project.

Sequence Diagram(s)

sequenceDiagram
  participant forecast_js as "static/js/forecast.js"
  participant api_forecast as "app.routes.api_forecast"
  participant predict as "app.services.model.predict"
  forecast_js->>api_forecast: GET /api/forecast with lat/lon/radius/hours/min_mag
  api_forecast->>predict: score forecast request
  predict-->>api_forecast: risk payload
  api_forecast-->>forecast_js: JSON response
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

I hopped through code with ears upright,
From USGS streams to forecast light.
The cache now purrs, the pages gleam,
And dashboards dance in rabbit dream. 🐇
Thump! QuakeRisk sings, all calm and bright.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/upgrade-earthquake-monitoring-dashboard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vrajpatell
vrajpatell merged commit 1dfdecf into master Jun 26, 2026
3 of 4 checks passed
@vrajpatell
vrajpatell deleted the codex/upgrade-earthquake-monitoring-dashboard branch June 26, 2026 14:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cf90fa6ff5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/__init__.py
app.config.from_object(config_class)
logging.basicConfig(level=logging.INFO)
init_db(app.config['SQLITE_DB_PATH'])
seed_bootstrap_if_empty(app.config['SQLITE_DB_PATH'], app.config['ENABLE_BOOTSTRAP_DATA'])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fetch live data before seeding demo rows

With the default ENABLE_BOOTSTRAP_DATA=true and an empty DB, this inserts bootstrap rows during app creation; upsert_earthquakes records last_successful_refresh_epoch as now, so the first /api/earthquakes call hits the TTL check in refresh_if_needed and returns cache instead of trying USGS. New deployments can show bootstrap_demo earthquakes for the full CACHE_TTL_SECONDS unless a user forces refresh.

Useful? React with 👍 / 👎.

Comment thread app/routes.py
feeds=current_app.config['FEEDS']; return current_app.config['USGS_FEED_URL'] if name=='custom' else feeds.get(name, feeds.get(current_app.config['DEFAULT_FEED_WINDOW'], current_app.config['USGS_FEED_URL']))
def refresh_if_needed(feed='day', force=False):
db=current_app.config['SQLITE_DB_PATH']; ttl=current_app.config['CACHE_TTL_SECONDS']; last=int(get_meta(db,'last_successful_refresh_epoch','0') or 0)
if not force and int(time.time())-last <= ttl: return 'cache'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Key the refresh TTL by requested feed

After any feed refresh sets last_successful_refresh_epoch, this global TTL check returns cache for every other feed name. For example, visiting the dashboard's default day feed and then selecting 30day within five minutes never fetches all_month.geojson, while query_earthquakes does not filter by source_feed, so the feed selector can show data from the wrong feed until force refresh or TTL expiry.

Useful? React with 👍 / 👎.

Comment thread app/services/cache.py
Comment on lines +25 to +26
for idx,col in [('idx_quakes_time','time_ms'),('idx_quakes_mag','magnitude'),('idx_quakes_lat','latitude'),('idx_quakes_lon','longitude'),('idx_quakes_region','region')]:
c.execute(f'CREATE INDEX IF NOT EXISTS {idx} ON earthquakes({col})')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Migrate old cache tables before adding indexes

On deployments that already have the pre-refactor SQLite table, CREATE TABLE IF NOT EXISTS earthquakes leaves the old schema in place without the new region column. This index creation then raises sqlite3.OperationalError: no such column: region during app startup, so upgrading with an existing cache/persistent disk can fail before serving traffic.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant