FallGuard

Cloud-deployed fall detection from video.

FallGuard is a full-stack computer vision demo. Pose landmarks are extracted on-device in the browser (MediaPipe Tasks Vision, WASM/GPU) and only the extracted head/hip coordinates — not raw video — are sent to a Google Cloud-hosted FastAPI backend, which applies lightweight temporal fall-detection logic over a one-second motion window, with v1.1 recovery confirmation and down-duration tracking. A legacy server-side pose path is also available for reference (see API).

Live Demo

• Frontend: fallguard-frontend
• Backend health check: fallguard-api /health

Author

• Theodore Ioannidis
• Computer Vision / AI Engineer

Features

• In-browser pose landmark extraction (MediaPipe Tasks Vision, WASM/GPU) — raw video never leaves the device
• Google Cloud-hosted FastAPI backend for temporal fall-event analysis
• Motion-based fall-event logic using head and mid-hip tracking
• Persistent alert until recovery is confirmed
• Down-duration tracking after a detected fall
• Real-time status panels for inference and motion metrics
• Per-IP rate limiting on both inference endpoints
• Unit-tested fall/recovery detection logic
• Legacy server-side MediaPipe Pose path kept for reference/comparison
• Homepage demo clip and production-style frontend presentation

Architecture

Browser webcam
  -> MediaPipe Tasks Vision (in-browser pose landmark extraction)
  -> POST /api/v1/process_landmarks  (JSON: head/mid_hip coordinates only)
  -> FastAPI route layer (rate limiting, validation)
  -> Temporal motion analysis
  -> Recovery confirmation and down-duration tracking
  -> JSON response with fall state, duration, joints, and optional metrics
  -> Frontend overlay and UI status updates

Legacy/reference path (raw frame upload, server-side pose extraction):

Browser webcam
  -> JPEG frame
  -> POST /api/v1/process_frame
  -> FastAPI route layer (rate limiting, validation)
  -> MediaPipe Pose extraction (server-side)
  -> same temporal motion analysis / recovery / response as above

Both routes feed the same alert-state and fall/recovery analysis core (fallguard/pipeline/logic.py), which is agnostic to where the joints came from.

API

Endpoint	Body	Used by
POST /api/v1/process_landmarks	JSON: camera_id, time_ms, joints (head, mid_hip coordinates)	Live demo frontend (default)
POST /api/v1/process_frame	multipart: camera_id, time_ms, frame_file (JPEG)	tools/test_client.py, kept as a server-side reference path
GET /health	—	Health check

Tech Stack

• Frontend: React, Vite, React Router, MediaPipe Tasks Vision (WASM)
• Backend: FastAPI, Uvicorn, Pydantic
• CV / ML: MediaPipe Pose (browser + server), OpenCV, NumPy
• Cloud: Google Cloud Run

Repository Structure

.
|- frontend/               React frontend
|- fallguard/              Backend package
|- main.py                 FastAPI app entrypoint
|- Dockerfile              Backend Cloud Run image
|- requirements.txt        Backend dependencies
`- tools/                  Utility scripts

Local Development

Backend

python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
python main.py

Frontend

cd frontend
npm install
npm run dev

Tests

Pure-Python unit tests, no external test framework required:

python -m unittest discover -s fallguard -p "test_*.py" -v

CI (.github/workflows/ci.yml) runs this plus ruff check and the frontend build on every push and pull request against main.

Environment Configuration

Backend

Important environment variables:

• FALLGUARD_CORS_ALLOWED_ORIGINS
• FALLGUARD_MAX_UPLOAD_BYTES
• FALLGUARD_MAX_CAMERA_ID_LENGTH
• FALLGUARD_MAX_ACTIVE_CAMERAS
• FALLGUARD_FRAME_CACHE_WINDOW_MS
• FALLGUARD_TARGET_LOOKBACK_MS
• FALLGUARD_LOOKBACK_TOLERANCE_MS
• FALLGUARD_RECOVERY_LOOKBACK_MS
• FALLGUARD_UPRIGHT_RATIO_THRESHOLD
• FALLGUARD_HEAD_LIFT_VELOCITY_THRESHOLD
• FALLGUARD_HIP_LIFT_VELOCITY_THRESHOLD
• FALLGUARD_RECOVERY_CONFIRMATION_FRAMES
• FALLGUARD_EXPOSE_DEBUG_METRICS
• FALLGUARD_RATE_LIMIT_MAX_REQUESTS
• FALLGUARD_RATE_LIMIT_WINDOW_MS

Frontend

Set:

• VITE_API_URL — full URL of the /api/v1/process_frame endpoint; the frontend derives the /api/v1/process_landmarks URL from it.

Production currently points to the deployed Cloud Run backend.

Deployment

Both services are deployed separately to Google Cloud Run (europe-west4):

• fallguard-api for inference
• fallguard-frontend for the web UI

This repo keeps both services together because they are tightly coupled and easier to understand as a single project.

Auto-deploy: every push to main that passes the backend/frontend CI jobs triggers a deploy job (.github/workflows/ci.yml) that redeploys both services via gcloud run deploy --source. Authentication uses Workload Identity Federation (no stored service account keys) — GitHub's OIDC token is exchanged for short-lived GCP credentials, scoped to a dedicated github-actions-deployer service account that can only deploy Cloud Run services in this project, not touch anything else.

Security and Privacy Notes

• The live demo extracts pose landmarks on-device in the browser; only head/hip coordinates (/api/v1/process_landmarks) are sent to the backend, not raw video.
• /api/v1/process_frame (raw JPEG upload, server-side pose extraction) still exists as a reference/comparison path and is not used by the live demo.
• The current system is suitable for a portfolio demo, not a certified medical or safety product.
• The backend validates upload size / payload shape, bounds session state, applies a per-IP rate limit, and uses per-session browser camera IDs.
• Fall-event image archival code exists in the backend, but it is not currently enabled.
• The public demo backend is publicly invokable on Cloud Run.

Current Limitations

• Single-person tracking only
• No authentication on the public inference endpoints
• Rate limiting is a simple per-process, per-IP fixed window (not distributed, resets on redeploy)
• The in-browser pose model retries on CPU delegate if GPU init fails, but CPU inference is noticeably slower on low-end devices
• Not intended for real-world safety monitoring

Roadmap Ideas

• Multi-person support
• Stronger authentication and abuse protection
• Move the per-frame HTTP POST to a WebSocket stream now that payloads are small
• Analytics dashboard and monitoring
• More robust deployment hardening

License / Use

Standard MIT License. This is a personal project and demo, not intended for commercial use or production deployment without significant enhancements and security hardening.