PixScan

A premium, fully offline hybrid desktop application to extract text (OCR), translate, and analyze images/screenshots from the clipboard using local open-source AI models.

Powered by Electron, React, TailwindCSS, Tesseract.js (WebAssembly OCR), and llama.cpp running Qwen2.5-1.5B-Instruct (~980MB) and Moondream2 1.6B VLM (~3.0GB) with CPU/GPU acceleration on Windows, Linux, and macOS.

---

Refined Hybrid Architecture

To balance high-fidelity OCR, context-aware translations, and advanced image analysis, PixScan uses a hybrid multi-model pipeline:

1. High-Fidelity OCR: Tesseract.js processes images at their native resolution within Node.js. This ensures near 100% accuracy on dense text, code blocks, and specs (avoiding the downsampling limits of VLMs).
2. Context-Aware Translation: Qwen2.5-1.5B-Instruct (via llama-server) translates text while maintaining layout, indentation, and code formatting.
3. Optional Vision Analysis: Swaps the backend process to run Moondream2 (a 1.6B local Vision-Language Model) for scene descriptions, user interface layout cataloging, and custom question answering. This is lazy-loaded to keep the initial setup footprint lightweight and operates with prompt cache disabled (--no-cache-prompt) to ensure clean analysis across different images.

graph TD
    A[Electron App] -->|1. OCR/Translation Tab| B[Tesseract.js + Qwen 1.5B]
    A -->|2. Vision Analyzer Tab| C[Moondream2 VLM]
    B -->|Offline OCR & Translation| D[Plain Text & Layout Code]
    C -->|Offline Visual QA & Description| E[Granular Scene Analysis]
    style B fill:#151d30,stroke:#202e4b,stroke-width:2px
    style C fill:#151d30,stroke:#202e4b,stroke-width:2px

Loading

Read our ADR.md for full architectural details, model swapping techniques, and trade-offs.

---

Key Features

• 100% Offline & Private: All OCR, translation, and image analysis run locally on your device.
• Granular Vision Analyzer Tab: Describe scenes, inspect interface layouts, list items, or ask custom questions offline.
• Lazy-Loaded VLM weights: Prompts users before downloading the additional ~3.0 GB vision weights, keeping the core OCR setup lightweight (~1.0 GB).
• Smooth Model Swapping: Gracefully swaps running models in the background without port conflicts or UI status glitches.
• Clipboard Integration: Capture screenshots and paste directly (Cmd+V / Ctrl+V) for instant conversion.
• Cross-Platform: Designed to run and build on Windows, Linux, and macOS with platform-appropriate binaries and extraction logic.
• GPU & CPU support: Leverages GPU acceleration (Metal on macOS) where available, falling back to clean CPU-only execution on Windows and Linux.

---

Setup & Running Locally

Prerequisites

• Node.js (v20+ recommended)
• Git (for cloning the repository)

Steps

1. Install dependencies:

   npm install

2. Run in development mode:

   npm run dev

   Note: On the first launch, the app will download llama-server (specific to your platform/architecture), the Qwen2.5-1.5B GGUF model, and the eng.traineddata.gz OCR model (~1.0 GB total). These will be saved in the platform's local application data directory (described below).

3. Capture & Process:

   • Take a screenshot.
   • Paste it inside the application (Cmd+V on macOS, Ctrl+V on Windows/Linux).
   • Select "Extract Only" for plain text OCR, or choose a target language for translation.
   • Click Convert Image to process.

---

Packaging & Building Executables

PixScan uses electron-builder to package installers. The built executables will be generated in the dist_electron/ folder.

To create the installer for your current operating system, run:

npm run dist

Building by OS

• macOS:

  npm run dist:mac

  Generates a .dmg installer and a .zip archive inside dist_electron/ for your current Mac architecture (ARM64 / Intel).

• Windows:

  npm run dist:win

  Generates a .exe installer (NSIS setup executable) inside dist_electron/. Note: Compiling Windows packages with native modules is recommended to be run from a Windows host machine.

• Linux:

  npm run dist:linux

  Generates .AppImage and .deb distribution packages inside dist_electron/.

---

Storage Directory

The model files, platform binaries, and configuration data are saved in the Electron userData folder named PixScan:

• macOS: ~/Library/Application Support/PixScan/
• Windows: C:\Users\<username>\AppData\Roaming\PixScan\
• Linux: ~/.config/PixScan/

Directory Structure

Within the storage directory:

• bin/: Platform-specific llama-server executable (and dynamic libraries .dylib/.so / dynamic link libraries .dll)
• models/:
  • qwen2.5-1.5b-instruct-q4_k_m.gguf (980 MB translation model)
  • moondream2-text-model-f16_ct-vicuna.gguf (2.8 GB VLM language weights)
  • moondream2-mmproj-f16-20250414.gguf (240 MB VLM vision projector)
• tessdata/: eng.traineddata.gz (4.1 MB OCR dataset)