diff --git a/ACQUISITION.md b/ACQUISITION.md index e3fd103..391d17a 100644 --- a/ACQUISITION.md +++ b/ACQUISITION.md @@ -1,15 +1,32 @@ # Acquisition Brief — BotScope **Date:** 2026-09-21 -**Status:** Briefing document only. **No acquisition has occurred** by virtue of this file. +**Status:** Briefing document only. **No acquisition has occurred** by virtue of this file. +**No valuation** is stated in this document. -## What the project does +## Problem -Internet-wide bot traffic census / local analyzer: federates public crawler IP panels and optional CDN estimates; local log/session analysis and Qt Observatory. +Operators and researchers lack a transparent, evidence-gated picture of automated Internet traffic versus human-likely traffic. Existing tools often force certainty, conflate a single site’s logs with “the Internet,” or require cloud accounts before a first measurement. -## Problem +BotScope addresses this with: + +- An **Internet-wide census surface** (Global Observatory) that federates public crawler/IP panels and optional CDN estimates +- A **local measurement workstation** (CLI + native Qt Observatory) for authorized logs, sessions, and live capture +- An **UNKNOWN-first** classification posture with provenance badges (OBSERVED / CLASSIFIED / INFERRED) + +## Product surfaces -Operators lack a transparent, evidence-gated picture of automation traffic vs human traffic. +| Surface | How to reach it | Notes | +|---------|-----------------|-------| +| CLI | `botscope ` | Headless analyze, hello, doctor, federation, … | +| First-run | `botscope hello` | Offline demo end-to-end; no GUI/network/API keys | +| Ease of access | `botscope access` / `botscope quickstart` | Install paths, data locations, privacy defaults | +| Desktop Observatory | `botscope` or `botscope gui` | Native Qt / PySide6 — not a website | +| Global Observatory | GUI **Global** page / `botscope federation` | Zero-auth public sources; Radar optional | +| Python API | `from botscope import Analyzer` | Same pipeline as CLI | +| PyPI package | `pip install botscope` / `botscope[gui]` | v2.0.0 | + +Network contribution stays **OFF by default**. Zero-config for local logs: no account, no cloud profile, no API key. ## What is included in a transaction (typical) @@ -24,30 +41,68 @@ Operators lack a transparent, evidence-gated picture of automation traffic vs hu - Operator-published IP range data / Cloudflare Radar data - Third-party dependency source - Buyer cloud accounts or secrets -- Fabricated user/revenue metrics (none claimed) +- Fabricated user/revenue/census metrics (none claimed) ## Maturity -v2.0.0 on PyPI; short git history (≈9 commits). Single human maintainer + Dependabot. - -## Deployment model - -pip install; CLI `botscope`; optional GUI; optional Cloudflare Radar token. +v2.0.0 on PyPI; short git history. Single human maintainer + Dependabot. See `docs/acquisition/EXECUTIVE_SUMMARY.md`. ## Technical differentiation -Evidence-gated classification with UNKNOWN-first posture; federated public panels; optional local GUI Observatory. +- Evidence-gated classification with UNKNOWN-first posture +- Federated public panels for an Internet-wide census story (local KPIs stay dataset-scoped) +- Offline-first first-run (`botscope hello`) and diagnostics (`botscope doctor`) +- Optional local GUI Observatory and live capture extras ## Transferable IP / third-party / limitations -See `docs/acquisition/IP_AUDIT.md`, `TRANSFER_MANIFEST.md`, `BOTSCOPE_DILIGENCE.md`. +See: + +- `docs/acquisition/IP_AUDIT.md` +- `docs/acquisition/TRANSFER_MANIFEST.md` +- `docs/acquisition/BOTSCOPE_DILIGENCE.md` +- `docs/acquisition/DEPENDENCY_AUDIT.md` + +## Demo path (buyer / evaluator) + +Fresh machine, no secrets required for the minimal path: + +```bash +pip install botscope +botscope doctor +botscope hello +botscope hello --keep-session demo.bscope +botscope access +``` + +With GUI extras: + +```bash +pip install 'botscope[gui]' +botscope gui +``` + +From a clone (contributors / diligence): + +```bash +git clone https://github.com/theworker02/botscope.git && cd botscope +python3 -m venv .venv && source .venv/bin/activate +pip install -e '.[dev]' +botscope hello +pytest -q +``` + +Expected: commands exit 0; demo output is labeled **DEMO**; no fabricated Internet-wide rates. Detailed script: `docs/acquisition/BUYER_DEMO.md`. First-hour operator guide: `docs/guides/EASE_OF_ACCESS.md`. ## Handoff / evaluation -See `docs/acquisition/HANDOFF_PLAN.md` and `BUYER_DEMO.md`. +- `docs/acquisition/HANDOFF_PLAN.md` +- `docs/acquisition/BUYER_DEMO.md` +- `docs/acquisition/BUYER_DUE_DILIGENCE_CHECKLIST.md` +- `docs/acquisition/CHANGE_OF_CONTROL_CHECKLIST.md` ## Acquisition contact GitHub [@theworker02](https://github.com/theworker02) · https://github.com/theworker02/botscope -No valuation is stated in this document. +Commercial / license questions: see root `COMMERCIAL.md` and `SUPPORT.md`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e7bc96..ea4141b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ All notable changes to BotScope are documented here. Only features that exist in the repository are listed. +## [Unreleased] + +### Added + +- **`botscope hello`** — offline first-run demo analysis with friendly summary (`--json`, `--keep-session PATH`); no GUI, network, or API keys required +- **`botscope access`** — ease-of-access checklist (install paths, zero-config claims, data locations, GUI, network-off guidance, doctor) +- Richer **`botscope quickstart`** covering install, hello, demo, analyze, GUI, doctor, Global Observatory, and privacy defaults +- **`botscope.onboarding`** helpers module (testable summary / checklist builders) +- Docs: expanded `ACQUISITION.md`, `docs/guides/EASE_OF_ACCESS.md`, FEATURES/README updates for hello/access + ## [2.0.0] — 2026-09-18 Major release: measurement workstation, Global Observatory estimates, bundled ML, and live multi-sensor capture. diff --git a/FEATURES.md b/FEATURES.md index 2d362a3..d168327 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -14,7 +14,7 @@ Labels reflect repository reality for BotScope **v2.0.0**. | Privacy transforms | IMPLEMENTED | `botscope.privacy` | | | Network client (OFF default) | IMPLEMENTED | `botscope.network` | | | Signatures packs | IMPLEMENTED | `botscope.signatures` | | -| CLI | IMPLEMENTED | `botscope.cli` | live/capture/flows/geo/eval/federation/… | +| CLI | IMPLEMENTED | `botscope.cli` | live/capture/flows/geo/eval/federation/hello/access/… | | Desktop Observatory GUI | IMPLEMENTED | `botscope.gui` | Native Qt/PySide6 (not a website) | | Global Observatory | IMPLEMENTED | `botscope.sources`, `gui.global_observatory` | Zero-auth federation; Radar OPTIONAL_AUTH | | Internet estimate headline | IMPLEMENTED | `botscope.estimation.internet` | Opens with ≥2 weighted traffic shares + uncertainty | @@ -35,6 +35,8 @@ Labels reflect repository reality for BotScope **v2.0.0**. | ASN offline table | IMPLEMENTED | `botscope.asn` | Optional local CSV/JSON | | Enrich composer | IMPLEMENTED | `botscope.enrich` | ASN/geo/static + optional rDNS | | Demo corpus | IMPLEMENTED | `botscope.demo` | Synthetic | +| First-run hello | IMPLEMENTED | `botscope.onboarding`, `cli` | `botscope hello` — offline demo summary; `--json` / `--keep-session` | +| Ease-of-access checklist | IMPLEMENTED | `botscope.onboarding`, `cli` | `botscope access` + richer `botscope quickstart` | | Labeled eval fixtures | IMPLEMENTED | `datasets/fixtures` | Mini labeled JSONL | | Provenance inspector | IMPLEMENTED | `botscope.provenance` | | | Quality scorecard | IMPLEMENTED | `botscope.quality` | | diff --git a/README.md b/README.md index 66f36c9..9f570a3 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,34 @@ BotScope is an **Internet-wide bot traffic census**: the Global Observatory fede --- +## Try in 60 seconds + +No GUI, no network, no API keys — just a labeled offline demo: + +```bash +pip install botscope +botscope hello +``` + +Optional: save a session or emit JSON: + +```bash +botscope hello --keep-session hello.bscope +botscope hello --json +``` + +Then explore the checklist and desktop app: + +```bash +botscope access +pip install "botscope[gui]" # if you want the Observatory +botscope gui +``` + +First-hour guide: [`docs/guides/EASE_OF_ACCESS.md`](docs/guides/EASE_OF_ACCESS.md). + +--- + ## Screenshots

@@ -131,6 +159,7 @@ Environment check and synthetic demo: ```bash botscope doctor +botscope hello botscope demo --output demo_analysis.bscope botscope open demo_analysis.bscope ``` @@ -161,7 +190,7 @@ print(result.automation_fraction) print(result.stats.by_category) ``` -More detail: [`docs/guides/QUICKSTART.md`](docs/guides/QUICKSTART.md) · [`docs/guides/INSTALLATION.md`](docs/guides/INSTALLATION.md) +More detail: [`docs/guides/EASE_OF_ACCESS.md`](docs/guides/EASE_OF_ACCESS.md) · [`docs/guides/QUICKSTART.md`](docs/guides/QUICKSTART.md) · [`docs/guides/INSTALLATION.md`](docs/guides/INSTALLATION.md) --- @@ -171,6 +200,8 @@ More detail: [`docs/guides/QUICKSTART.md`](docs/guides/QUICKSTART.md) · [`doc | Command | Purpose | |---------|---------| +| `hello` | Offline first-run demo summary (no GUI/network/API keys) | +| `access` | Ease-of-access checklist (install, data dirs, privacy) | | `doctor` | Environment / policy diagnostics | | `demo` | Analyze bundled synthetic corpus (always labeled DEMO) | | `analyze` | Analyze an authorized log or PCAP | diff --git a/SUPPORT.md b/SUPPORT.md index 3066d17..2a54ffa 100644 --- a/SUPPORT.md +++ b/SUPPORT.md @@ -1,8 +1,10 @@ # Support -- **Docs:** start with [README.md](README.md), [docs/guides/QUICKSTART.md](docs/guides/QUICKSTART.md), and [FEATURES.md](FEATURES.md) +- **Docs:** start with [README.md](README.md), [docs/guides/EASE_OF_ACCESS.md](docs/guides/EASE_OF_ACCESS.md), [docs/guides/QUICKSTART.md](docs/guides/QUICKSTART.md), and [FEATURES.md](FEATURES.md) +- **First run:** `pip install botscope` then `botscope hello` (GUI: `pip install 'botscope[gui]'`) - **Bugs / features:** GitHub Issues (use the templates under `.github/ISSUE_TEMPLATE/`) - **Security:** [SECURITY.md](SECURITY.md) +- **Acquisition:** [ACQUISITION.md](ACQUISITION.md) · [docs/acquisition/](docs/acquisition/) - **Diagnostics:** run `botscope doctor` or `botscope doctor --bundle support.zip` and attach only sanitized output -BotScope is an alpha research/engineering tool. Please label DEMO analyses clearly and do not treat local shares as Internet-wide statistics. +BotScope is a **v2.0.0 beta** research/engineering tool. Please label DEMO analyses clearly. Local Observatory shares stay dataset-scoped; the **Global Observatory** census and any gated Internet headline are separate from a single local log and must not be conflated. diff --git a/docs/README.md b/docs/README.md index 10f436f..c182e0f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,13 +2,14 @@ BotScope documentation mirrors repository reality. Status labels you may see: **IMPLEMENTED**, **PARTIAL**, **EXPERIMENTAL**, **PLANNED**, **RESEARCH**, **NOT IMPLEMENTED**. -Start here if you are new: [guides/QUICKSTART.md](guides/QUICKSTART.md). +Start here if you are new: [guides/EASE_OF_ACCESS.md](guides/EASE_OF_ACCESS.md) or [guides/QUICKSTART.md](guides/QUICKSTART.md). ## Guides | Doc | Description | |-----|-------------| -| [guides/QUICKSTART.md](guides/QUICKSTART.md) | Install → doctor → demo → first analysis | +| [guides/EASE_OF_ACCESS.md](guides/EASE_OF_ACCESS.md) | First-hour path: hello, access, data locations, privacy | +| [guides/QUICKSTART.md](guides/QUICKSTART.md) | Install → doctor → hello/demo → first analysis | | [guides/INSTALLATION.md](guides/INSTALLATION.md) | Extras, venv, verification | | [guides/FAQ.md](guides/FAQ.md) | Common questions | | [guides/TROUBLESHOOTING.md](guides/TROUBLESHOOTING.md) | Failure modes and fixes | @@ -71,6 +72,7 @@ Internal phase notes and checklists live under [development/](development/). Pre - [../README.md](../README.md) — project overview - [../FEATURES.md](../FEATURES.md) — feature matrix +- [../ACQUISITION.md](../ACQUISITION.md) — acquisition briefing - [../CHANGELOG.md](../CHANGELOG.md) — releases - [../CONTRIBUTING.md](../CONTRIBUTING.md) — contribution guidelines - [../SECURITY.md](../SECURITY.md) — vulnerability reporting diff --git a/docs/acquisition/BUYER_DEMO.md b/docs/acquisition/BUYER_DEMO.md index 77136f8..47cbabf 100644 --- a/docs/acquisition/BUYER_DEMO.md +++ b/docs/acquisition/BUYER_DEMO.md @@ -9,6 +9,7 @@ git clone https://github.com/theworker02/botscope.git && cd botscope python3 -m venv .venv && source .venv/bin/activate pip install -e '.[dev]' botscope doctor +botscope hello botscope demo pytest -q ``` diff --git a/docs/guides/EASE_OF_ACCESS.md b/docs/guides/EASE_OF_ACCESS.md new file mode 100644 index 0000000..342545b --- /dev/null +++ b/docs/guides/EASE_OF_ACCESS.md @@ -0,0 +1,192 @@ +# Ease of access — first-hour guide + +Status: **IMPLEMENTED** (matches `botscope hello`, `botscope access`, `botscope quickstart`) + +This guide is for new users and acquisition evaluators who want a measurable result quickly, without accounts, API keys, or network contribution. + +Network contribution stays **OFF by default**. Demo shares are **synthetic** — not Internet-wide census metrics. + +## Goals for the first hour + +1. Install BotScope +2. Produce a labeled demo analysis offline (`botscope hello`) +3. Know where local data lives +4. Optionally open the desktop Observatory or analyze an authorized log +5. Know how to keep network paths disabled + +## 0–5 minutes: install + +**CLI / library only:** + +```bash +pip install botscope +``` + +**With native desktop Observatory (Qt):** + +```bash +pip install 'botscope[gui]' +``` + +**All optional extras** (capture, network client, ML, analytics, GUI, dev): + +```bash +pip install 'botscope[all]' +``` + +From a git clone (contributors): + +```bash +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -e '.[gui,dev]' +``` + +Verify the environment: + +```bash +botscope doctor +``` + +## 5–10 minutes: first result (`hello`) + +```bash +botscope hello +``` + +What this does: + +- Runs the bundled synthetic demo corpus end-to-end +- Works **without GUI**, **without network**, and **without API keys** +- Prints event count, automation / human / unknown fractions, top categories, and next steps +- Always labels output as **DEMO DATA** + +Useful options: + +```bash +botscope hello --json +botscope hello --keep-session hello.bscope +``` + +`--keep-session` writes a local `.bscope` folder you can `botscope open` or load in the GUI. + +Print the same path any time: + +```bash +botscope quickstart +botscope access +``` + +## 10–20 minutes: understand the surfaces + +| Command | Purpose | +|---------|---------| +| `botscope hello` | Offline first-run demo summary | +| `botscope access` | Ease-of-access checklist (install, data dirs, privacy) | +| `botscope quickstart` | Richer install → hello → demo → analyze → GUI → doctor → Global | +| `botscope demo` | Full demo session to a `.bscope` path | +| `botscope analyze PATH` | Authorized log / PCAP analysis | +| `botscope gui` / `botscope` | Native Qt Observatory | +| `botscope doctor` | Environment / policy diagnostics | +| `botscope federation` | Global Observatory public-source snapshot | + +## Where data lives + +All of the following are **local** unless you explicitly enable network features: + +| Kind | Location | +|------|----------| +| Config (e.g. network.json) | platformdirs user config for `botscope` | +| HTTP source cache | platformdirs user cache for `botscope` | +| UX settings / recents | `~/.config/botscope` (or `%LOCALAPPDATA%\BotScope` on Windows); override with `BOTSCOPE_UX_DIR` | +| Analysis sessions | Paths you pass (`--output`, `--keep-session`) — `.bscope` folders on disk you control | + +Print resolved paths: + +```bash +botscope access +# or machine-readable: +botscope access --json +``` + +## Desktop Observatory + +```bash +botscope gui +# or simply: +botscope +``` + +- **Demo** loads the synthetic corpus (DEMO DATA banner) +- **Open** / drag-and-drop for authorized logs, PCAPs, or `.bscope` sessions +- **Global** federates zero-auth public sources; Cloudflare Radar token in Settings is optional and never required for local analysis + +GUI map: [../GUI.md](../GUI.md). + +## Analyze your own authorized traffic + +Only analyze systems and traffic you own or have permission to measure. + +```bash +botscope analyze /path/to/access.log --output my.bscope +botscope query my.bscope --expr 'classification eq "AI CRAWLER"' +botscope report my.bscope --format markdown --output report.md +``` + +Python: + +```python +from botscope import Analyzer + +result = Analyzer().analyze("access.log") +print(result.automation_fraction) +print(result.stats.by_category) +``` + +## Privacy defaults + +- Network **contribution**: OFF (`enabled=false` in local network config) +- Local analysis: no cloud profile required +- Query redaction: ON by default in Observatory settings +- IP hashing / truncation: OFF until you enable them +- Optional Cloudflare Radar token: stored locally; leave empty for fully offline local work + +## How to disable network forever (practical) + +1. Do **not** install the `[network]` extra unless you need the opt-in client +2. Leave contribution disabled (default) — never flip `enabled` in network config +3. Leave the Cloudflare Radar token empty +4. Set `BOTSCOPE_NO_IDENTITY_RANGES=1` to skip published crawler-range fetches during analyze +5. Prefer `botscope hello` / offline demo paths for evaluation without egress +6. Run `botscope doctor` and confirm contribution / policy checks stay OFF + +`botscope hello` already uses identity ranges disabled so the first-run path does not attempt range fetches. + +## Global Observatory (optional in the first hour) + +The product’s Internet-wide census story lives on the GUI **Global** page and `botscope federation`. Public panels are zero-auth; CDN estimates from Cloudflare Radar are optional. Local KPIs remain dataset-scoped — do not treat a single log’s automation fraction as a worldwide rate. + +## Buyer / diligence path + +Acquisition briefing: [../../ACQUISITION.md](../../ACQUISITION.md) +Buyer demo script: [../acquisition/BUYER_DEMO.md](../acquisition/BUYER_DEMO.md) + +Minimal evaluator loop: + +```bash +pip install botscope +botscope doctor +botscope hello +botscope access +``` + +## Next reading + +| Goal | Doc | +|------|-----| +| Install extras | [INSTALLATION.md](INSTALLATION.md) | +| Short quickstart | [QUICKSTART.md](QUICKSTART.md) | +| Feature matrix | [../../FEATURES.md](../../FEATURES.md) | +| Query language | [../QUERY.md](../QUERY.md) | +| Limitations | [../research/LIMITATIONS.md](../research/LIMITATIONS.md) | +| Troubleshooting | [TROUBLESHOOTING.md](TROUBLESHOOTING.md) | diff --git a/docs/guides/FAQ.md b/docs/guides/FAQ.md index 3bb6770..5aa7a1e 100644 --- a/docs/guides/FAQ.md +++ b/docs/guides/FAQ.md @@ -2,10 +2,34 @@ **Is network upload on by default?** No. Contribution is OFF by default. -**Can BotScope estimate global bot traffic?** No. Local shares only; Internet-wide estimation is intentionally refused. +**Can BotScope estimate global bot traffic?** +It depends which surface you mean: -**What does DEMO mean?** Synthetic/example data. Never publish demo outputs as observational results. +- **Local Observatory / a single log or PCAP:** reports **dataset-scoped** shares only. Do not treat one sensor as the whole Internet. +- **Global Observatory:** builds an **Internet-wide census** from federated zero-auth public sources (crawler IP panels, crawl catalogs, and similar). Optional Cloudflare Radar CDN estimates require an explicit token and stay off until configured. +- **Gated Internet headline:** may open only when reliability rules are met (for example ≥2 weighted traffic shares with uncertainty). Otherwise BotScope keeps an honest **UNKNOWN** / insufficient-evidence posture rather than inventing a global percentage. + +See `docs/research/GLOBAL_ESTIMATION.md` and the Global Observatory UI. + +**What does DEMO mean?** Synthetic/example data. Never publish demo outputs as observational results. The GUI shows a DEMO DATA banner on the bundled corpus. **Why so many UNKNOWN labels?** Preferring UNKNOWN over forced certainty is a design goal. **Is confidence a probability?** No — it is a heuristic score. + +**Do I need an account or API key?** +No for local log/PCAP analysis and the bundled demo. Global federation uses public sources without an account; Radar is optional. + +**How do I try BotScope in under a minute?** + +```bash +pip install botscope +botscope hello +# GUI: +pip install 'botscope[gui]' +botscope +``` + +**Where does data live?** Locally (sessions, preferences). Network contribution defaults OFF. + +**Acquisition / commercial?** See [`ACQUISITION.md`](../../ACQUISITION.md) and [`COMMERCIAL.md`](../../COMMERCIAL.md). diff --git a/docs/guides/QUICKSTART.md b/docs/guides/QUICKSTART.md index cb5b1dc..13a3dfb 100644 --- a/docs/guides/QUICKSTART.md +++ b/docs/guides/QUICKSTART.md @@ -17,16 +17,17 @@ pip install -e ".[gui,dev]" See [INSTALLATION.md](INSTALLATION.md) for optional extras (`capture`, `network`, `ml`, `analytics`, `all`). -## 2. Doctor + demo +## 2. Doctor + hello + demo ```bash botscope doctor +botscope hello botscope demo --output demo_analysis.bscope botscope open demo_analysis.bscope botscope report demo_analysis.bscope --format markdown --output report.md ``` -The demo corpus is always labeled **DEMO**. Treat its shares as synthetic, not Internet-wide statistics. +The demo corpus is always labeled **DEMO**. Treat its shares as synthetic, not Internet-wide statistics. Ease-of-access checklist: `botscope access`. First-hour guide: [EASE_OF_ACCESS.md](EASE_OF_ACCESS.md). ## 3. Desktop Observatory diff --git a/docs/research/LIMITATIONS.md b/docs/research/LIMITATIONS.md index 4df4258..1ed4dfd 100644 --- a/docs/research/LIMITATIONS.md +++ b/docs/research/LIMITATIONS.md @@ -2,9 +2,12 @@ Status: IMPLEMENTED (documented constraints) -- Single-site/sensor corpora do not imply Internet-wide prevalence -- User-Agent strings are spoofable; identity verification is incomplete in v0.1 -- Confidence is not a calibrated probability -- PCAP/live capture are not fully implemented -- Demo data must not be published as observational science -- Optional enrichment network lookups are OFF by default / PARTIAL +- Single-site/sensor corpora do not imply Internet-wide prevalence. Local Observatory KPIs stay **dataset-scoped**; Global Observatory census views and any gated Internet headline are separate surfaces with their own evidence rules. +- User-Agent strings are spoofable; verified identity ranges reduce but do not eliminate spoofing. +- Confidence is not a calibrated probability (unless an explicit calibration report with labels is produced). +- Offline PCAP ingest and authorized live capture **are implemented** but remain capability-gated: classic pcap via stdlib; pcapng via optional `scapy`; live sniff needs `botscope[capture]`, explicit authorization, and OS permissions. Treat edge cases as PARTIAL, not “missing entirely.” +- Demo / synthetic data must not be published as observational science (DEMO DATA banner / `--demo`). +- Optional enrichment network lookups and Cloudflare Radar are **OFF by default**. +- Network contribution to any shared panel is **OFF by default**. +- Federated public source panels can be stale, incomplete, or operator-specific; provenance badges matter. +- ML classifier paths never override verified identity and are not a prevalence claim. diff --git a/src/botscope/capabilities/registry.py b/src/botscope/capabilities/registry.py index 20f9cb0..ab64889 100644 --- a/src/botscope/capabilities/registry.py +++ b/src/botscope/capabilities/registry.py @@ -86,6 +86,22 @@ def to_dict(self) -> dict[str, Any]: FeatureCapability("manifests", "Measurement manifests", FeatureStatus.IMPLEMENTED, "botscope.manifests", "A2", "Reproducibility metadata"), FeatureCapability("notebook", "Analysis notebook", FeatureStatus.EXPERIMENTAL, "botscope.notebook", "A2", "Lightweight cells + load/filter"), FeatureCapability("demo", "Bundled demo corpus", FeatureStatus.IMPLEMENTED, "botscope.demo", "A2", "Synthetic access log"), + FeatureCapability( + "onboarding-hello", + "First-run hello command", + FeatureStatus.IMPLEMENTED, + "botscope.onboarding", + "Shared", + "Offline demo summary via botscope hello", + ), + FeatureCapability( + "onboarding-access", + "Ease-of-access checklist", + FeatureStatus.IMPLEMENTED, + "botscope.onboarding", + "Shared", + "botscope access + quickstart helpers", + ), FeatureCapability("capture-helpers", "Capture plan/status", FeatureStatus.IMPLEMENTED, "botscope.capture", "A2", "No unauthorized probing"), FeatureCapability("enrich", "Local enrichment tags", FeatureStatus.IMPLEMENTED, "botscope.enrich", "A2", "ASN/geo composer + optional rDNS"), FeatureCapability("behavior", "Behavior profiles", FeatureStatus.IMPLEMENTED, "botscope.behavior", "A2", "Per-source features, entropy, bot-like score"), diff --git a/src/botscope/cli/main.py b/src/botscope/cli/main.py index 1b72bbe..ff960a2 100644 --- a/src/botscope/cli/main.py +++ b/src/botscope/cli/main.py @@ -832,30 +832,46 @@ def gui_cmd(no_welcome: bool) -> None: launch_gui(show_welcome=not no_welcome) -@main.command("quickstart") -def quickstart_cmd() -> None: - """Print the fastest path from install to a first measurement.""" - console.print( - """ -[bold]BotScope quickstart[/bold] +@main.command("hello") +@click.option("--json", "as_json", is_flag=True, help="Emit machine-readable JSON.") +@click.option( + "--keep-session", + type=click.Path(path_type=Path), + default=None, + help="Optional path to save a .bscope session from the demo run.", +) +def hello_cmd(as_json: bool, keep_session: Path | None) -> None: + """Run the bundled demo corpus offline and print a friendly first-run summary. + + Works without GUI, network, or API keys. Always labeled DEMO DATA. + """ + from botscope.onboarding import format_hello_summary, run_hello_analysis - 1. Install GUI extras: - [cyan]pip install 'botscope[gui]'[/cyan] + summary = run_hello_analysis(keep_session=keep_session) + if as_json: + console.print_json(data=summary) + return + console.print(format_hello_summary(summary)) - 2. Launch Observatory (native desktop): - [cyan]botscope[/cyan] - or [cyan]botscope gui --no-welcome[/cyan] - 3. Or analyze from the CLI: - [cyan]botscope demo[/cyan] - [cyan]botscope analyze path/to/access.log --output run.bscope[/cyan] - [cyan]botscope batch logs/*.log --output-dir batch_out[/cyan] +@main.command("access") +@click.option("--json", "as_json", is_flag=True, help="Emit machine-readable JSON.") +def access_cmd(as_json: bool) -> None: + """Print an ease-of-access checklist (install, hello, data locations, privacy).""" + from botscope.onboarding import access_checklist, format_access_checklist - 4. Drag a log, PCAP, or .bscope folder onto the Observatory window. + if as_json: + console.print_json(data={"checklist": access_checklist()}) + return + console.print(format_access_checklist()) -Tips: Settings → theme / privacy stay local. Network contribution defaults OFF. -""".strip() - ) + +@main.command("quickstart") +def quickstart_cmd() -> None: + """Print the fastest path from install to a first measurement.""" + from botscope.onboarding import quickstart_guide + + console.print(quickstart_guide()) @main.command("batch") diff --git a/src/botscope/onboarding.py b/src/botscope/onboarding.py new file mode 100644 index 0000000..a9c52a3 --- /dev/null +++ b/src/botscope/onboarding.py @@ -0,0 +1,268 @@ +"""First-run onboarding and ease-of-access helpers. + +Status: IMPLEMENTED — offline-first; no network, GUI, or API keys required. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from botscope.demo import DEMO_NOTICE + +# Friendly next steps shown after `botscope hello`. +HELLO_NEXT_STEPS: tuple[str, ...] = ( + "Open the desktop Observatory: botscope gui", + "Re-run the labeled demo session: botscope demo --output demo_analysis.bscope", + "Analyze an authorized log: botscope analyze path/to/access.log --output run.bscope", + "Check the environment: botscope doctor", + "Print the full access checklist: botscope access", +) + + +def format_fraction(value: float | None, *, digits: int = 1) -> str: + """Render a 0–1 fraction as a percentage string, or 'n/a'.""" + if value is None: + return "n/a" + return f"{100.0 * value:.{digits}f}%" + + +def _escape_rich(text: str) -> str: + """Escape square brackets so Rich does not treat pip extras as markup tags.""" + return text.replace("[", "\\[") + + +def top_categories( + by_category: dict[str, int], + *, + limit: int = 5, +) -> list[tuple[str, int]]: + """Return the top categories by event count (descending).""" + items = sorted(by_category.items(), key=lambda kv: (-kv[1], kv[0])) + return items[: max(0, limit)] + + +def data_locations() -> dict[str, str]: + """Describe where BotScope keeps local state (no uploads by default).""" + from platformdirs import user_cache_dir, user_config_dir + + from botscope.ux.recents import settings_path + + return { + "config": str(Path(user_config_dir("botscope", "botscope"))), + "cache": str(Path(user_cache_dir("botscope", "botscope"))), + "ux_settings": str(settings_path()), + "sessions": ( + "Wherever you choose — .bscope session folders are written to the " + "--output / --keep-session path you pass (local disk only)." + ), + } + + +def run_hello_analysis(*, keep_session: Path | None = None) -> dict[str, Any]: + """Run the bundled demo corpus end-to-end offline and return a summary dict. + + Uses ``use_identity_ranges=False`` so no network fetch is attempted. + No GUI and no API keys are required. + """ + from botscope.api.analyzer import Analyzer + from botscope.demo import ensure_demo_log + + log_path = ensure_demo_log() + result = Analyzer(use_identity_ranges=False).analyze( + log_path, + output=keep_session, + is_demo=True, + ) + by_category = dict(result.stats.by_category) + tops = top_categories(by_category, limit=5) + return { + "notice": DEMO_NOTICE, + "is_demo": True, + "offline": True, + "events": len(result.events), + "automation_fraction": result.automation_fraction, + "human_fraction": result.human_fraction, + "unknown_fraction": result.unknown_fraction, + "top_categories": [{"category": cat, "count": count} for cat, count in tops], + "by_category": by_category, + "session_path": str(result.session_path) if result.session_path else None, + "next_steps": list(HELLO_NEXT_STEPS), + } + + +def format_hello_summary(summary: dict[str, Any]) -> str: + """Pretty multi-line summary for the terminal (Rich markup allowed).""" + auto = format_fraction(summary.get("automation_fraction")) + human = format_fraction(summary.get("human_fraction")) + unknown = format_fraction(summary.get("unknown_fraction")) + lines = [ + "[bold]BotScope hello[/bold] — offline demo analysis complete", + "", + f"[yellow]{summary.get('notice', DEMO_NOTICE)}[/yellow]", + "", + f" Events analyzed: [cyan]{summary.get('events', 0):,}[/cyan]", + f" Automation fraction: [cyan]{auto}[/cyan]", + f" Human-likely: [cyan]{human}[/cyan]", + f" Unknown: [cyan]{unknown}[/cyan]", + "", + " Top categories:", + ] + tops = summary.get("top_categories") or [] + if not tops: + lines.append(" (none)") + else: + for row in tops: + cat = row.get("category", "?") + count = row.get("count", 0) + lines.append(f" • {cat}: {count:,}") + session = summary.get("session_path") + if session: + lines.extend(["", f" Session saved: [green]{session}[/green]"]) + lines.extend(["", " Next steps:"]) + for step in summary.get("next_steps") or HELLO_NEXT_STEPS: + lines.append(f" → {step}") + lines.append("") + lines.append( + "[dim]Network contribution stays OFF by default. " + "Demo shares are synthetic — not Internet-wide census metrics.[/dim]" + ) + return "\n".join(lines) + + +def access_checklist() -> list[dict[str, str]]: + """Structured ease-of-access checklist items.""" + locs = data_locations() + return [ + { + "id": "install_core", + "title": "Install (CLI / library)", + "detail": "pip install botscope", + }, + { + "id": "install_gui", + "title": "Install with desktop Observatory", + "detail": "pip install 'botscope[gui]'", + }, + { + "id": "install_all", + "title": "Install all optional extras", + "detail": "pip install 'botscope[all]'", + }, + { + "id": "zero_config", + "title": "Zero-config local analysis", + "detail": ( + "No account, no cloud profile, and no API key required to classify " + "authorized logs offline." + ), + }, + { + "id": "hello", + "title": "First result in one command", + "detail": "botscope hello # offline demo; optional --json / --keep-session PATH", + }, + { + "id": "data_locations", + "title": "Where data lives", + "detail": ( + f"Config: {locs['config']} · Cache: {locs['cache']} · " + f"UX settings: {locs['ux_settings']} · Sessions: {locs['sessions']}" + ), + }, + { + "id": "open_gui", + "title": "Open the GUI", + "detail": "botscope gui # or: botscope (native Qt Observatory, not a website)", + }, + { + "id": "disable_network", + "title": "Keep / force network off", + "detail": ( + "Contribution defaults OFF (botscope/network.json enabled=false). " + "Skip the [network] extra; set BOTSCOPE_NO_IDENTITY_RANGES=1 to skip " + "published crawler-range fetches; leave Cloudflare Radar token empty." + ), + }, + { + "id": "privacy_defaults", + "title": "Privacy defaults", + "detail": ( + "Query redaction ON; IP hashing/truncation OFF until you enable them " + "in Settings. Analysis stays on your machine." + ), + }, + { + "id": "doctor", + "title": "Environment check", + "detail": "botscope doctor # optional: --json or --bundle PATH", + }, + { + "id": "docs", + "title": "First-hour guide", + "detail": "docs/guides/EASE_OF_ACCESS.md · botscope quickstart", + }, + ] + + +def format_access_checklist() -> str: + """Rich multi-line access checklist for the terminal.""" + lines = [ + "[bold]BotScope — ease of access checklist[/bold]", + "", + "Zero-friction path: install → [cyan]botscope hello[/cyan] → optional GUI.", + "Network contribution is [green]OFF by default[/green]. No fabricated census metrics.", + "", + ] + for i, item in enumerate(access_checklist(), start=1): + lines.append(f" [bold]{i}. {_escape_rich(item['title'])}[/bold]") + lines.append(f" {_escape_rich(item['detail'])}") + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +def quickstart_guide() -> str: + """Rich multi-line quickstart covering install through Global Observatory.""" + return """ +[bold]BotScope quickstart[/bold] + + [bold]1. Install paths[/bold] + Core CLI: [cyan]pip install botscope[/cyan] + With GUI: [cyan]pip install 'botscope\\[gui]'[/cyan] + All extras: [cyan]pip install 'botscope\\[all]'[/cyan] + From source: [cyan]pip install -e '.\\[gui,dev]'[/cyan] + + [bold]2. Hello (fastest first result — offline)[/bold] + [cyan]botscope hello[/cyan] + [cyan]botscope hello --json[/cyan] + [cyan]botscope hello --keep-session hello.bscope[/cyan] + + [bold]3. Demo session + report[/bold] + [cyan]botscope demo --output demo_analysis.bscope[/cyan] + [cyan]botscope open demo_analysis.bscope[/cyan] + [cyan]botscope report demo_analysis.bscope --format markdown --output report.md[/cyan] + + [bold]4. Analyze your authorized log[/bold] + [cyan]botscope analyze path/to/access.log --output run.bscope[/cyan] + [cyan]botscope batch logs/*.log --output-dir batch_out[/cyan] + + [bold]5. Desktop Observatory (native Qt — not a website)[/bold] + [cyan]botscope[/cyan] + or [cyan]botscope gui[/cyan] + Drag a log, PCAP, or .bscope folder onto the window. + + [bold]6. Doctor[/bold] + [cyan]botscope doctor[/cyan] + + [bold]7. Global Observatory[/bold] + In the GUI: open the [cyan]Global[/cyan] page (zero-auth public panels). + CLI snapshot: [cyan]botscope federation[/cyan] + Cloudflare Radar CDN estimates need an optional token in Settings — never required. + + [bold]8. Privacy defaults[/bold] + Network contribution: OFF · analysis local · query redaction ON by default. + Ease-of-access checklist: [cyan]botscope access[/cyan] + Guide: docs/guides/EASE_OF_ACCESS.md + +Tips: Settings → theme / privacy stay local. Demo shares are synthetic, not census metrics. +""".strip() diff --git a/tests/cli/test_cli.py b/tests/cli/test_cli.py index 80c1c38..c5c3a24 100644 --- a/tests/cli/test_cli.py +++ b/tests/cli/test_cli.py @@ -36,8 +36,35 @@ def test_cli_quickstart(): runner = CliRunner() r = runner.invoke(main, ["quickstart"]) assert r.exit_code == 0, r.output + assert "botscope hello" in r.output assert "botscope demo" in r.output - assert "batch" in r.output.lower() + assert "batch" in r.output.lower() or "analyze" in r.output.lower() + assert "doctor" in r.output.lower() + assert "privacy" in r.output.lower() + + +def test_cli_hello(tmp_path): + runner = CliRunner() + out = tmp_path / "hello.bscope" + r = runner.invoke(main, ["hello", "--keep-session", str(out), "--json"]) + assert r.exit_code == 0, r.output + assert "is_demo" in r.output + assert "events" in r.output + assert "top_categories" in r.output + r2 = runner.invoke(main, ["hello"]) + assert r2.exit_code == 0, r2.output + assert "BotScope hello" in r2.output or "Events analyzed" in r2.output + + +def test_cli_access(): + runner = CliRunner() + r = runner.invoke(main, ["access"]) + assert r.exit_code == 0, r.output + assert "ease of access" in r.output.lower() or "checklist" in r.output.lower() + assert "botscope hello" in r.output + r2 = runner.invoke(main, ["access", "--json"]) + assert r2.exit_code == 0, r2.output + assert "checklist" in r2.output def test_cli_batch(tmp_path): diff --git a/tests/test_onboarding.py b/tests/test_onboarding.py new file mode 100644 index 0000000..9c2f91c --- /dev/null +++ b/tests/test_onboarding.py @@ -0,0 +1,96 @@ +"""Tests for first-run onboarding helpers.""" + +from __future__ import annotations + +from pathlib import Path + +from botscope.demo import DEMO_NOTICE +from botscope.onboarding import ( + HELLO_NEXT_STEPS, + access_checklist, + format_access_checklist, + format_fraction, + format_hello_summary, + quickstart_guide, + run_hello_analysis, + top_categories, +) + + +def test_format_fraction(): + assert format_fraction(None) == "n/a" + assert format_fraction(0.5) == "50.0%" + assert format_fraction(0.1234, digits=2) == "12.34%" + + +def test_top_categories_ordering_and_limit(): + by_cat = {"B": 2, "A": 5, "C": 5, "D": 1} + tops = top_categories(by_cat, limit=3) + assert tops == [("A", 5), ("C", 5), ("B", 2)] + assert top_categories({}, limit=5) == [] + assert top_categories(by_cat, limit=0) == [] + + +def test_access_checklist_covers_key_paths(): + items = access_checklist() + ids = {item["id"] for item in items} + assert { + "install_core", + "install_gui", + "hello", + "zero_config", + "data_locations", + "open_gui", + "disable_network", + "doctor", + }.issubset(ids) + assert "pip install 'botscope[gui]'" in ( + next(i["detail"] for i in items if i["id"] == "install_gui") + ) + text = format_access_checklist() + assert "botscope hello" in text + assert "OFF by default" in text + assert "pip install botscope" in text + # Rich escapes must still render extras literally when printed + assert "botscope\\[gui]" in text or "botscope[gui]" in text + + +def test_quickstart_guide_richer_surface(): + text = quickstart_guide() + for needle in ( + "botscope hello", + "pip install", + "botscope demo", + "botscope analyze", + "botscope gui", + "botscope doctor", + "Global Observatory", + "Privacy defaults", + "botscope access", + ): + assert needle in text + + +def test_run_hello_analysis_offline(tmp_path: Path): + session = tmp_path / "hello.bscope" + summary = run_hello_analysis(keep_session=session) + assert summary["is_demo"] is True + assert summary["offline"] is True + assert summary["notice"] == DEMO_NOTICE + assert summary["events"] > 0 + assert isinstance(summary["automation_fraction"], float) + assert summary["top_categories"] + assert summary["session_path"] is not None + assert Path(summary["session_path"]).exists() + assert summary["next_steps"] == list(HELLO_NEXT_STEPS) + + text = format_hello_summary(summary) + assert "BotScope hello" in text + assert "Events analyzed" in text + assert "Next steps" in text + + +def test_run_hello_analysis_without_session(): + summary = run_hello_analysis(keep_session=None) + assert summary["events"] > 0 + assert summary["session_path"] is None