From 4e611d91ff1ae833600dc536e8a352287d09aee5 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Tue, 8 Sep 2026 19:00:52 -0500 Subject: [PATCH 01/18] feat(serial_plotter): add espp::Telemetry binary telemetry service + example Phase 2 of the Serial Plotter: a firmware-side binary telemetry transport the web app can plot over WebUSB / Web Serial, alongside the existing text/CSV path. - include/telemetry_service.hpp: espp::Telemetry, a small device->host protocol on the stream_frame framing (dispatcher module 3). Firmware declares named float channels (SCHEMA) and pushes SAMPLE frames (device timestamp + one float per channel, batchable); host requests are GET_SCHEMA and SET_STREAM (enable/disable + rate). Follows the CoreDumpService pattern: build frames under a mutex, invoke the user send callback with the lock released; usable with a Dispatcher (handle()) or standalone (feed()). - CMakeLists.txt / idf_component.yml: the component now ships firmware, so it registers include/ and depends on base_component + stream_frame again. - example/: an esp32s3 app that streams four synthetic channels over USB vendor + CDC, wires a Dispatcher per transport (module 3 + discovery advertising app="serial_plotter.html"), and honors GET_SCHEMA / SET_STREAM. The web-app WebUSB transport that decodes SCHEMA/SAMPLE into the same uPlot plot lands in a follow-up commit. Verified: the example builds clean against ESP-IDF v6.1 (esp32s3). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/serial_plotter/CMakeLists.txt | 19 +- .../serial_plotter/example/CMakeLists.txt | 49 +++ components/serial_plotter/example/README.md | 42 +++ .../example/main/CMakeLists.txt | 5 + .../example/main/telemetry_example.cpp | 185 +++++++++++ .../serial_plotter/example/sdkconfig.defaults | 32 ++ components/serial_plotter/idf_component.yml | 13 +- .../include/telemetry_service.hpp | 314 ++++++++++++++++++ 8 files changed, 647 insertions(+), 12 deletions(-) create mode 100644 components/serial_plotter/example/CMakeLists.txt create mode 100644 components/serial_plotter/example/README.md create mode 100644 components/serial_plotter/example/main/CMakeLists.txt create mode 100644 components/serial_plotter/example/main/telemetry_example.cpp create mode 100644 components/serial_plotter/example/sdkconfig.defaults create mode 100644 components/serial_plotter/include/telemetry_service.hpp diff --git a/components/serial_plotter/CMakeLists.txt b/components/serial_plotter/CMakeLists.txt index 1afe3ab49..460175edd 100644 --- a/components/serial_plotter/CMakeLists.txt +++ b/components/serial_plotter/CMakeLists.txt @@ -1,8 +1,11 @@ -# serial_plotter currently ships only a self-contained browser webapp -# (web/serial_plotter.html), auto-hosted by the docs "web apps" pipeline and -# usable offline over file://. There is no firmware source yet: a telemetry -# service (headers + example) built on stream_frame / dispatcher is a planned -# follow-up. This registration keeps the directory a valid ESP-IDF component so -# repo tooling and the component registry stay consistent; it gains INCLUDE_DIRS -# and REQUIRES when the telemetry service lands. -idf_component_register() +# The serial_plotter component ships: +# - web/serial_plotter.html: the browser data plotter (Web Serial + WebUSB), +# auto-hosted by the docs "web apps" pipeline and usable offline over file://. +# - include/telemetry_service.hpp: espp::Telemetry, a binary telemetry emitter +# (dispatcher module 3 over stream_frame) that the web app plots over WebUSB. +# The header only depends on base_component + stream_frame; an app pairs it with +# a Dispatcher and a transport (e.g. usb_device) — see the example. +idf_component_register( + INCLUDE_DIRS "include" + REQUIRES base_component stream_frame +) diff --git a/components/serial_plotter/example/CMakeLists.txt b/components/serial_plotter/example/CMakeLists.txt new file mode 100644 index 000000000..a6a731fe1 --- /dev/null +++ b/components/serial_plotter/example/CMakeLists.txt @@ -0,0 +1,49 @@ +# The following lines of boilerplate have to be in your project's CMakeLists +# in this exact order for cmake to work correctly +cmake_minimum_required(VERSION 3.20) + +# Build the example (and the espp components it pulls in) as C++23. Must be +# set before the project.cmake include / project() call so IDF picks it up. +set(CMAKE_CXX_STANDARD 23) + +# NOTE: the IDF component manager is intentionally left ENABLED here (unlike most +# espp examples) so that it can fetch the managed `espressif/esp_tinyusb` +# dependency declared by the usb_device component's idf_component.yml. To avoid +# the component manager scanning every espp component manifest (some board +# components declare target-specific constraints that would fail on esp32s3), +# EXTRA_COMPONENT_DIRS is narrowed to just the components this example uses; the +# in-repo espp components there satisfy the `espp/*` dependencies locally. +include($ENV{IDF_PATH}/tools/cmake/project.cmake) + +# add only the component directories that we want to use +set(EXTRA_COMPONENT_DIRS + "../../../components/base_component" + "../../../components/dispatcher" + "../../../components/format" + "../../../components/logger" + "../../../components/serial_plotter" + "../../../components/stream_frame" + "../../../components/task" + "../../../components/usb_device" +) + +# With the component manager disabled (IDF_COMPONENT_MANAGER=0, e.g. in CI so the +# build does not need the as-yet unpublished espp/* components in the registry), +# esp_tinyusb/tinyusb are not fetched; add the vendored submodule copies under +# external/ to the search path. esp_tinyusb's CMakeLists adds `tinyusb` to its +# REQUIRES when the manager is off, so both directories must be discoverable. +if(DEFINED ENV{IDF_COMPONENT_MANAGER} AND "$ENV{IDF_COMPONENT_MANAGER}" STREQUAL "0") + list(APPEND EXTRA_COMPONENT_DIRS + "../../../external/esp-usb/device/esp_tinyusb" + "../../../external/tinyusb" + ) +endif() + +set( + COMPONENTS + "main esptool_py base_component dispatcher format logger serial_plotter stream_frame task usb_device esp_tinyusb" + CACHE STRING + "List of components to include" + ) + +project(telemetry_example) diff --git a/components/serial_plotter/example/README.md b/components/serial_plotter/example/README.md new file mode 100644 index 000000000..8f3e7f588 --- /dev/null +++ b/components/serial_plotter/example/README.md @@ -0,0 +1,42 @@ +# Serial Plotter — USB telemetry example + +Streams synthetic float channels from an ESP32-S3 to the browser **Serial +Plotter** web app over USB, using `espp::Telemetry` (a binary telemetry emitter +carried on the `stream_frame` framing, dispatcher module 3). + +The hosted app — — +connects on the **vendor (WebUSB)** or **CDC (Web Serial)** interface, reads the +channel **schema**, and plots the live **sample** stream. It is the binary, +higher-rate, device-timestamped counterpart to the app's text/CSV transport. + +## What it does + +- Declares four channels — `sine`, `cosine`, `noise`, `ramp` — as the schema. +- A producer task emits one sample (a `float` per channel) every ~10 ms (100 Hz), + timestamped with the device clock. +- Exposes the stream over USB vendor **and** CDC; a `Dispatcher` on each transport + routes module-3 frames to the emitter and serves capability discovery so the + browser **Device Hub** lists this device and links to `serial_plotter.html`. +- The web app can pause/resume the stream and request a rate (`SET_STREAM`), and + requests the schema on connect (`GET_SCHEMA`). + +Swap the synthetic generator for your real signals: build a `std::array` +in channel order and call `telemetry.emit(...)`. + +## Build & run + +```sh +idf.py -p /dev/ttyACM0 flash monitor # target esp32s3 (set in sdkconfig.defaults) +``` + +Then open the Serial Plotter web app, click **Connect (USB)**, and pick the +"espp Serial Plotter" device. The system console/logs go to the separate +built-in USB-Serial-JTAG. + +## Notes + +- Native USB (vendor/WebUSB + CDC) needs an ESP32-S3 (also S2 / P4) — not the + classic ESP32. `sdkconfig.defaults` pins `esp32s3` and enables the TinyUSB + vendor + CDC classes. +- WebUSB / Web Serial are Chromium-only and need a secure context (`https`, + `http://localhost`, or `file://`). diff --git a/components/serial_plotter/example/main/CMakeLists.txt b/components/serial_plotter/example/main/CMakeLists.txt new file mode 100644 index 000000000..996af95ae --- /dev/null +++ b/components/serial_plotter/example/main/CMakeLists.txt @@ -0,0 +1,5 @@ +idf_component_register( + SRC_DIRS "." + INCLUDE_DIRS "." + REQUIRES serial_plotter dispatcher stream_frame task usb_device esp_tinyusb +) diff --git a/components/serial_plotter/example/main/telemetry_example.cpp b/components/serial_plotter/example/main/telemetry_example.cpp new file mode 100644 index 000000000..ac1056bbe --- /dev/null +++ b/components/serial_plotter/example/main/telemetry_example.cpp @@ -0,0 +1,185 @@ +// USB telemetry -> Serial Plotter web app example. +// +// Streams a few synthetic float channels from an ESP32-S3 to the browser over +// USB using the espp::Telemetry emitter (a small binary protocol carried on the +// stream_frame framing, dispatcher module 3). The hosted `serial_plotter.html` +// web app connects on the vendor (WebUSB) OR CDC (Web Serial) interface, reads +// the SCHEMA (channel names), and plots the SAMPLE stream live — the binary, +// higher-rate, device-timestamped counterpart to the app's text/CSV transport. +// +// Replace the synthetic generator below with your real signals: build a +// std::array in channel (schema) order and call telemetry.emit(...). +// The system console/logs go to the separate built-in USB-Serial-JTAG. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "dispatcher.hpp" +#include "logger.hpp" +#include "stream_frame.hpp" +#include "task.hpp" +#include "telemetry_service.hpp" +#include "usb_device.hpp" + +using namespace std::chrono_literals; +namespace sf = espp::stream_frame; + +extern "C" void app_main(void) { + espp::Logger logger({.tag = "Telemetry", .level = espp::Logger::Verbosity::INFO}); + logger.info("Starting USB telemetry (Serial Plotter) example"); + + // --- the telemetry emitter: one float per named channel, in schema order --- + espp::Telemetry telemetry({ + .channels = {"sine", "cosine", "noise", "ramp"}, + .stream_on_start = true, // stream immediately; the web app can pause via SET_STREAM + .period_ms = 10, // default 100 Hz (the web app may request another rate) + .log_level = espp::Logger::Verbosity::WARN, + }); + + // --- USB: vendor (WebUSB) + CDC (Web Serial), both carry the protocol ------- + espp::UsbDevice::Config usb_cfg; + usb_cfg.manufacturer = "espp"; + usb_cfg.product = "espp Serial Plotter"; + usb_cfg.log_level = espp::Logger::Verbosity::WARN; + espp::UsbDevice::VendorFunction vendor; + vendor.interface_name = "espp Telemetry (WebUSB)"; + vendor.webusb = true; + vendor.landing_page_url = "esp-cpp.github.io/espp/apps/serial_plotter.html"; + usb_cfg.vendor = vendor; + espp::UsbDevice::CdcFunction cdc; + cdc.interface_name = "espp Telemetry (CDC)"; + usb_cfg.cdc = cdc; + espp::UsbDevice usb(usb_cfg); + + // Reply / stream on whichever transport the host last talked on (one at a time). + enum class Transport { Vendor, Cdc }; + std::atomic active_transport{Transport::Vendor}; + std::mutex tx_mutex; + // Every device->host write (samples, request replies, discovery replies) goes + // through this one tx_mutex-guarded helper so the sample-producer task and the + // RX worker never write the TinyUSB FIFO at the same time. + auto send_to = [&](Transport dest, std::span bytes) { + std::lock_guard lock(tx_mutex); + const bool ok = (dest == Transport::Cdc) ? usb.write_cdc(bytes) : usb.write_vendor(bytes); + if (!ok) + logger.warn_rate_limited("dropped a {}-byte frame (USB TX backpressure or disconnect)", + bytes.size()); + }; + // The emitter sends to the active transport (set by the RX worker per request). + telemetry.set_send( + [&](std::span bytes) { send_to(active_transport.load(), bytes); }); + + // --- dispatchers: route module-3 frames to the telemetry service + discovery + espp::Dispatcher vendor_dispatcher, cdc_dispatcher; + const espp::Dispatcher::ModuleInfo info{.name = "Serial Plotter", + .app = "serial_plotter.html", + .description = "Live device telemetry plotting"}; + auto handler = [&](const sf::Frame &frame) { telemetry.handle(frame); }; + vendor_dispatcher.register_module(espp::Telemetry::kModule, handler, info); + cdc_dispatcher.register_module(espp::Telemetry::kModule, handler, info); + vendor_dispatcher.set_device_info(usb_cfg.product); + cdc_dispatcher.set_device_info(usb_cfg.product); + vendor_dispatcher.serve_discovery( + [&](std::span f) { send_to(Transport::Vendor, f); }); + cdc_dispatcher.serve_discovery([&](std::span f) { send_to(Transport::Cdc, f); }); + + // --- USB RX plumbing: queue in the TinyUSB callback, dispatch from a worker - + std::mutex rx_mutex; + std::condition_variable rx_cv; + std::deque>> rx_queue; + size_t rx_queued_bytes = 0; + bool rx_overflow = false; + static constexpr size_t kMaxQueuedRxBytes = 8 * sf::kMaxFrameSize; + auto enqueue_rx = [&](Transport source, std::span data) { + { + std::lock_guard lock(rx_mutex); + if (rx_queued_bytes + data.size() > kMaxQueuedRxBytes) { + rx_queue.clear(); + rx_queued_bytes = 0; + rx_overflow = true; + } else { + rx_queue.emplace_back(source, std::vector(data.begin(), data.end())); + rx_queued_bytes += data.size(); + } + } + rx_cv.notify_one(); + }; + usb.set_vendor_receive_callback( + [&](std::span data) { enqueue_rx(Transport::Vendor, data); }); + usb.set_cdc_receive_callback( + [&](std::span data) { enqueue_rx(Transport::Cdc, data); }); + + std::error_code usb_ec; + if (!usb.initialize(usb_ec)) + logger.error("Failed to initialize USB device: {} - no host transport available", + usb_ec.message()); + + espp::Task rx_task( + {.callback = [&](std::mutex &, std::condition_variable &) -> bool { + std::deque>> chunks; + bool overflowed = false; + { + std::unique_lock lock(rx_mutex); + rx_cv.wait_for(lock, 100ms, [&] { return !rx_queue.empty() || rx_overflow; }); + std::swap(chunks, rx_queue); + rx_queued_bytes = 0; + overflowed = rx_overflow; + rx_overflow = false; + } + if (overflowed) { + vendor_dispatcher.reset(); + cdc_dispatcher.reset(); + return false; + } + for (const auto &[source, chunk] : chunks) { + // Single writer of active_transport: match the chunk being dispatched + // so replies/samples go back on the transport the request arrived on. + active_transport.store(source); + (source == Transport::Vendor ? vendor_dispatcher : cdc_dispatcher).feed(chunk); + } + return false; + }, + .task_config = {.name = "telemetry_rx", .stack_size_bytes = 8192}}); + rx_task.start(); + + // --- synthetic signal producer: emit one SAMPLE per period ------------------ + // Replace this with your real signals. Values are in schema (channel) order. + const auto t0 = std::chrono::steady_clock::now(); + espp::Task gen_task( + {.callback = [&](std::mutex &m, std::condition_variable &cv) -> bool { + const float t = + std::chrono::duration(std::chrono::steady_clock::now() - t0).count(); + const float two_pi = 6.2831853f; + const float noise = static_cast(std::rand()) / RAND_MAX * 0.4f - 0.2f; + const std::array values = { + std::sin(two_pi * 1.0f * t), // sine @ 1 Hz + std::cos(two_pi * 0.5f * t), // cosine @ 0.5 Hz + noise, // uniform noise + std::fmod(t, 2.0f) - 1.0f, // -1..1 sawtooth ramp + }; + telemetry.emit(values); // no-op while streaming is paused + std::unique_lock lock(m); + cv.wait_for(lock, std::chrono::milliseconds(std::max(1, telemetry.period_ms()))); + return false; // keep running + }, + .task_config = {.name = "telemetry_gen", .stack_size_bytes = 4096}}); + gen_task.start(); + + logger.info("Telemetry ready. Open the Serial Plotter web app and connect over WebUSB / Web " + "Serial (channels: sine, cosine, noise, ramp)."); + + while (true) { + std::this_thread::sleep_for(1s); + } +} diff --git a/components/serial_plotter/example/sdkconfig.defaults b/components/serial_plotter/example/sdkconfig.defaults new file mode 100644 index 000000000..2499651f9 --- /dev/null +++ b/components/serial_plotter/example/sdkconfig.defaults @@ -0,0 +1,32 @@ +# This example uses the native USB-OTG peripheral (vendor / WebUSB + CDC), which +# is only available on the ESP32-S3 (also S2 / P4) -- NOT the classic ESP32. Pin +# the target here so a bare `idf.py build` does not fall back to esp32. +CONFIG_IDF_TARGET="esp32s3" + +# Common ESP-related +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 + +# Enable the TinyUSB vendor-specific class (THE key enablement for the vendor / +# WebUSB interface). esp_tinyusb gates CFG_TUD_VENDOR behind +# CONFIG_TINYUSB_VENDOR_COUNT; setting it > 0 compiles in the vendor class +# driver so espp::UsbDevice's vendor function (bInterfaceClass 0xFF + WebUSB) +# works. The telemetry web app plots over this WebUSB interface. +CONFIG_TINYUSB_VENDOR_COUNT=1 + +# The CDC function carries the system console (esp_tusb_init_console) AND the +# telemetry frames (so the web app also works over Web Serial); the HID count +# just keeps the usb_device component's HID references compiling (no HID +# interface is instantiated here). +CONFIG_TINYUSB_CDC_ENABLED=y +CONFIG_TINYUSB_CDC_COUNT=1 +CONFIG_TINYUSB_HID_COUNT=1 + +# Telemetry frames are small (a SAMPLE is a 4-byte timestamp + 4 bytes/channel), +# but give the TX FIFOs headroom so a burst of batched samples is never +# truncated by TinyUSB backpressure. The vendor RX FIFO only carries short +# requests (GET_SCHEMA / SET_STREAM), so 512 bytes is plenty there. +CONFIG_TINYUSB_VENDOR_RX_BUFSIZE=512 +CONFIG_TINYUSB_VENDOR_TX_BUFSIZE=4096 +CONFIG_TINYUSB_CDC_TX_BUFSIZE=4096 + +CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y diff --git a/components/serial_plotter/idf_component.yml b/components/serial_plotter/idf_component.yml index 22e11aef3..f1b9a6743 100644 --- a/components/serial_plotter/idf_component.yml +++ b/components/serial_plotter/idf_component.yml @@ -1,17 +1,22 @@ ## IDF Component Manager Manifest File license: "MIT" -description: "Browser Web Serial data plotter (uPlot): auto-parses columnar serial output and plots a high number of points, with CSV save/load and optional 2D/3D modes. A firmware-side telemetry service is a planned follow-up." +description: "Browser data plotter (Web Serial + WebUSB) that auto-parses columnar serial output and plots it, plus espp::Telemetry: a binary telemetry emitter (dispatcher module over stream_frame) the web app plots over WebUSB." url: "https://github.com/esp-cpp/espp/tree/main/components/serial_plotter" repository: "git://github.com/esp-cpp/espp.git" maintainers: - William Emfinger documentation: "https://esp-cpp.github.io/espp/apps/serial_plotter.html" +examples: + - path: example tags: - webapp - plotting - serial - web-serial + - webusb - telemetry -# No dependencies: this component currently ships only the browser webapp and -# has no firmware source, so it imposes no ESP-IDF version constraint. The -# planned telemetry service will add the deps (and an idf floor) it needs. +dependencies: + idf: + version: '>=5.0' + espp/base_component: '>=1.0' + espp/stream_frame: '>=1.0' diff --git a/components/serial_plotter/include/telemetry_service.hpp b/components/serial_plotter/include/telemetry_service.hpp new file mode 100644 index 000000000..3c6bf96a7 --- /dev/null +++ b/components/serial_plotter/include/telemetry_service.hpp @@ -0,0 +1,314 @@ +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "base_component.hpp" +#include "stream_frame.hpp" + +namespace espp { +/// @brief Binary telemetry emitter for the Serial Plotter web app. +/// +/// A tiny device->host protocol carried over the espp `stream_frame` framing +/// (so it can share one USB vendor / CDC stream with other modules via +/// `espp::Dispatcher`). Firmware declares a fixed set of named float channels +/// (the SCHEMA) and pushes SAMPLE frames — a device timestamp plus one float +/// per channel — which the hosted `serial_plotter.html` web app decodes and +/// plots, exactly like the columnar Web-Serial path but binary, higher rate, +/// and with device-accurate timestamps. +/// +/// This is the purpose-built counterpart to the app's text (CSV-style) +/// Web-Serial transport: instead of parsing printed columns, the device sends +/// typed samples directly. +/// +/// ## Wire protocol (dispatcher module id 3) +/// +/// Every message is a `stream_frame` frame with `module == kModule`. The frame +/// `type` byte's high bit distinguishes direction: host->device requests are +/// `0x0X`, device->host frames are `0x8X` (which sets the frame reply flag). +/// +/// - **GET_SCHEMA** (host->device, no payload): request the current SCHEMA. +/// - **SET_STREAM** (host->device, `[enabled u8][period_ms u16]`): enable or +/// disable streaming and request a sample period (informational — the +/// firmware's emit cadence is authoritative; the requested period is exposed +/// via period_ms() so an app can honor it). Answered with OK. +/// - **SCHEMA** (device->host): `[version u8][flags u8][nchannels u8]` then, per +/// channel, `[type u8][name_len u8][name bytes]`. `version == kSchemaVersion`, +/// `flags == 0` (reserved), channel `type == 0` (f32). Sent on GET_SCHEMA, on +/// set_channels(), and on demand via send_schema(). +/// - **SAMPLE** (device->host): one or more packed records, each +/// `[timestamp u32 microseconds][f32 × nchannels]` little-endian. A frame may +/// batch several records (payload size is an exact multiple of the record +/// size) for higher throughput. +/// - **OK** (device->host, `[request_type u8]`): acknowledges a request. +/// - **ERROR** (device->host, `[request_type u8][code u32][utf8 message]`). +/// +/// ## Threading +/// +/// emit() is typically called from a producer task while requests are handled +/// on a transport RX task; both are safe to call concurrently. Frames are built +/// under an internal mutex and the user `send` callback is always invoked with +/// the mutex released, so a re-entrant transport cannot deadlock. +class Telemetry : public espp::BaseComponent { +public: + /// Dispatcher module id owned by the telemetry protocol (the frame `module` + /// byte). Device->host Type values keep the high bit set, which the framing + /// maps to the reply flag. + static constexpr uint8_t kModule = 3; + + /// Version byte at the head of a SCHEMA payload, so the wire format can evolve. + static constexpr uint8_t kSchemaVersion = 1; + + /// Frame `type` values within the telemetry module. + enum class Type : uint8_t { + // host -> device + GetSchema = 0x01, ///< request the current SCHEMA + SetStream = 0x02, ///< [enabled u8][period_ms u16]: enable/disable + rate + // device -> host (high bit set) + Schema = 0x81, ///< channel schema (see class docs) + Sample = 0x82, ///< one or more [timestamp u32 us][f32 x nchannels] records + Ok = 0x83, ///< [request_type u8]: request acknowledged + Error = 0x84, ///< [request_type u8][code u32][utf8 message] + }; + + /// Channel value type (only 32-bit float today; reserved for future widening). + enum class ChannelType : uint8_t { F32 = 0 }; + + /// @brief Function used to transmit one encoded frame to the host. + /// @param frame The complete encoded frame bytes (header + payload + CRC). + using send_fn = std::function frame)>; + + /// Configuration for the Telemetry emitter. + struct Config { + std::vector channels; ///< Channel names, in sample order (>= 1). + send_fn send{nullptr}; ///< Transmits an encoded frame (may be set later). + bool stream_on_start{true}; ///< Start with streaming enabled. + uint16_t period_ms{20}; ///< Default requested sample period (informational). + espp::Logger::Verbosity log_level{espp::Logger::Verbosity::WARN}; ///< Logger verbosity. + }; + + /// @brief Construct the emitter. + /// @param config Channel names, the (optional) send function, and defaults. + explicit Telemetry(const Config &config) + : BaseComponent("Telemetry", config.log_level) + , channels_(config.channels) + , send_(config.send) + , streaming_(config.stream_on_start) + , period_ms_(config.period_ms) {} + + /// @brief Set (or replace) the transmit function, e.g. after USB init. + void set_send(send_fn fn) { + std::lock_guard lock(mutex_); + send_ = std::move(fn); + } + + /// @brief Push one sample with an explicit device timestamp. + /// @param values One value per channel, in schema order (size must match the + /// channel count, else the sample is dropped with a rate-limited warning). + /// @param timestamp_us Device timestamp in microseconds (u32; wraps ~71 min). + /// @note No-op while streaming is disabled or no send function is configured. + void emit(std::span values, uint32_t timestamp_us) { + if (!streaming_.load()) + return; + std::vector frame; + send_fn s; + { + std::lock_guard lock(mutex_); + if (values.size() != channels_.size()) { + logger_.warn_rate_limited("emit(): {} values for {} channels; dropping", values.size(), + channels_.size()); + return; + } + if (!send_) + return; + s = send_; + std::vector p; + p.reserve(4 + 4 * values.size()); + espp::stream_frame::put_u32(p, timestamp_us); + for (float v : values) + put_f32(p, v); + frame = build(Type::Sample, p); + } + s(frame); // send outside the lock + } + + /// @brief Push one sample timestamped with the current device time. + void emit(std::span values) { emit(values, now_us()); } + + /// @brief Redefine the channel set at runtime and send a fresh SCHEMA. + void set_channels(std::vector channels) { + { + std::lock_guard lock(mutex_); + channels_ = std::move(channels); + } + send_schema(); + } + + /// @brief The current channel names (schema order). + std::vector channels() const { + std::lock_guard lock(mutex_); + return channels_; + } + + /// @brief Whether streaming is currently enabled. + bool streaming() const { return streaming_.load(); } + + /// @brief Enable or disable streaming (SAMPLE emission). + void set_streaming(bool on) { streaming_.store(on); } + + /// @brief The host-requested sample period in milliseconds (informational). + uint16_t period_ms() const { return period_ms_.load(); } + + /// @brief Send the current SCHEMA frame now (device->host). + void send_schema() { + std::vector frame; + send_fn s; + { + std::lock_guard lock(mutex_); + if (!send_) + return; + s = send_; + frame = build(Type::Schema, build_schema_payload_locked()); + } + s(frame); + } + + /// @brief Dispatcher handler: process one frame addressed to this module. + /// + /// Register with `dispatcher.register_module(Telemetry::kModule, ...)`. Ignores + /// reply-flagged frames (device->host pushes are never host requests). + void handle(const espp::stream_frame::Frame &frame) { + if (frame.is_reply()) + return; + handle_request(frame.type, frame.payload); + } + + /// @brief Feed raw transport bytes through an internal frame parser (for use + /// without a Dispatcher). Processes every complete frame for this module. + void feed(std::span data) { + std::vector frames; + { + std::lock_guard lock(mutex_); + frames = parser_.feed(data); + } + for (const auto &frame : frames) { + if (frame.module != kModule || frame.is_reply()) + continue; + handle_request(frame.type, frame.payload); + } + } + + /// @brief Discard any partially-buffered bytes in the internal parser (e.g. + /// on transport reconnect). Only relevant when using feed(). + void reset_parser() { + std::lock_guard lock(mutex_); + parser_.reset(); + } + +protected: + /// Handle one request `type` + `payload`; builds any reply under the lock and + /// transmits it with the lock released. + void handle_request(uint8_t type, std::span payload) { + switch (static_cast(type)) { + case Type::GetSchema: + send_schema(); + break; + case Type::SetStream: { + if (payload.size() < 3) { + send_error(type, "SET_STREAM payload too short"); + return; + } + const bool enabled = payload[0] != 0; + const uint16_t period = + static_cast(payload[1] | (static_cast(payload[2]) << 8)); + if (period != 0) + period_ms_.store(period); + streaming_.store(enabled); + logger_.debug("SET_STREAM enabled={} period_ms={}", enabled, period_ms_.load()); + send_ok(type); + break; + } + default: + // Unknown telemetry type: report it (the module id already matched). + send_error(type, "unknown telemetry message"); + break; + } + } + + /// Serialize the SCHEMA payload. Caller must hold `mutex_`. + std::vector build_schema_payload_locked() const { + std::vector p; + p.push_back(kSchemaVersion); + p.push_back(0); // flags (reserved) + p.push_back(static_cast(channels_.size())); + for (const auto &name : channels_) { + p.push_back(static_cast(ChannelType::F32)); + const uint8_t len = static_cast(std::min(name.size(), 255)); + p.push_back(len); + p.insert(p.end(), name.begin(), name.begin() + len); + } + return p; + } + + void send_ok(uint8_t request_type) { + const uint8_t p[] = {request_type}; + send_frame(build(Type::Ok, p)); + } + + void send_error(uint8_t request_type, std::string_view message) { + logger_.warn("{} (type 0x{:02x})", message, request_type); + std::vector p; + p.push_back(request_type); + espp::stream_frame::put_u32(p, 0); // reserved code + p.insert(p.end(), message.begin(), message.end()); + send_frame(build(Type::Error, p)); + } + + /// Transmit an already-built frame via the configured send function (copies + /// the function pointer under the lock, then sends with the lock released). + void send_frame(std::vector frame) { + send_fn s; + { + std::lock_guard lock(mutex_); + s = send_; + } + if (s) + s(frame); + } + + /// Build an encoded frame for a telemetry message type. Device->host types + /// (high bit set) map to the frame reply flag. + static std::vector build(Type type, std::span payload = {}) { + const bool reply = (static_cast(type) & 0x80) != 0; + return espp::stream_frame::build_frame(reply, kModule, static_cast(type), payload); + } + + /// Append a little-endian IEEE-754 float32 to a byte buffer. + static void put_f32(std::vector &out, float value) { + uint32_t bits; + std::memcpy(&bits, &value, sizeof(bits)); + espp::stream_frame::put_u32(out, bits); + } + + /// Current device time in microseconds (truncated to u32; wraps ~71 min). + static uint32_t now_us() { + using namespace std::chrono; + return static_cast( + duration_cast(steady_clock::now().time_since_epoch()).count()); + } + + mutable std::mutex mutex_; + std::vector channels_; + send_fn send_; + std::atomic streaming_; + std::atomic period_ms_; + espp::stream_frame::StreamParser parser_; +}; +} // namespace espp From b7c613f9c18c6d8af5ecaed535bb1097ecef7b5d Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Tue, 8 Sep 2026 19:12:50 -0500 Subject: [PATCH 02/18] feat(serial_plotter): WebUSB binary telemetry transport in the web app + docs Adds a WebUSB transport to serial_plotter.html that plots the espp::Telemetry binary stream, alongside the existing text/CSV Web Serial path. - A new "USB" button connects over WebUSB (claims the vendor 0xFF interface's bulk IN/OUT pair), requests the schema (GET_SCHEMA), and starts the stream (SET_STREAM). It decodes stream_frame frames (vendored codec: magic/flags/ crc32 matching components/stream_frame), routes module 3, maps SCHEMA -> series and SAMPLE -> the same ring buffers / uPlot plot, using the device timestamp (u32 microseconds, unwrapped) as the X axis. Serial and USB are mutually exclusive; pause/clear/filter/save/modes all work over USB too. - Docs: a serial_plotter component doc page (Telemetry API via include-build-file) + index + example include, registered in the main toctree and the Doxygen INPUT list; web_apps.rst and the README/meta note the WebUSB path. Verified: the JS crc32 matches the C++ golden (0xCBF43926); schema/sample frames built + parsed + decoded correctly (headless); the app loads clean with the USB button and the CSV/serial path still plots (browser). Live WebUSB I/O uses the same UsbTransport pattern as the other espp consoles. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/serial_plotter/README.md | 32 ++- .../serial_plotter/web/serial_plotter.html | 255 +++++++++++++++++- doc/Doxyfile | 2 + doc/en/index.rst | 1 + doc/en/serial_plotter/index.rst | 11 + doc/en/serial_plotter/serial_plotter.rst | 51 ++++ .../serial_plotter/serial_plotter_example.md | 2 + doc/en/web_apps.rst | 4 +- 8 files changed, 334 insertions(+), 24 deletions(-) create mode 100644 doc/en/serial_plotter/index.rst create mode 100644 doc/en/serial_plotter/serial_plotter.rst create mode 100644 doc/en/serial_plotter/serial_plotter_example.md diff --git a/components/serial_plotter/README.md b/components/serial_plotter/README.md index 592ebd52f..af3eab649 100644 --- a/components/serial_plotter/README.md +++ b/components/serial_plotter/README.md @@ -1,18 +1,19 @@ # Serial Plotter -A self-contained browser tool for reading columnar serial data and plotting it -efficiently — modeled on [esp-cpp/uart_serial_plotter](https://github.com/esp-cpp/uart_serial_plotter), -but running entirely in a Chromium-based browser over the Web Serial API. No -install, no CDN, no network access. +A self-contained browser tool for reading data and plotting it efficiently — +modeled on [esp-cpp/uart_serial_plotter](https://github.com/esp-cpp/uart_serial_plotter), +but running entirely in a Chromium-based browser. No install, no CDN, no network +access. Two transports feed the same plot: + +- **Web Serial (text / CSV)** — auto-parses columnar output (a header line plus + numeric rows) from any device that prints it. +- **WebUSB (binary telemetry)** — an espp device streams typed float channels + via `espp::Telemetry` (see `include/telemetry_service.hpp` and the + [example](example/)) for higher rate and device-accurate timestamps. - **Hosted:** - **Offline:** open `web/serial_plotter.html` directly via a `file://` URL. -> This component currently ships the webapp only. A firmware-side binary -> **telemetry** transport (a `stream_frame` / `dispatcher` module for -> higher-bandwidth, typed channels) is a planned follow-up; the same webapp will -> gain a WebUSB transport that feeds the same plot. - ## Screenshots The demo data below is a Lorenz attractor (`time,x,y,z`) loaded via **Load CSV**. @@ -60,12 +61,19 @@ rest, with the per-series filter bar: (or any matching CSV) to view it offline with no device connected. - **Serial controls.** Baud selector, pause / resume, clear, and a DTR/RTS device reset. +- **Binary telemetry over WebUSB.** Connect with **USB** to an espp device + running `espp::Telemetry`: the app reads the channel schema and plots the + device-timestamped sample stream (decoded from the `stream_frame` framing, + dispatcher module 3) into the same plot. Requests the schema on connect and + can pause/resume the device stream. ## Requirements -Web Serial is available only in Chromium-based browsers (Chrome, Edge, Opera) and -needs a secure context — it works from `https`, `http://localhost`, or `file://`. -In an unsupported browser the app still loads and can **Load CSV** for viewing. +Web Serial and WebUSB are available only in Chromium-based browsers (Chrome, +Edge, Opera) and need a secure context — they work from `https`, +`http://localhost`, or `file://`. In an unsupported browser the app still loads +and can **Load CSV** for viewing. Native USB telemetry needs an ESP32-S3 (also +S2 / P4) device; see [`example/`](example/). ## Third-party diff --git a/components/serial_plotter/web/serial_plotter.html b/components/serial_plotter/web/serial_plotter.html index 312a526bb..4a4375ad7 100644 --- a/components/serial_plotter/web/serial_plotter.html +++ b/components/serial_plotter/web/serial_plotter.html @@ -4,7 +4,7 @@ espp Serial Plotter - +