Embedded edge vision framework — real-time computer vision inference on an ESP32 camera node.
EdgeCV explores moving lightweight CV workloads off centralised servers and onto low-cost embedded hardware. The current prototype runs inference entirely on an ESP32-CAM with an OV3660 camera — no server, no cloud, no network connection required.
Two demos are included:
- MNIST digit recogniser — the original prototype; classifies handwritten digits from the camera feed using an INT8 CNN and the emlearn-micropython runtime
- Custom object classifier — the active project; a full pipeline for any two-class object recogniser, currently trained to distinguish guava vs powerbank
A fully offline, real-time image classifier running on a microcontroller that costs under $10:
| Metric | Value |
|---|---|
| Inference time | 184 ms per frame (~5 FPS) — debug-build number, see below |
| Preprocessing (JPEG decode + convert) | ~27 ms per frame |
| Model size (int8) | 27 KB (embedded in firmware flash) |
| Tensor arena | 48 KB used of 96 KB, internal DRAM |
| Camera | OV3660, 96×96 JPEG |
| Classification | guava vs powerbank — see Measured accuracy |
The 184 ms figure was measured under a debug build and is being superseded. It was taken at 160 MHz with
-Ogand assertions at level 2, and with the tensor arena in 40 MHz quad-SPI PSRAM. The build now requests 240 MHz,-O2, silent assertions and 80 MHz PSRAM, and places the arena in internal DRAM. The arena placement was likely the largest single cost: TFLM walks the arena with poor cache locality, and PSRAM random access on the ESP32 is roughly an order of magnitude slower than internal SRAM. The arena needs only 48 KB and fits internal DRAM easily — the old 256 KB request is what forced it out to PSRAM. The replacement timing has not been measured on hardware yet. The serial log now prints which heap the arena landed in; check that line before comparing any two timing numbers. | Open-world rejection | UNKNOWN output for unrecognised objects | | Temporal stabilization | confidence averaging / EMA / vote / hysteresis | | WiFi live stream | MJPEG to browser + JSON classification endpoint | | Power source | USB (no external compute) |
The model runs from flash (embedded as a C array via xxd), loads in milliseconds, and classifies at approximately 5 frames per second. No SD card, no WiFi, no host PC needed after flashing.
EdgeCV/
├── README.md
├── training_devlog.md # Full development log — every phase, bug, and fix
│
├── mnistDigits/ # MNIST digit recogniser — self-contained
│ ├── src/
│ │ ├── cam_sanity.py
│ │ ├── downscale.py
│ │ └── mnist_cnn_camera.py
│ ├── models/
│ │ └── mnist_cnn_int8.tmdl
│ └── firmware/
│ ├── esp-dev.bin
│ └── esp-cam.bin
│
└── classifier/ # Custom two-class object classifier
├── prepare_dataset.py # Step 1 — ingest + leakage-free temporal split + sanitize
├── sanitizer_dataset.py # hand removal + duplicate filtering (called by step 1)
├── train_classifier.py # Step 2 — training script (TinyML CNN, int8)
├── eval_model.py # Step 3 — measure the int8 model (run before flashing)
├── test_laptop.py # Step 4 — webcam validation before flashing
├── images_of_guava.zip # Raw captures, as recorded
├── images_of_powerbank.zip
├── dataset/ # Generated by prepare_dataset.py — gitignored
│ ├── raw/<class>/ # Every raw frame, untouched
│ ├── train/<class>/ # Sanitized training frames
│ ├── test/<class>/ # Held-out TAIL of each clip, raw, never trained on
│ └── manifest.json # sha256 + split + timestamp per image — COMMITTED
├── model.keras # Trained Keras model (best weights)
├── model.tflite # Int8 TFLite model — deploy this
├── model_history.png # Training accuracy/loss curves
├── requirements.txt # Python dependencies
├── scripts/
│ └── live_infer.py # Laptop viewer — receives TCP stream, displays in OpenCV window
└── firmware/ # ESP-IDF project (C++) — build and flash
├── CMakeLists.txt # Top-level project CMakeLists
├── sdkconfig.defaults # Base config: PSRAM, SDMMC, main task stack
├── sdkconfig.defaults.camera.additions # Camera-specific additions
├── dependencies.lock # Pinned component versions
├── managed_components/ # Auto-downloaded by idf.py update-dependencies
└── main/
├── CMakeLists.txt # Selects active .cc file via BUILD_MODE cmake var (probe/stream/wifi)
├── idf_component.yml # Dependencies: esp-tflite-micro, esp32-camera
├── model_data.h # C header declaring the embedded model array
├── model_data.cc # Generated by xxd — contains the model bytes
├── main_tinyml_probe.cc # Serial-only inference (int8, no WiFi) — DEFAULT
├── main_stream_raw.cc # TCP raw stream + serial inference + mDNS
├── main_wifi_stream.cc # WiFi + MJPEG browser stream + inference
├── main_edgecv_v2.cc # Phase 2 — profiling + rejection + stabilization
├── main_model_probe.cc # Probe: model load + AllocateTensors only
└── main_camera_probe.cc # Probe: model + camera coexistence check
| Component | Notes |
|---|---|
| ESP32-CAM | Main MCU + camera board (AI Thinker module) |
| OV3660 Camera Module | Attached via parallel interface |
| USB–UART adapter | Flashing and serial monitor |
| Host PC (Linux / WSL) | Tooling: idf.py, esptool, picocom |
No SD card is required — the model is embedded directly in firmware flash as a C array.
pip install -r classifier/requirements.txt
# or manually:
pip install tensorflow pillow numpy scikit-learn matplotlib opencv-python imagehash- ESP-IDF v5.3 — install from https://docs.espressif.com/projects/esp-idf/
- Components are downloaded automatically by
idf.py update-dependencies:espressif/esp-tflite-microespressif/esp32-camera
See mnistDigits/ — self-contained MicroPython demo. Refer to the original README section (archived below the classifier docs) for flashing instructions.
images_of_*.zip raw captures (4 fps video, one clip per class)
│
▼
prepare_dataset.py ingests archives -> dataset/raw/
TEMPORAL split on the capture timeline with a
2 s guard band, BEFORE sanitising
sanitises the train portion only
writes dataset/{train,test}/ + manifest.json
│
▼
train_classifier.py trains the TinyML scratch CNN on dataset/train/
(temporal train/val split)
exports int8 TFLite + model.tflite.provenance.json
│
▼
eval_model.py measures the INT8 model on dataset/test/:
confusion matrix, per-class P/R/F1,
rejection-threshold sweep, contamination check
│
▼
test_laptop.py validates model on laptop webcam before flashing
│
▼
xxd -i model.tflite > main/model_data.cc embeds model into firmware
│
▼
idf.py build flash compiles and flashes to ESP32-CAM
│
▼
OV3660 → JPEG → jpg2rgb565 decode → RGB565 unpack → int8 tensor → TFLite Micro → class + confidence
Input: (96, 96, 3) RGB [Rescaling(1/255) baked in — uint8 at inference]
→ Conv2D(8, 3×3, stride=2) → ReLU → MaxPool(2×2) output: 23×23×8
→ Conv2D(16, 3×3) → ReLU → MaxPool(2×2) output: 10×10×16
→ Conv2D(16, 3×3) → ReLU output: 8×8×16
→ Flatten → Dropout(0.5) → Dense(16, ReLU) → Dense(1, sigmoid)
Output: single sigmoid neuron
≥ 0.5 → powerbank (class index 1)
< 0.5 → guava (class index 0)
Quantized to int8: ~50 KB. Fits in ESP32 flash with the default partition scheme — no SD card needed.
cd classifier
python prepare_dataset.pyIngests images_of_*.zip and builds a leakage-free split. It:
- Extracts raw captures to
dataset/raw/<class>/ - Splits each class on its capture timeline, holding out the last 30% as test, with a 2 s guard band dropped at the seam so the last train frame and first test frame are never consecutive frames of one motion
- Runs
sanitizer_dataset.py's hand removal on the train portion only — the test portion stays raw, because raw is what the camera sees at deployment - Writes
dataset/manifest.jsonwith the sha256, split and capture timestamp of every image
Why a temporal split and not a random one. The captures are continuous video, not independent photographs:
| class | frames | span | capture sessions | largest gap |
|---|---|---|---|---|
| guava | 155 | 38.5 s @ 4 fps | 1 | 265 ms |
| powerbank | 164 | 40.8 s @ 4 fps | 1 | 264 ms |
Frames 250 ms apart are near-identical (median adjacent dHash distance 8; 382 guava pairs sit within dHash 10 at a mean 2.2 s separation), and the perceptual-hash dedup at threshold 5 only removes 40/155 and 20/164 of them. A random split therefore places near-duplicates on both sides of the boundary and measures memorisation. Splitting on the timeline is the best available fix — but see Measured accuracy for why it still is not enough.
The previous pipeline leaked the test set into training.
sanitizer_dataset.pyread fromdataset/test_guava/and wrote hand-removed copies intodataset/guava/; training then usedguava/and evaluated ontest_guava/, described as "never seen during training". Every test image was the raw original of a photo whose inpainted twin was in the training set. Separately, the group-aware split was a no-op: the sanitizer renames its output to a fresh sequential index, so every group had exactly one member and the split degenerated into a random one.prepare_dataset.pyfixes both.
python train_classifier.pyPaths are now relative to the script — no editing needed. The script:
- Loads
dataset/train/<class>/and splits train/val on the capture timeline (setTEMPORAL_VAL_SPLIT = Falseto fall back to the old group split) - Trains with early stopping (patience=8) and LR reduction on plateau
- Evaluates per-class accuracy on
dataset/test/<class>/ - Exports
model.tflite(int8 quantized, calibrated on training data only) - Writes
model.tflite.provenance.jsonrecording exactly which images fitted the model, so a later evaluation can prove whether its test set was genuinely held out
python eval_model.py --split test --dump-missesEvaluates the int8 .tflite — the artefact that ships to flash — not the
float Keras model. Reports a confusion matrix, per-class precision/recall/F1,
balanced accuracy, a rejection-threshold sweep for tuning
CONFIDENCE_THRESHOLD, and a contamination check against the model's
provenance file. Needs an interpreter: pip install ai-edge-litert (smallest),
or tensorflow, or tflite-runtime.
python test_laptop.pyValidates the model against your laptop webcam before flashing. Edit LABELS at the top to match your classes.
cd classifier/firmware/main
xxd -i ../../model.tflite > model_data.ccThis regenerates model_data.cc with the new model bytes. The header model_data.h stays unchanged.
cd classifier/firmware
# First time only — download ESP-IDF components
idf.py update-dependencies
# Build and flash — serial-only inference (default)
idf.py -p /dev/ttyUSB0 build flash monitor
# Build and flash — TCP stream to laptop (live_infer.py)
idf.py -DBUILD_MODE=stream -p /dev/ttyUSB0 build flash monitor
# Build and flash — WiFi + MJPEG browser stream (edit WiFi credentials first)
idf.py -DBUILD_MODE=wifi -p /dev/ttyUSB0 build flash monitorThe default build mode is probe (main_tinyml_probe.cc). Pass -DBUILD_MODE=<mode> to select
a different entry point. Valid values: probe, stream, wifi, edgecv_v2, model_probe, camera_probe.
When running main_stream_raw.cc, use live_infer.py on your laptop to
display the camera feed in real time. Inference runs on the ESP32 and prints
to serial — the laptop script does display only.
# Install dependencies (one time)
pip install opencv-python numpy
# Run the viewer
cd classifier/scripts
python live_infer.pyThe viewer connects to classifier-espcam.local:5555 via mDNS. On Windows,
Bonjour must be installed for .local resolution to work. If it fails,
hardcode the IP shown in the serial monitor:
# live_infer.py — change this line
ESP32_HOST = "192.168.x.x" # IP from serial monitor outputAll build modes:
# probe (default) — serial-only inference, no WiFi
idf.py build flash monitor
# stream — TCP raw stream + serial inference + mDNS (laptop viewer: python live_infer.py)
idf.py -DBUILD_MODE=stream build flash monitor
# wifi — WiFi + MJPEG stream + browser UI (edit WIFI_SSID and WIFI_PASSWORD first)
idf.py -DBUILD_MODE=wifi build flash monitor
# edgecv_v2 — Phase 2: per-stage profiling + open-world rejection + temporal stabilization
idf.py -DBUILD_MODE=edgecv_v2 build flash monitor
# model_probe — confirms model loads and AllocateTensors succeeds (no camera)
idf.py -DBUILD_MODE=model_probe build flash monitor
# camera_probe — confirms camera can coexist with resident model
idf.py -DBUILD_MODE=camera_probe build flash monitorThe mode is passed as a CMake cache variable (-D), not an environment variable.
The legacy APP_MAIN=filename.cc env-var still works as a fallback.
I (xxxx) TINYML: Tensor arena: 96 KB in INTERNAL DRAM
I (xxxx) TINYML: AllocateTensors OK — arena used 48 KB of 96 KB
I (xxxx) TINYML: Input tensor: dims=[1,96,96,3] type=9 scale=1.000000 zero_point=-128
I (xxxx) TINYML: Output tensor: dims=[1,1] type=9 scale=0.003922 zero_point=-128
I (xxxx) TINYML: --- Frame 0 ---
I (xxxx) TINYML: guava : 0.0625 (6.2%)
I (xxxx) TINYML: powerbank : 0.9375 (93.8%)
I (xxxx) TINYML: PREDICTION : powerbank (93.8%)
I (xxxx) TINYML: TIMING : pre=6.8ms invoke=184.6ms total=191.6ms
The model has layers.Rescaling(1/255) baked into its weights. On the ESP32, do not divide by 255 manually. Convert camera pixels as:
int8_val = (int8_t)(uint8_rgb_pixel - 128)
This is equivalent to applying zero_point = -128. Verify by checking the printed zero_point value on boot — it should be -128.
The input tensor's scale is 1.000000, not 1/255. Because Rescaling(1/255)
sits inside the graph, the int8 input carries raw 0–255 pixel values offset by
-128; the 0.003922 figure belongs to the output tensor. Earlier versions of
this README and of train_classifier.py printed 0.003922 for the input, which
was wrong. train_classifier.py now reads both values back out of the exported
.tflite and prints what the converter actually produced.
Produced by python eval_model.py. Numbers, not adjectives — the repo
previously reported quality as ">90% confidence on clean samples", but
confidence is not accuracy, and misclassified/ contains a powerbank frame
called guava at 0.97 confidence.
Confusion matrix (rows = actual, cols = predicted)
guava powerbank total
guava 155 0 155
powerbank 3 161 164
Overall accuracy : 99.06% (316/319)
Balanced accuracy: 99.09%
class precision recall f1 support
guava 0.981 1.000 0.990 155
powerbank 1.000 0.982 0.991 164
All three errors are powerbank → guava. This is a memorisation ceiling, not
a generalisation result: this model was fitted by the old pipeline on all 319
frames, so it has seen every image it is being scored on. eval_model.py says
so rather than printing a reassuring number — it refuses to claim a
contamination figure for any model without a .provenance.json.
This is how CONFIDENCE_THRESHOLD in main_edgecv_v2.cc should be chosen.
conf = max(p, 1-p); the equivalent MARGIN_THRESHOLD is 2·thresh − 1.
| thresh | margin | accepted | accuracy among accepted | rejected |
|---|---|---|---|---|
| 0.50 | 0.00 | 319/319 | 99.06% | 0 |
| 0.70 | 0.40 | 315/319 | 99.37% | 4 |
| 0.75 | 0.50 | 312/319 | 99.68% | 7 |
| 0.90 | 0.80 | 301/319 | 99.67% | 18 |
| 0.95 | 0.90 | 286/319 | 100.00% | 33 |
| 0.99 | 0.98 | 232/319 | 100.00% | 87 |
The actionable finding: one error survives at 0.95 confidence. Raising the
threshold to 0.90 throws away 18 frames (6%) and still passes that error
through. Rejection thresholds cannot catch a confidently wrong prediction —
only better data can. For a single-sigmoid binary model the three
REJECT_MODE_* options in main_edgecv_v2.cc all reduce to this one knob
(max(p,1-p) < t is exactly |p−(1−p)| < 2t−1), which the file's own comment
already notes.
| Object instances per class | 1 |
| Capture sessions per class | 1 |
| Backgrounds / lighting conditions | 1 |
| Total footage | 38.5 s + 40.8 s |
A temporal split makes the metric honest. It does not make the dataset adequate. With one guava, one powerbank, one background and one lighting setup, even a perfectly clean held-out tail only answers "can it recognise the back half of this one clip" — it cannot answer "can it recognise a guava". The model has never been shown a second guava, so nothing measurable here predicts what it does when it sees one.
What would fix it, in rough order of value: several physical instances per
class; separate capture sessions on different days, surfaces and lighting; a
trained background/negative class recorded from the rover's own camera at
chassis height (floor, walls, chair legs, wheels), which is strictly better
than any confidence threshold on a 2-class sigmoid; and varied distance and
viewing angle. Once there are multiple sessions, the split should hold out
whole sessions rather than a tail — prepare_dataset.py already detects
session boundaries from gaps in the capture timeline.
| Symptom | Likely cause | Fix |
|---|---|---|
| Board stuck in reset | RTS/DTR held high by USB adapter | Use --lower-rts --lower-dtr in picocom |
esp_camera_init failed: ESP_FAIL |
OV3660 doesn't support PIXFORMAT_RGB888 |
Use PIXFORMAT_YUV422 — the firmware already does this |
AllocateTensors() failed |
Arena too small | Increase TENSOR_ARENA_SIZE, but keep it under what internal DRAM can serve — past that the allocation falls back to PSRAM and inference slows by roughly an order of magnitude. Check the Tensor arena: N KB in ... log line. |
| Predictions correct but inverted | Class index order wrong | Check that folder names are exactly guava and powerbank (alphabetical = [0] guava, [1] powerbank) |
| Classifies background not object | Background bias in training data | Vary backgrounds; run sanitizer to remove hands |
| Model outputs ~50% for both classes | Training collapse or wrong input type | Check that fill_input_tensor_int8() is writing to input->data.int8, not input->data.f |
Build error: format '%d' expects int but got int32_t |
dims->data[] and params.zero_point are int32_t |
Cast to (int) in log format arguments |
ImportError: imagehash |
Missing dependency | pip install imagehash |
cam_hal: FB-OVF spam |
CAMERA_GRAB_LATEST with slow consumer |
Change to CAMERA_GRAB_WHEN_EMPTY in camera config |
| Inference produces garbage with RGB565 | Buffer sized for 3 bytes/px, loop treats packed uint16_t as raw bytes | Allocate W*H*2 bytes; unpack uint16_t with bit shifts before int8 cast |
| Everything classified as UNKNOWN | Rejection thresholds too strict | Lower CONFIDENCE_THRESHOLD and/or MARGIN_THRESHOLD in main_edgecv_v2.cc |
| Prediction flips every frame at boundary | No temporal stabilization active | Enable STABILITY_CONF_AVG or STABILITY_HYSTERESIS |
cam_hal: NO-SOI with JPEG pixel format |
OV3660 JPEG pipeline fails at 96×96 | Use PIXFORMAT_YUV422 + frame2jpg() for software encode |
mDNS component not found build error |
mdns not in idf_component.yml | Run idf.py add-dependency "espressif/mdns" |
socket.gaierror on .local hostname |
getaddrinfo() bypasses Windows mDNS |
Use socket.gethostbyname() in the Python script, or hardcode the IP |
Serial says Tensor arena: 96 KB in PSRAM (SLOW) |
Internal DRAM exhausted before the arena was allocated | Expected in wifi mode (the WiFi stack takes internal RAM). Reduce TENSOR_ARENA_SIZE toward the ~48 KB actually used, or accept the slowdown |
| Board unstable / random resets after the perf config change | 80 MHz quad PSRAM is not reliable on every ESP32 module revision | Revert CONFIG_SPIRAM_SPEED_80M and CONFIG_ESPTOOLPY_FLASHFREQ_80M in sdkconfig.defaults first, then rm sdkconfig && idf.py reconfigure |
| Perf settings seem to have no effect | idf.py does not retro-apply sdkconfig.defaults to an existing sdkconfig |
rm sdkconfig && idf.py reconfigure — defaults are only read when sdkconfig is absent |
eval_model.py: no interpreter found |
No TFLite runtime installed | pip install ai-edge-litert (smallest), or tensorflow, or tflite-runtime |
IPSTR/IP2STR type mismatch error |
Macro expects esp_ip4_addr_t not struct in_addr |
Use inet_ntoa(client_addr.sin_addr) instead |
picocom --baud 115200 --lower-rts --lower-dtr /dev/ttyUSB0Exit with Ctrl-A then Ctrl-X.
WSL note: Use usbipd-win to attach the USB adapter:
usbipd list
usbipd attach --wsl --busid <BUSID>Working end-to-end on hardware.
- TinyML scratch CNN (int8, ~50 KB) embedded in firmware flash
- OV3660 captures YUV422, converted to RGB in firmware (BT.601)
- Inference: ~184 ms per frame (~5 fps theoretical max)
- Classification: 99.06% on the 319 captured frames — but this is a memorisation ceiling measured on the model's own training data, not a generalisation result. See Measured accuracy.
- No SD card, no WiFi, no server required
- MNIST digit recognition on ESP32-CAM hardware
- Custom training pipeline (scratch TinyML CNN, int8 quantization)
- Dataset sanitizer (hand removal + duplicate filtering)
- Laptop webcam tester
- Scratch TinyML CNN — working end-to-end on ESP32-CAM at 184ms/frame
- WiFi + MJPEG live view (memory headroom confirmed, architecture designed)
- Drive rover motors based on classification output
- Collect more varied training data to improve edge-case accuracy
- Per-class confidence threshold tuning
- Multi-node distributed camera system
- OTA model updates
- Where the tensor arena lives matters more than how big it is. The arena was being allocated
MALLOC_CAP_SPIRAMfirst with a 256 KB request, whileAllocateTensors()reported only ~48 KB in use. TFLM walks the arena with poor cache locality and ESP32 PSRAM is quad-SPI behind a small cache, so that placement is roughly an order of magnitude slower per access than internal SRAM — for no benefit, since 48 KB fits internal DRAM easily. Oversizing an arena is not free when the size is what pushes it into slower memory. - Check the build configuration before optimising code. The published 184 ms/frame was measured at 160 MHz with
-Ogand assertions at level 2 — a debug build at two-thirds clock. Several of the "we need a faster model" conclusions that follow from a number like that are really conclusions aboutsdkconfig. idf.pynever retro-appliessdkconfig.defaults. Defaults are read only whensdkconfigdoes not exist. Editing a defaults file and rebuilding changes nothing, silently. Also:SDKCONFIG_DEFAULTSmust be set in the top-levelCMakeLists.txtfor a second defaults file to be merged at all —sdkconfig.defaults.camera.additionshad never reached a build, despite the README instructing that it be merged.- Confidence is not accuracy. A sigmoid output far from 0.5 means the model is committed, not that it is right. This model's worst error is a powerbank called guava at 0.97 confidence, which no rejection threshold can catch without discarding most of the correct predictions too.
- A test set built by transforming the training set is not a test set. The sanitizer read
test_guava/and wroteguava/; training usedguava/and "validated" ontest_guava/. Every held-out image was the raw original of a training image. The fix has to split before the transform, not after. - Check whether your frames are photographs or video. 155 files looks like a dataset; 155 files whose timestamps are 250 ms apart with no gap over 265 ms is a 38-second clip. No split strategy recovers independence that the capture never had, and a perceptual-hash dedup at a tight threshold removes only the most obvious duplicates (40/155 here) while leaving the rest.
- A group-aware split is only as good as its grouping key.
build_group_splitgrouped by base filename to keep related images together, but the sanitizer renamed every output to a fresh sequential index — so every group had one member and the split silently degenerated into the random split it was written to replace. - The OV3660 does not support
PIXFORMAT_RGB888— capture YUV422 and convert to RGB in firmware using BT.601 coefficients to match PIL's colour space. - TFLite int8 models expect
int8input, notfloat32. Writing float bytes into an int8 tensor pointer (input->data.finstead ofinput->data.int8) silently corrupts every pixel before inference. layers.Rescaling(1/255)inside the model means the conversion is baked into the quantized weights. Do not normalise pixel values before filling the tensor on the device.- If a model classifies background brightness instead of object identity, vary backgrounds in training data. Pointing the camera at a white ceiling vs a dark desk should not flip the prediction.
- Group-aware train/val splitting matters when the dataset contains multiple versions of the same physical photo (e.g. original + hand-cleaned). A naive random split leaks the same scene into both sets and gives optimistic validation accuracy.
int32_tislong inton the ESP32 toolchain. Use(int)casts when passingdims->data[]orparams.zero_pointto%dformat specifiers, or the build fails with-Werror=format.- Scratch CNNs on distinct object classes outperform MobileNetV2 transfer learning when the dataset is small and the objects are visually different. MobileNetV2 adds training time, model size, and inference latency without benefit in that regime.
CAMERA_GRAB_LATESTcauses constant frame buffer overflow when the consumer is slower than the DMA. UseCAMERA_GRAB_WHEN_EMPTYunless you specifically need the absolute latest frame.- When changing JPEG decode functions, update the buffer size, the pixel loop, and the tensor fill together — they are coupled. RGB565 is 2 bytes/pixel packed uint16_t, not 3 separate bytes.
- A binary classifier is always closed-world — it will confidently classify a wall or keyboard as guava or powerbank. Add rejection logic before using predictions for anything real.
- Temporal stabilization is essential before driving actuators. Use hysteresis for motor control specifically: it prevents rapid alternation at decision boundaries that causes physical wear.
- The OV3660 JPEG pipeline fails at 96×96 (
NO-SOI). Capture YUV422 and encode to JPEG withframe2jpg()for transmission — this also keeps inference using the same YUV frame without an extra decode step. IP2STR()andIPSTRmacros expectesp_ip4_addr_t*, notstruct in_addr*. Useinet_ntoa()for POSIX socket addresses.mdnsis not bundled with ESP-IDF v5.3 — add it withidf.py add-dependency "espressif/mdns".- On Windows,
socket.getaddrinfo()with explicit flags bypasses the Bonjour mDNS resolver. Usesocket.gethostbyname()instead.