Updates Maps to 0.5.0

Apps/Maps/CHANGELOG.md

@@ -1,5 +1,50 @@
 # Changelog
 
+## 0.5.0
+
+Offline search of places, streets, addresses and kinds of places in the downloaded maps.
+
+- Menu → Search, / or F: results while typing from 3 letters, nearest first, with type, distance and direction;
+  addresses as "street number" in either order; diacritics and Greek/Cyrillic in Latin letters don't matter; a
+  shorter beginning is tried when nothing matches. Nearby by category, the last 10 chosen results.
+- A chosen result is shown on the map with a mark and its sheet with Go and Save as waypoint; streets fit the screen.
+- House numbers get the nearest street within 50 m, up to 100 m marked as estimated; village addresses in regions
+  where most numbers are plain.
+- The search index per region (`maps/<region>/search.idx`, about 1.5 MB for Bratislava) is made on the device in
+  idle time, the region on screen first, and again after the region's tiles are updated; big regions get one index
+  per block of 15 x 15 tiles. Bratislava: about 5 minutes on a T-Deck, the same file the PC tool makes. On the T-Deck
+  a keystroke takes 1-150 ms (Bratislava).
+- House numbers also take named paths and pedestrian ways as their street (88 % right in Bratislava against OSM
+  addresses, 87 % without).
+- Results show the town or village they are in ("Street · Holice"), and the city district inside a city or town
+  ("Address · Ružinov, Bratislava"), so the same street name in two places can be told apart. Estimated from the
+  places in the map data; works with indexes already on the card.
+
+**Fixes**
+- While a region downloads, the tiles on screen get their POIs and house numbers after a second without input,
+  instead of after the whole download (they were not prepared while any download ran). The tile cache makes room
+  for them, and a tile on screen that the download moves into the new region stays first in line.
+- Letters the UI font lacks (such as š in "Spišská Nová Ves") are drawn as their base letter instead of being left
+  out.
+
+## 0.4.1
+
+Leaving the route while navigating is noticed sooner when the GPS is good, and not by mistake when it is poor.
+
+- The off-route limit follows the GPS precision: the median accuracy of the last 10 s of fixes (a single bad fix
+  doesn't count) gives a limit of 1.5 x accuracy + 5 m, between 15 m and 60 m (was 40 m, or accuracy + 25 m).
+- Three positions in a row beyond the limit are off route (was 10 s). With poor GPS a few more are needed (25 m
+  accuracy: 8), because GPS noise of that size alone crosses the limit now and then on a long walk.
+- Going into another street is noticed earlier: moving faster than 1 m/s more than 60 degrees away from the route's
+  direction, or being clearly nearer to a side street than to the route (with good GPS), counts from two thirds of
+  the limit.
+- The first new route is searched at once, later ones at most every 10 s (was 20 s); a new route settles for two
+  positions before it is checked.
+- Worse than 50 m accuracy: "Weak GPS" on the navigation bar, the route is followed without recalculating.
+- Fixes without an accuracy count as 30 m (Tactility gives HDOP x 5 m when the GPS reports HDOP).
+- For testing: the "replay" debug command can add GPS noise and a set accuracy, "offroute" changes the rule's
+  values on the device, and the serial log has one line per position while navigating.
+
 ## 0.4.0
 
 Offline navigation: routes for walking, cycling and driving, found on the device from the downloaded maps, with a

Apps/Maps/README.md

@@ -4,7 +4,7 @@ Offline vector maps for [Tactility](https://github.com/TactilityProject/Tactilit
 downloads an area from [OpenFreeMap](https://openfreemap.org/) by itself and draws it fullscreen, with
 anti-aliased roads, water, parks and buildings, from tiles on the SD card.
 
-Version 0.4.0, tested on a LilyGO T-Deck (ESP32-S3, 8 MB PSRAM); it also runs on ESP32-P4. Changes are listed in
+Version 0.5.0, tested on a LilyGO T-Deck (ESP32-S3, 8 MB PSRAM); it also runs on ESP32-P4. Changes are listed in
 [CHANGELOG.md](CHANGELOG.md).
 
 ## Install
@@ -132,8 +132,13 @@ S, Esc).
 - **Start** follows the route: the map turns with your direction (heading up), your position sits low on the screen
   so you see further ahead, and a bar at the top shows the next turn with an arrow and the street, e.g. "In 120 m
   turn left onto Štefánikova", and below it the distance and time left. The X at the right end of the bar (or Esc)
-  stops navigating; the menu button is below the bar. The route ahead is blue, the part you passed grey. More than 40 m off the route for 10 seconds:
-  "Off route — recalculating", and a new route from where you are (at most every 20 s). Within about 25 m of the
+  stops navigating; the menu button is below the bar. The route ahead is blue, the part you passed grey.
+- **Leaving the route** is noticed as soon as the GPS allows: the limit follows the GPS precision of the last
+  10 seconds (about 15 m with a good fix, about 40 m between tall buildings), and three positions in a row beyond
+  it, or going into another street, mean "Off route — recalculating" (with poor GPS a few more positions are
+  needed, so it doesn't happen by mistake). The first new route comes at once, later ones at most every 10 s.
+  When the GPS is very poor (worse than about 50 m) the bar says "Weak GPS" and the app keeps following the route
+  without recalculating. Within about 25 m of the
   destination: "You have arrived"; following stops and the line stays.
 - **Start and destination** snap to the nearest road the mode may use within 200 m (500 m for a car: the last bit is
   on foot), leaving out small pieces of road that lead nowhere for that mode, such as a service road inside a
@@ -148,6 +153,29 @@ S, Esc).
   tiles in idle time (the amber dot). Without a road within 200 m of the start or the destination, or without
   a connection between them in the downloaded map, the app says so.
 
+**Search (0.5.0).** Menu → Search, or / or F on the keyboard, opens the search sheet over the map:
+type a place, a street, an address ("Štefánikova 4", "4 Štefánikova", "Hlavná 12/A") or a kind of place ("cafe",
+"pharmacy"). Results come from 3 letters on, while you type, without a network: the nearest first, each with its type
+in plain words ("Café", "Street", "Address"), the distance and a small arrow pointing to it (from your position, or
+from the map centre, which the sheet says once). Diacritics don't matter ("stefanikova" finds "Štefánikova"), and
+Greek and Cyrillic names are found in Latin letters too ("kyiv"). Nothing found: the sheet shows what the longest
+matching beginning of your text finds ("Showing results for 'stefan'"). With an empty field the sheet offers "Nearby
+by category" (food, café, shops, supermarket, pharmacy, toilets, stops, parking, fuel, bank/ATM, hospital, viewpoint)
+and your last 10 chosen results. Every result names the town or village it is in, and the city district inside a city
+("Street · Holice", "Supermarket · Petržalka, Bratislava"), estimated from the nearest places on the map, a bigger
+place counting from further away. Choosing a result moves the map there with a red mark and opens its sheet with Go,
+Save as waypoint and Close; a street is shown whole. Roll or W/S move through the list, Enter opens, Esc (or
+Backspace on an empty field) closes; on touch boards the on-screen keyboard appears.
+House numbers in the map data carry no street name: the app gives each one the nearest named street within 50 m
+(88 % right in Bratislava against real OSM addresses), or one within 100 m marked "Street estimated from the map". In
+regions where most addresses are "village + number" (much of Slovakia), plain numbers are also found under the
+nearest village ("Village estimated from the map"; 78 % right on Žitný ostrov). Always check the place on the map before you
+go. The search index (about 1.5 MB for a city) is made on the device for each downloaded region, once the map has
+been left alone for 20 s and no download runs; the work dot then says "Building search index". Bratislava (99 tiles)
+takes about 5 minutes on a T-Deck; it is made again when the region's tiles are updated. Regions larger than 15 x 15
+z14 tiles get one index per block of 15 x 15 tiles. Until a region's index is ready, search doesn't find anything in
+it. The PC tool `host/search_build --tiles` makes the same file.
+
 **Menu:** Download map, Downloaded maps, Waypoints, Add waypoint, Record track, GPX files, Location, Settings, About and
 licences, Exit. W/S, trackball or finger to move; Enter, click or tap to choose; Esc, Backspace or tap
 outside to close.

Apps/Maps/main/CMakeLists.txt

@@ -6,3 +6,7 @@ tactility_component_register(
     REQUIRES graphics-module binfont-module location-module http-module esp_http_client esp_rom mbedtls
 )
 target_compile_options(${COMPONENT_LIB} PRIVATE -O2 -Wno-format-truncation)
+# The search index must match the PC's byte for byte (M6): no fused multiply-add in its float maths (MSVC on the PC
+# does not fuse; GCC would, and one house at the 50 m limit then went to another street)
+file(GLOB SEARCH_SOURCES Source/search/*.cpp)
+set_source_files_properties(${SEARCH_SOURCES} PROPERTIES COMPILE_OPTIONS "-ffp-contract=off")

Apps/Maps/main/Source/app/BinFontText.cpp

@@ -2,13 +2,15 @@
 #include "BinFontText.h"
 
 #include "../platform/Platform.h"
+#include "../search/TextFold.h"
 
 #include <binfont/binfont.h>
 #include <binfont/render.h>
 #include <graphics/pixel_buffer.h>
 #include <lvgl.h>
 #include <lvgl/fonts.h>
 
+#include <algorithm>
 #include <cstdio>
 #include <cstring>
 
@@ -36,14 +38,21 @@ uint32_t nextCodepoint(const char*& text) {
     if (first < 0x80) {
         return first;
     }
-    if ((first & 0xE0) == 0xC0 && *text != '\0') {
-        return ((first & 0x1Fu) << 6) | (static_cast<uint8_t>(*text++) & 0x3Fu);
+    // 2, 3 or 4 bytes; a broken sequence is skipped and shown as '?'
+    const int extra = (first & 0xE0) == 0xC0 ? 1 : (first & 0xF0) == 0xE0 ? 2 : (first & 0xF8) == 0xF0 ? 3 : 0;
+    uint32_t codepoint = first & (0x3Fu >> extra);
+    int taken = 0;
+    while (taken < extra && (static_cast<uint8_t>(*text) & 0xC0) == 0x80) {
+        codepoint = (codepoint << 6) | (static_cast<uint8_t>(*text++) & 0x3Fu);
+        taken++;
     }
-    // Longer sequences are skipped and shown as '?'
-    while ((static_cast<uint8_t>(*text) & 0xC0) == 0x80) {
-        text++;
+    if (extra == 0 || taken < extra) {
+        while ((static_cast<uint8_t>(*text) & 0xC0) == 0x80) {
+            text++;
+        }
+        return '?';
     }
-    return '?';
+    return codepoint;
 }
 
 } // namespace
@@ -116,10 +125,65 @@ void BinFontText::close() {
     owned = false;
 }
 
+namespace {
+
+/** Upper case Latin letters beyond Latin-1 (Latin Extended-A and -B pairs, Latin Extended Additional). */
+bool isUpperLatin(uint32_t c) {
+    if (c >= 0xC0 && c <= 0xDE) {
+        return c != 0xD7;
+    }
+    if ((c >= 0x100 && c <= 0x137) || (c >= 0x14A && c <= 0x177) || (c >= 0x1E00 && c <= 0x1EFF) || (c >= 0x218 && c <= 0x21B)) {
+        return (c & 1) == 0;
+    }
+    if ((c >= 0x139 && c <= 0x148) || (c >= 0x179 && c <= 0x17E)) {
+        return (c & 1) == 1;
+    }
+    return c == 0x178 || c == 0x181 || c == 0x186 || c == 0x189 || c == 0x18A || c == 0x191 || c == 0x198 || c == 0x1A0 || c == 0x1AF;
+}
+
+} // namespace
+
+const char* BinFontText::withFallback(const char* text) {
+    // Only rewritten when a glyph is missing
+    bool missing = false;
+    for (const char* p = text; *p != '\0' && !missing;) {
+        BinFontGlyph glyph;
+        missing = !binfont_get_glyph(font, nextCodepoint(p), &glyph);
+    }
+    if (!missing) {
+        return text;
+    }
+    size_t used = 0;
+    const char* p = text;
+    while (*p != '\0' && used + 8 < sizeof(fallbackText)) {
+        const char* start = p;
+        const uint32_t codepoint = nextCodepoint(p);
+        BinFontGlyph glyph;
+        if (binfont_get_glyph(font, codepoint, &glyph)) {
+            while (start < p && used + 1 < sizeof(fallbackText)) {
+                fallbackText[used++] = *start++;
+            }
+            continue;
+        }
+        // The search's fold: diacritics off, lower case ("Š" -> "s"), then the case back
+        char one[8] = {};
+        memcpy(one, start, std::min<size_t>(static_cast<size_t>(p - start), sizeof(one) - 1));
+        char folded[16];
+        foldLatin(one, folded, sizeof(folded));
+        for (size_t i = 0; folded[i] != '\0' && used + 1 < sizeof(fallbackText); i++) {
+            const char c = folded[i];
+            fallbackText[used++] = isUpperLatin(codepoint) && c >= 'a' && c <= 'z' ? static_cast<char>(c - 'a' + 'A') : c;
+        }
+    }
+    fallbackText[used] = '\0';
+    return fallbackText;
+}
+
 int BinFontText::textWidth(const char* text) {
     if (font == nullptr) {
         return 0;
     }
+    text = withFallback(text);
     // The same advances and kerning binfont_draw_text_rgb565() applies
     int32_t widthX16 = 0;
     uint32_t previous = 0;
@@ -153,7 +217,7 @@ void BinFontText::drawText(Canvas& canvas, int x, int y, const char* text, uint1
             return;
         }
     }
-    binfont_draw_text_rgb565(target, x, y + ascentPixels, font, text, color, PIXEL_BUFFER_CONVERSION_AVERAGE);
+    binfont_draw_text_rgb565(target, x, y + ascentPixels, font, withFallback(text), color, PIXEL_BUFFER_CONVERSION_AVERAGE);
 }
 
 bool BinFontText::glyph(uint32_t codepoint, LabelGlyph& out, bool withMask) {

Apps/Maps/main/Source/app/BinFontText.h

@@ -42,6 +42,12 @@ class BinFontText final : public Font, public LabelFont {
     void drawText(Canvas& canvas, int x, int y, const char* text, uint16_t color) override;
 
 private:
+    /**
+     * The text to draw: letters the font has no glyph for (Tactility's UI font covers ASCII and Latin-1, so "Spisska"
+     * would lose its s with caron) become their base letter, keeping upper case. The text itself when nothing is missing.
+     */
+    const char* withFallback(const char* text);
+    char fallbackText[256] {};
     BinFont* font = nullptr;
     /** Borrowed fonts belong to Tactility and are not closed. */
     bool owned = false;

Apps/Maps/main/Source/app/Debug.cpp

@@ -113,6 +113,12 @@ DebugScript::Step DebugScript::next(uint32_t now) {
                 step.key = 10;
             } else if (strcmp(argument, "bs") == 0) {
                 step.key = '\b';
+            } else if (strcmp(argument, "space") == 0) {
+                step.key = ' ';
+            } else if (strcmp(argument, "up") == 0) {
+                step.key = 0x2191;
+            } else if (strcmp(argument, "down") == 0) {
+                step.key = 0x2193;
             } else {
                 step.key = static_cast<uint8_t>(argument[0]);
             }
@@ -163,10 +169,32 @@ DebugScript::Step DebugScript::next(uint32_t now) {
             step.action = Action::WaitRoute;
             return step;
         }
-        if (strcmp(command, "replay") == 0 && sscanf(argument, "%31s", step.name) == 1) {
+        if (strcmp(command, "replay") == 0 && sscanf(argument, "%31s %f %f %f", step.name, &step.values[0], &step.values[1], &step.values[2]) >= 1) {
+            // replay <file> [accuracy m] [noise m] [seed]
             step.action = Action::Replay;
             return step;
         }
+        if (strcmp(command, "searchindex") == 0) {
+            step.action = Action::SearchIndex;
+            step.zoom = strncmp(argument, "force", 5) == 0 ? 1 : 0;
+            // "searchindex force <region>": that region only
+            if (step.zoom != 0 && sscanf(argument, "force %31s", step.name) != 1) {
+                step.name[0] = '\0';
+            }
+            return step;
+        }
+        if (strcmp(command, "waitsearchindex") == 0) {
+            step.action = Action::WaitSearchIndex;
+            return step;
+        }
+        if (strcmp(command, "filecheck") == 0 && sscanf(argument, "%31s", step.name) == 1) {
+            step.action = Action::FileCheck;
+            return step;
+        }
+        if (strcmp(command, "offroute") == 0 && sscanf(argument, "%31s %f", step.name, &step.values[0]) == 2) {
+            step.action = Action::OffRoute;
+            return step;
+        }
         float v[6] = {};
         const int found = sscanf(argument, "%f %f %f %f %f %f", &v[0], &v[1], &v[2], &v[3], &v[4], &v[5]);
         struct TouchCommand {

Apps/Maps/main/Source/app/Debug.h

@@ -20,7 +20,13 @@ bool saveBmp(const char* path, const uint16_t* pixels, int width, int height);
  * Navigation (M5c): "routebench <file> <walk|bike|car|all>" runs the route search over the pairs of a file in the app's
  * folder (host/routing/export_pairs.py) and logs each; "navigate <lat> <lon> <walk|bike|car>" asks for a route from the
  * position (or the screen centre) as "Navigate here" does; "waitroute" waits until no search runs; "replay <file>"
- * plays a GPX track from the app's folder as positions, one point per second ("replay stop" ends it).
+ * plays a GPX track from the app's folder as positions, one point per second ("replay stop" ends it);
+ * "replay <file> <accuracy> <noise> [seed]" reports that accuracy in metres and moves the points by correlated random
+ * noise of that many metres (0.4.1). "offroute <name> <value>" sets a value of the off-route rule (OffRouteTuning).
+ * Search (M6): "searchindex [force [region]]" starts building the search indexes now ("force" also rebuilds current
+ * ones, of that region only when one is named),
+ * "waitsearchindex" waits until the build is over, "filecheck <path>" logs the size and CRC-32 of a file under the
+ * app's folder (to compare an index built on the device with the PC's).
  */
 class DebugScript {
 public:
@@ -38,7 +44,11 @@ class DebugScript {
         RouteBench,
         Navigate,
         WaitRoute,
-        Replay
+        Replay,
+        OffRoute,
+        SearchIndex,
+        WaitSearchIndex,
+        FileCheck
     };
 
     enum class TouchKind : uint8_t { Tap, Hold, Drag, Pinch, Twist };