0.14.3: transport and node lifecycle

README.md

@@ -168,7 +168,8 @@ What this does not solve, said plainly:
   ones; [docs/AUTHORITY-OPERATOR.md](docs/AUTHORITY-OPERATOR.md) says how, and how to pin and re-pin).
 - **Relay eviction.** `relay-auth` is not proven, and a relay token holder can make the relay
   replace another node's connection (close 4004). A squatter gets no Core Secure session — it
-  cannot prove the key — and an evicted node re-handshakes, but the relay path can be interrupted.
+  cannot prove the key — and an evicted node retries on a long backoff and re-handshakes, but the
+  relay path can be interrupted (SECURITY.md has the detail).
 - **Key compromise.** There is no key rotation: a node whose private key leaks must be
   replaced, under a new nodeId.
 - **Your own machine.** Another process running as the same user can read the identity file.

SECURITY.md

@@ -308,9 +308,16 @@ Every store a peer can feed has a fixed bound. Reaching one never stops the node
 - **Relay eviction.** `relay-auth` identity is not proven (MMP §4.4.1), and the relay replaces a
   connection that re-authenticates under the same nodeId (close 4004, §4.4.7). A token holder
   can therefore interrupt another node's relay path. It cannot impersonate it: it gets no
-  session without the key. The client stops for good on 4004 and 4006 (the existing holder is the
-  legitimate one), and says so in status, so an eviction is visible rather than a reconnect loop.
-  Every other close is reconnected with backoff, 4007 included: relay-auth key proof (MMP spec PR
+  session without the key. Until 0.14.3 the client stopped for good on 4004 and 4006, so one
+  claim switched a member off until it restarted. Since 0.14.3 it retries on a long backoff (1 min,
+  doubling to 30 min), says each close loudly in its log and status, and detects a ping-pong (three
+  identity closes in 30 min: two live processes replacing each other), which it reports as that and
+  retries only every 30 min. A token holder that keeps claiming the nodeId still keeps the member
+  off the relay for most of the time, and a host that exits on `identity-collision` still exits on
+  the first claim: exit on `info.pingPong` instead. Only a relay-side key proof ends this; it needs
+  a spec ruling (MMP §4.4.7 says a client MUST NOT reconnect after 4004, which this departs from)
+  and is not implemented. `relayIdentityRetry: false` keeps §4.4.7 as written. Every other close
+  is reconnected with backoff, 4007 included: relay-auth key proof (MMP spec PR
   meshcognition-website#20) is deferred past 0.14.0, and its review found that never reconnecting
   after 4007 makes a lockout.
 - **No key rotation** (MMP §3.4). A compromised key means a new identity.

bin/sym-daemon.js

@@ -65,7 +65,8 @@ require('../lib/core/state-root').assertTestSandbox();
 // ~/.sym. Building it from the home dir sent a rooted daemon's room, tasks and relay.env to the
 // user's real ~/.sym.
 const SYM_DIR = require('../lib/core/state-root').SYM_STATE_DIR;
-const { getSocketPath, getLogDir, listenExclusive, socketAnswers, socketIdentity, socketIsStill } = require('../lib/platform');
+const { getSocketPath, getLogDir, listenExclusive, socketAnswers, socketIdentity, socketIsStill, stampSocket, isMappedIpcEndpoint, checkIpcEndpoint, unixSocketPathMax } = require('../lib/platform');
+const { LineReader } = require('../lib/line-reader');
 const SOCKET_PATH = getSocketPath();
 // Stable name: use SYM_NODE_NAME env, or platform-scoped default
 // (not hostname — macOS appends random suffixes to hostname on WiFi,
@@ -244,25 +245,23 @@ let listeningServer = null;
 /** One IPC connection: newline-delimited JSON in, results and events out. */
 function onIPCConnection(socket) {
   const socketId = nextSocketId++;
-  let buffer = '';
-
-  socket.on('data', (data) => {
-    buffer += data.toString();
-    // One line is at most IPC_MAX_LINE: a client that never sends a newline cannot grow the buffer.
-    if (buffer.length > IPC_MAX_LINE && buffer.indexOf('\n') === -1) {
-      log(`IPC client ${socketId} sent ${buffer.length} bytes without a newline; closing it`);
-      buffer = '';
+  // Lines are cut on bytes and decoded whole (lib/line-reader.js, R3-7): decoding each chunk on its own
+  // stored a task title split across two reads with U+FFFD in it. One line is at most IPC_MAX_LINE
+  // BYTES: a client that never sends a newline cannot grow the buffer. A line that is not UTF-8 is
+  // refused like one that is not JSON.
+  const reader = new LineReader({
+    maxLineBytes: IPC_MAX_LINE,
+    strict: true,
+    onOverflow: (n) => {
+      log(`IPC client ${socketId} sent a line of more than ${IPC_MAX_LINE} bytes (${n} so far); closing it`);
       socket.destroy();
-      return;
-    }
-    let idx;
-    while ((idx = buffer.indexOf('\n')) !== -1) {
-      const line = buffer.slice(0, idx);
-      buffer = buffer.slice(idx + 1);
-      if (line.trim()) takeIPCLine(socketId, socket, line);
-    }
+    },
+    onInvalid: () => sendIPC(socket, { type: 'result', action: null, error: 'not UTF-8' }),
+    onLine: (line) => { if (line.trim()) takeIPCLine(socketId, socket, line); },
   });
 
+  socket.on('data', (data) => reader.push(data));
+
   socket.on('close', () => {
     listeners.delete(socketId);
   });
@@ -279,9 +278,15 @@ function onListening() {
   // chmod not applicable on Windows named pipes
   if (process.platform !== 'win32') {
     try { fs.chmodSync(SOCKET_PATH, 0o700); } catch {}
+    // A token in the socket file's times, so a file made later at the same inode is never taken for
+    // this one (lib/platform.js socketIdentity, R3-14).
+    stampSocket(SOCKET_PATH);
     ownSocket = socketIdentity(SOCKET_PATH);
   }
   log(`IPC server listening: ${SOCKET_PATH}`);
+  if (isMappedIpcEndpoint(SOCKET_PATH)) {
+    log(`(the configured IPC path is longer than the ${unixSocketPathMax()} bytes a Unix socket path holds: mapped to ${SOCKET_PATH}, which every sym client computes the same way)`);
+  }
 }
 
 /** A server that only logs its errors once it is listening: an accept error never ends the daemon. */
@@ -296,6 +301,9 @@ async function startIPCServer() {
   if (!fs.existsSync(SYM_DIR)) {
     fs.mkdirSync(SYM_DIR, { recursive: true });
   }
+  // A configured path longer than a Unix socket path can be is mapped (lib/platform.js unixEndpoint,
+  // R3-6) to a private directory of this user's under /tmp, made here and checked by every client.
+  checkIpcEndpoint(SOCKET_PATH, { create: true });
   // Bind first, and probe only a path that is taken: a socket file is removed only when nothing
   // answers on it and it is still the file probed. Probing first and then removing whatever was
   // there let two daemons starting together each take the path from the other.
@@ -786,11 +794,13 @@ function showStatus() {
     return;
   }
 
+  checkIpcEndpoint(SOCKET_PATH); // a mapped endpoint only in this user's private directory (R3-6)
   const client = net.createConnection(SOCKET_PATH, () => {
     client.write(JSON.stringify({ type: 'status' }) + '\n');
   });
 
   let data = '';
+  client.setEncoding('utf8'); // whole characters across reads (R3-7)
   client.on('data', (chunk) => {
     data += chunk;
     if (data.includes('\n')) {

bin/sym.js

@@ -46,7 +46,7 @@ const fs = require('fs');
 const path = require('path');
 const { execSync, spawn } = require('child_process');
 
-const { getSocketPath, getLogDir } = require('../lib/platform');
+const { getSocketPath, getLogDir, checkIpcEndpoint } = require('../lib/platform');
 const { isValidRoom } = require('../lib/rooms');
 // The CLI's state lives where its daemon's does (SYM_STATE_DIR, default ~/.sym): rooted in the
 // home, two SYM_STATE_DIR deployments shared one pid file (`sym stop` for one stopped the other),
@@ -885,12 +885,14 @@ function broadcastQuestion(question) {
       try { socket.removeAllListeners('data'); socket.destroy(); } catch {}
       resolve(v);
     };
+    checkIpcEndpoint(SOCKET_PATH); // a mapped endpoint only in this user's private directory (R3-6)
     const socket = net.createConnection(SOCKET_PATH, () => {
       socket.write(JSON.stringify({ type: 'hello' }) + '\n');
     });
+    socket.setEncoding('utf8'); // whole characters across reads (R3-7)
     let buffer = '';
     socket.on('data', (data) => {
-      buffer += data.toString();
+      buffer += data;
       let idx;
       while ((idx = buffer.indexOf('\n')) !== -1) {
         const line = buffer.slice(0, idx); buffer = buffer.slice(idx + 1);
@@ -984,12 +986,14 @@ function cmdListen() {
     console.error('sym-daemon is not running. Start it with: sym start');
     process.exit(1);
   }
+  checkIpcEndpoint(SOCKET_PATH); // a mapped endpoint only in this user's private directory (R3-6)
   const socket = net.createConnection(SOCKET_PATH, () => {
     socket.write(JSON.stringify({ type: 'listen' }) + '\n');
   });
+  socket.setEncoding('utf8'); // whole characters across reads (R3-7)
   let buffer = '';
   socket.on('data', (data) => {
-    buffer += data.toString();
+    buffer += data;
     let idx;
     while ((idx = buffer.indexOf('\n')) !== -1) {
       const line = buffer.slice(0, idx);
@@ -1160,6 +1164,7 @@ function cmdIPC(msg, formatter) {
     process.exit(1);
   }
 
+  checkIpcEndpoint(SOCKET_PATH); // a mapped endpoint only in this user's private directory (R3-6)
   const socket = net.createConnection(SOCKET_PATH, () => {
     // For status, no registration needed — send raw
     if (msg.type === 'status') {
@@ -1171,6 +1176,7 @@ function cmdIPC(msg, formatter) {
     // virtual node — design D8).
     socket.write(JSON.stringify({ type: 'hello' }) + '\n');
   });
+  socket.setEncoding('utf8'); // whole characters across reads (R3-7)
 
   let buffer = '';
   let registered = false;
@@ -1181,7 +1187,7 @@ function cmdIPC(msg, formatter) {
   }
 
   socket.on('data', (data) => {
-    buffer += data.toString();
+    buffer += data;
     let idx;
     while ((idx = buffer.indexOf('\n')) !== -1) {
       const line = buffer.slice(0, idx);

docs/WIRE-0.14.0.md

@@ -170,7 +170,13 @@ of it; on the LAN the transport itself closes and nothing is sent. What sym choo
   case-insensitively, the first occurrence winning. The daemon also browses `_<room>._tcp` where
   that is a valid RFC 6335 name (the §5.1 migration browse), and never advertises it.
 - The relay client reconnects with backoff after every close but 4004 and 4006 (4005, 4008, 1001,
-  1009, 1011, 1013, and 4007); forgets a relay's `features` with its socket; and sends relay-auth's
+  1009, 1011, 1013, and 4007). Since 0.14.3 the backoff starts over only when the relay has taken the
+  auth (relay-peers), not when the socket opens; 4004 and 4006 are retried too, on a long backoff
+  (1 min, doubling to 30 min, every 30 min after a ping-pong: three identity closes in 30 min), which
+  departs from §4.4.7's "MUST NOT automatically reconnect" (`relayIdentityRetry: false` keeps it); and
+  a message over the per-message bound (MAX_FRAME_SIZE + 4096) is dropped on its own instead of
+  closing the connection (the ws ceiling is 8 MiB, and the roster is read up to it). It forgets a
+  relay's `features` with its socket; and sends relay-auth's
   nodeId lowercase, with `room` (when not default) and `engine`. It paces at 20 frames a second,
   burst 200, under the update's floors (§19.1: 25 a second, burst 300), counting relay-auth and
   relay-pong inside the same bucket, ahead of every other frame.