Skip to content

fix(sync): stop losing notes and edits during Google Drive sync - #464

Merged
penge merged 16 commits into
masterfrom
fix/sync-data-loss
Oct 24, 2026
Merged

penge merged 16 commits into
masterfrom
fix/sync-data-loss

Conversation

@penge

@penge penge commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Fixes several ways Google Drive Sync could silently lose notes or edits, ordered below by impact. Code paths are relative to src/background/google-drive/.

Fixes

  1. Synced notes deleted after any edit
    • Problem: listFiles read only the first page of the Drive listing. Every synced, untouched note whose file was not on that page looked deleted in Drive and was removed locally. With more than 100 notes this happened on every sync, and Drive can also return a short page below that.
    • Fix: listFiles requests pageSize=1000, follows nextPageToken until the listing is complete, and returns nothing rather than a partial listing.
    • Code: api.ts (listFiles), files/list.ts
  2. Text typed during a sync overwritten
    • Problem: sync read the notes, spent seconds on Drive requests, then wrote that copy back over everything changed in the meantime. Typed text, lock, pin, raw, renames, deletions and new notes were all lost or reverted. The editor kept showing the text, so the loss surfaced only on the next open.
    • Fix: sync re-reads the notes before writing and merges per note, matching notes by their Drive file across renames. Changes made during the sync win, content pulled from Drive for an edited note is appended as it is for any two-sided edit, and a file created for a note deleted or renamed meanwhile is removed from Drive. lastSync marks the time the notes were read, so Auto Sync pushes those edits next.
    • Code: sync/merge.ts, sync/sync-notes.ts
  3. A merged note never uploaded
    • Problem: when pull merged a note edited on both sides, it stamped the note with the file's time, so the note looked synced and push skipped it. The merged text stayed on this device only, and the next edit of the file elsewhere replaced it. A note merged while being typed in during a sync had the same gap when the other device's clock was ahead: push skipped it, and the next sync appended the other device's text a second time.
    • Fix: every merged note is stamped later than both now and its file's time plus 1 second, so push uploads it whatever the devices' clocks say.
    • Code: sync/pull/pull-update.ts, sync/merge.ts
  4. One transient error disabled sync and later duplicated text
    • Problem: a failed folder lookup or file listing called stop(), which revoked the permission, removed the sync settings and unlinked every note. Re-enabling sync matched notes to files by name and appended Drive content to every note that differed.
    • Fix: a failed folder lookup or listing skips the run and is retried later. Sync stops only when the folder is really gone or the user declined to sign in.
    • Code: preconditions/run-sync-preconditions.ts, api.ts (getFolder)
  5. Failed responses saved as note content or file link
    • Problem: Drive error responses were used as data. A failed download replaced a note's content with an empty string and marked it synced, and a failed create or update stored a file link without an id.
    • Fix: call treats an error status, a non-JSON body or a body that fails to arrive as a failed request. A failed download fails the whole run, so nothing is written or pushed, and a failed create or update keeps the note's link as it was.
    • Code: call.ts, sync/pull/pull-update.ts, sync/pull/pull-create.ts, sync/push.ts
  6. Notes with a quote in their name never uploaded
    • Problem: the note name was pasted into the request's JSON metadata unescaped, so a name like Say "hi" produced invalid JSON and Drive rejected the upload. Combined with the stored link without an id, the note looked synced but was never in Drive. Image uploads had the same problem with the dropped file's name.
    • Fix: request metadata is serialized with JSON.stringify. Notes already affected recover on the next sync, as their broken link is dropped and the file is created.
    • Code: bodies.ts
  7. Duplicating a note made it overwrite the original
    • Problem: a duplicate copied the original's Drive file link, so both notes shared one file and each sync wrote whichever was edited last over the other, in Drive and then locally.
    • Fix: a duplicate starts without a file link and gets its own file. Notes that already share a file are repaired on the next sync: the original, created first, keeps the link and every other note gets its own file.
    • Code: sync/pull/pull-unshare.ts, sync/pull.ts, plus notes/state/duplicate-note.ts outside the sync folder
  8. Drive files sharing a name overwrote or kept growing a note
    • Problem: when a file was renamed in Drive to the name of another local note, pull wrote the renamed note over that note. And with two Drive files of one name, an unlinked note was merged with each of them in turn, appending text again on every sync.
    • Fix: the renamed note keeps its old name when the new one is taken, and a note is matched to a file by name only while it has no file link, so a second file of that name is left alone.
    • Code: sync/pull/pull-update.ts
  9. Edited note whose Drive file was deleted never synced again
    • Problem: the note was kept because it had local edits, but stayed linked to the deleted file, so every push failed against it.
    • Fix: pull unlinks such a note, so push creates a new file for it.
    • Code: sync/pull/pull-delete.ts
  10. Stopping sync during a run was undone
    • Problem: a running sync wrote the sync settings and every file link back after the user stopped it.
    • Fix: sync checks that it is still enabled before writing and writes nothing otherwise.
    • Code: sync/sync-notes.ts
  11. Sync stuck "in progress" after a failure
    • Problem: an HTML error page or a dropped connection threw out of sync() without clearing its in-progress flag, so every later sync was refused until the extension restarted.
    • Fix: such responses read as failed requests, and any failure during a run ends it through the failure handler, which clears the flag.
    • Code: call.ts, sync/sync.ts
  12. Duplicate "My Notes" folder when enabling sync
    • Problem: a failed folder lookup was taken as "no folder", so enabling sync created a second "My Notes" folder.
    • Fix: a failed lookup aborts enabling sync instead, and each step that stops it logs why.
    • Code: sync/initiate.ts

To test a sync run end to end, its steps moved from sync() into syncNotes in sync/sync-notes.ts, which receives Google Drive and storage as arguments. sync() keeps the in-progress flag and the messages to the tabs.

Known limitations

  • A save from the editor landing in the milliseconds between the re-read and the write is still overwritten, as chrome.storage has no transactions.
  • A keystroke saved before the tab has processed the sync result can still write the older notes back.

Test plan

  • Unit tests for fixes 1 and 3 to 9, and end-to-end tests of syncNotes with Drive and storage stubbed for fixes 2, 3 and 10
  • npm run lint, npm run test, npm run build pass, on every commit
  • 1: with one file per listing page forced, editing one note keeps the other synced notes
  • 2: with a 15 s delay inside the sync, text typed during it is still there after reload
  • 2: a note edited in Drive and locally during the same sync ends up with both texts
  • 3: after a two-sided merge, the Drive file holds the merged text right after that sync
  • 4: with every listing failing, a sync run fails without turning sync off
  • 6: a note named "hi" appears in Drive
  • 7: a duplicated note gets its own file, and a duplicate made before the fix is repaired
  • 8: a file renamed in Drive to another note's name leaves that note untouched, and two files of one name merge into an unlinked note only once
  • 9: a note edited after its file was deleted in Drive gets a new file
  • 10: stopping sync during a delayed run leaves sync off
  • 11: after a failed run, the next sync starts instead of reporting "already in progress"
  • 12: enabling sync while folder lookups fail creates no second "My Notes" folder
  • CI passes

penge added 13 commits October 22, 2026 18:45
- Request `pageSize=1000` and follow `nextPageToken` until the listing
  is complete; reading only the first page made every synced note whose
  file was not on it look deleted in Google Drive, and pull removed it
- Return `undefined` when any page cannot be retrieved, so a partial
  listing is never taken for the full one
- Add `listFiles` tests for one page, several pages with an empty one in
  between, and a failed page

Signed-off-by: penge <ppenge@gmail.com>
- Treat an error status, a body that is not JSON, and a body that fails
  to arrive as a failed request in `call`, so an error page or error
  JSON is never read as a file's content or metadata, and an HTML error
  page no longer throws out of sync
- Read the request outcome from `ok` in `deleteFile`
- Add `call` tests

Signed-off-by: penge <ppenge@gmail.com>
- Fail the sync run when pull cannot download a file, instead of storing
  an empty note marked as synced; acting on the rest of the pull could
  push local content over a remote edit that was not fetched
- Keep a note's file link as it was when its file cannot be created or
  updated, instead of storing a link without an id
- End the sync run on any failure, so sync is never left marked as in
  progress
- Add tests for a failed download in `pullCreate` and `pullUpdate`, and
  for a failed create or update in `push`

Signed-off-by: penge <ppenge@gmail.com>
- Tell a failed folder lookup from a missing folder in `getFolder`
- Skip the sync run on a failed folder lookup or file listing, instead
  of stopping sync, which revoked the permission, removed the sync
  settings and unlinked every note
- Keep stopping sync when the user declines to sign in, so they are not
  asked again on every Auto Sync run

Signed-off-by: penge <ppenge@gmail.com>
- Stop enabling sync when the lookup of the "My Notes" or "assets"
  folder fails, instead of taking it as a missing folder and creating a
  second one
- Log why enabling sync stopped at each step that can end it

Signed-off-by: penge <ppenge@gmail.com>
A note modified since the last sync is kept when its file is deleted in
Google Drive, but stayed linked to that file, so every push failed
against it and the note was never synced again. Pull now drops the link,
so push creates a new file for it.

Signed-off-by: penge <ppenge@gmail.com>
- Keep a note's name when its file is renamed in Google Drive to the
  name of another local note, instead of writing it over that note
- Match an unlinked note by name only while it has no file link, so a
  second file with the same name is no longer merged into the note on
  every sync
- Add tests for a rename onto a used name and for two files with one
  name

Signed-off-by: penge <ppenge@gmail.com>
- Serialize the metadata of file create, update and image upload
  requests with `JSON.stringify`, so a note or image name with a quote,
  backslash or line break produces valid JSON; Google Drive rejected
  such requests, and the note was never uploaded
- Add a test that parses the metadata for a name with a quote, a
  backslash and a line break

Signed-off-by: penge <ppenge@gmail.com>
- Leave out the original's file link when duplicating a note, so sync
  uploads the duplicate as a new file instead of writing it over the
  original's file
- Unlink notes that already share a file with an older note before
  pull, keeping the link on the note created first, so existing
  duplicates get their own file on the next sync
- Add tests for the duplicate without a link, created later than its
  original, and for unlinking shared links

Signed-off-by: penge <ppenge@gmail.com>
- Re-read the notes after pull and push and merge the sync result into
  them per note instead of writing the notes read before sync back;
  typed text, lock, pin, raw, renames, deletions and new notes made
  meanwhile were lost or reverted
- Match notes by their Google Drive file across renames, keep changes
  made meanwhile, and append content pulled for a note edited meanwhile
- Delete files that push created for notes deleted or renamed meanwhile
- Set `lastSync` to the time the notes were read, so Auto Sync pushes
  edits made during the sync next
- Copy each note in `push` before linking it, so the notes read before
  sync stay as they were
- Add merge tests, including one through the real `pull` and `push`

Signed-off-by: penge <ppenge@gmail.com>
Stopping sync removes the sync settings and unlinks every note. A sync
still running wrote both back when it finished; it now checks that sync
is still enabled and writes nothing otherwise.

Signed-off-by: penge <ppenge@gmail.com>
When pull merged a note with its file, it stamped the note with the
file's time, so the note looked synced and push skipped it. The merged
content stayed on this device only, and the next edit of the file
elsewhere replaced it.

- Stamp a merged note 1 second after the later of the note's and the
  file's time, so push uploads the merged content in the same sync,
  whatever this device's clock says
- Use real dates in the merge tests of `pullUpdate`
- Add a sync test that runs the real `pull` and `push` and checks the
  merged note is uploaded

Signed-off-by: penge <ppenge@gmail.com>
- Move `syncNotes` into its own file, with Google Drive and storage
  passed in as `drive` and `storage`, and call `onDone` from `sync`
  instead, so `sync` alone manages the in-progress flag and the
  messages to the tabs
- Test `syncNotes` through its real steps: a merged note is uploaded
  and stored, text typed during the sync is kept, a file created for a
  note deleted meanwhile is deleted while a pulled one is kept, and a
  sync stopped meanwhile writes and deletes nothing
- Stub storage with the reads each test expects, in order, each with
  the key and the value storage holds for it, and fail on a read out of
  that order or one that never happens
- Remove the tests that repeated the steps of a sync run themselves,
  covered now by the `syncNotes` tests

Signed-off-by: penge <ppenge@gmail.com>
@penge penge self-assigned this Oct 23, 2026
penge added 3 commits October 24, 2026 18:39
A note edited here while the same sync pulled an edit from another
device was stamped with this device's clock. When the other device's
clock was ahead, the note looked older than its file, so push skipped
it, and the next sync merged the other device's text in a second time.

- Stamp a merged note later than both now and its file's time plus 1
  second, with one `newerThanFile` used by pull and by the merge after
  pull and push, so push uploads it whatever the devices' clocks say
- Pin "now" in the tests that check a merged note's time
- Add a test with the other device's clock 5 minutes ahead

Signed-off-by: penge <ppenge@gmail.com>
A note duplicated before duplicates dropped their file link still shares
that file with its original during the sync that separates them. The
merge matched an edited note to its counterpart by file first, which
could pick the other note of the pair: an edited original got its own
old text appended, and an edited duplicate took the original's content
and lost its new file.

- Match a note whose name exists before and after sync to the note of
  that name, and match by file only for notes renamed meanwhile
- Add tests for editing the original and for editing the duplicate
  during that sync

Signed-off-by: penge <ppenge@gmail.com>
A merged note is stamped at the later of now and its file's time plus 1
second. Each test title now says which of the two its expected time
comes from, so a failure points at the case that broke.

Signed-off-by: penge <ppenge@gmail.com>
@penge
penge merged commit 75a157a into master Oct 24, 2026
1 check passed
@penge
penge deleted the fix/sync-data-loss branch October 24, 2026 17:12
This was referenced Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant