Skip to content

fix(words): match the text's pauses to the voice's silences by position, and warn when the voice ran on - #108

Merged
umiyosh merged 2 commits into
masterfrom
fix/issue-71-phrase-gaps
Nov 18, 2026
Merged

umiyosh merged 2 commits into
masterfrom
fix/issue-71-phrase-gaps

Conversation

@umiyosh

@umiyosh umiyosh commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Fixes #71. Part of #87.

Summary

The narration's phrases are now matched to the silences in the voice by position, not by silence length alone. A short silence inside a phrase (a small っ, a breath) and a pause the voice skipped in the same scene no longer push a phrase head off by up to a second. narrate names a scene whose voice ran on through a pause.

Why

charTimes split the voice at silences of 80 ms or more and joined the shortest gaps until there was one run per phrase. When a っ's closure and a skipped 、 met in one scene, the wrong gaps were joined.

How

Part What it does
Positions on voiced time Everything is placed on voiced time: the scene's time with its silences taken out
Expected pause position Each pause (、。) is expected where its share of the voiced characters falls on that time
Choosing silences A small dynamic program picks, in order, for each pause the silence nearest its expected place. A pause can also take no silence, when the voice ran on (cost 0.15 s). A silence that ends no phrase costs its length, so a long one is rarely left out.
Characters in a phrase Spread evenly over the phrase's voiced time, so a っ's silence no longer stretches the characters around it
charTiming(x, text) Also returns how many pauses the voice ran on through; charTimes is unchanged for its callers
ranOnLines narrate prints a words: the voice of "<id>" ran on through N of M pauses … line for each such scene

Test

  • New test in tests/words.test.ts, with synthetic speech that has a skipped 、 and a っ-like 0.12 s silence in one scene. Every phrase head is within 50 ms, and the scene counts one pause run on. It failed before (か was 0.52 s late).
  • The three existing charTimes tests pass unchanged.
  • New ranOnLines test: only the scene that ran on is named.
  • make check passes.

Not measured here: the TTS scenes from #59, since their recordings are not in the repo. The acceptance item "the #59 figures do not get worse" needs a run on real narration.

…-gaps

# Conflicts:
#	skills/motion-render/scripts/cli.ts
#	skills/motion-render/scripts/words.ts
#	tests/words.test.ts
@umiyosh
umiyosh merged commit 85ab2b6 into master Nov 18, 2026
1 check passed
@umiyosh
umiyosh deleted the fix/issue-71-phrase-gaps branch November 18, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

words: phrases mis-timed when a small っ leaves a silence or the voice skips a pause

1 participant