Book Aligner

Book Aligner aligns a book's source and translation after their chapters have been matched and packages them into a side-by-side EPUB. It splits prose into sentences, embeds contiguous spans of up to four units, and finds the lowest-cost exact monotonic alignment.

Install

Book Aligner requires Python 3.11 or newer and uv. The lockfile fixes all dependency versions. pyproject.toml also excludes releases less than seven days old.

uv sync --frozen

The first alignment downloads the wtpsplit sat-12l-sm ONNX model. Configure these environment variables before running align:

• OPENAI_API_KEY is required whenever the embedding cache misses. The CLI loads it from .env.
• BOOK_ALIGNER_CACHE changes the SQLite cache location. The default is ~/.cache/book-aligner/embeddings.sqlite3.

The cache key combines the embedding model with a SHA-256 hash of the exact span text. Each API batch is committed as soon as it succeeds, so an interrupted alignment can reuse completed work.

Input format

The input is UTF-8 JSON. Every field shown below is required unless this section says otherwise. The validator rejects unknown fields. Book titles, authors, and IDs must be non-empty. IDs must also be unique on their side of the pair.

language records a BCP 47 language code such as en, es, or pt-BR; the CLI does not guess it. A chapter may also include non-empty chapter_title and subchapter_title fields. These titles identify chapters but are never aligned, embedded, or scored.

Blocks are ordered. Only kind: "prose" is split into sentences. Every other kind, including verse, heading, and epigraph, remains one unit. text must preserve the original whitespace because the EPUB writer reconstructs text from character offsets.

{
  "source": {
    "id": "novel-en",
    "language": "en",
    "title": "Example",
    "author": "A. Writer"
  },
  "target": {
    "id": "novel-es",
    "language": "es",
    "title": "Ejemplo",
    "author": "A. Writer"
  },
  "chapters": [
    {
      "id": "matched-001",
      "source": {
        "id": "en-ch-01",
        "chapter_title": "Chapter One",
        "subchapter_title": "Arrival",
        "blocks": [
          {
            "id": "en-p-001",
            "kind": "prose",
            "text": "First sentence. Second sentence."
          },
          {
            "id": "en-v-001",
            "kind": "verse",
            "text": "A line of verse\nAnother line"
          }
        ]
      },
      "target": {
        "id": "es-ch-01",
        "chapter_title": "Capítulo uno",
        "blocks": [
          {
            "id": "es-p-001",
            "kind": "prose",
            "text": "Primera oración. Segunda oración."
          },
          {
            "id": "es-v-001",
            "kind": "verse",
            "text": "Un verso\nOtro verso"
          }
        ]
      }
    }
  ]
}

Validation:

uv run book-aligner validate book-pair.json

Align

printf 'OPENAI_API_KEY=your-key-here\n' > .env
uv run book-aligner align book-pair.json alignment.json
uv run book-aligner --verbose align book-pair.json alignment.json

The aligner processes each chapter independently. Prose sentence IDs encode the stable block ID and a one-based sentence number. Each reference also records a zero-based sentence_index, its original block_id, inclusive char_start, and exclusive char_end. Non-prose units reference the full original block. The aligner embeds each candidate span directly; it does not average sentence vectors.

Progress bars on stderr cover segmentation, cache lookups, embedding requests, alignment, and confidence scoring. Each run also appends timestamped messages to log.txt beside the input JSON.

Review the warnings after alignment, they may require manual correction.

Output schema

The output is UTF-8 JSON. It contains:

• Source and target IDs, languages, titles, and authors.
• The exact segmentation and embedding model names.
• One entry per matched chapter, including IDs, supplied titles, calibration statistics, and ordered alignment groups.
• References, scores, and review warnings for each alignment group.

It does not copy or rewrite the book text.

The exact shape of one alignment group is:

{
  "id": "matched-001:a00001",
  "source": [
    {
      "id": "s:8:en-p-001:1",
      "block_id": "en-p-001",
      "kind": "sentence",
      "char_start": 0,
      "char_end": 15,
      "sentence_index": 0
    }
  ],
  "target": [
    {
      "id": "s:8:es-p-001:1",
      "block_id": "es-p-001",
      "kind": "sentence",
      "char_start": 0,
      "char_end": 16,
      "sentence_index": 0
    }
  ],
  "scores": {
    "cosine": 0.82,
    "normalized_semantic_cost": 0.1,
    "length_compatibility": 0.98,
    "semantic_margin": 0.12,
    "total_cost": 0.102041,
    "confidence": 0.91
  },
  "warnings": []
}

At chapter level, stats contains source_units, target_units, calibrated skip_cost, character_ratio, exact-path total_cost, and review_groups. A skipped group has an empty reference array on one side. Non-prose references omit sentence_index.

scores.cosine is the raw cosine similarity. normalized_semantic_cost is the cross-document normalized distance. length_compatibility measures the size match after adjusting for the chapter's source-to-target character ratio. semantic_margin compares the selected spans with their nearest alternatives. confidence compares the selected mapping cost with the cost of skipping all its units. It is a review heuristic, not a probability. A skipped side has null semantic scores and zero confidence.

Warnings appear on stderr and beside the affected references in the JSON. The possible values are source_skipped, target_skipped, low_confidence, ambiguous_semantics, length_mismatch, and maximum_span. Review every warned group manually. The CLI does not ask an LLM to repair it.

Alignment objective

For each chapter, the CLI evaluates every monotonic state and every (m,n) mapping where 1 <= m,n <= 4, plus one-unit insertions and deletions. The result is exact for this transition set. Whole-document aligners often prune or constrain the search instead.

Each transition cost uses:

1. Cosine distance between directly embedded contiguous spans.
2. Each span's mean distance to all spans on the other side, following VecAlign's normalization approach.
3. A chapter-level source-to-target character ratio and log2(1 + shorter/longer) length compatibility.
4. An m*n multiplier that makes a large many-to-many group pay more than smaller groups.
5. A skip cost set to the 20th percentile of the chapter's normalized one-to-one costs.
6. The nearest competing-span margin, which only affects review warnings.

BertAlign uses sentence embeddings and a two-step dynamic program based on approximate anchors. OmniAlign also combines sentence embeddings with dynamic programming. Book Aligner skips anchor and pruning passes because its chapters are already matched and small enough for exact search.

The embedding model is text-embedding-3-large at its default 3,072 dimensions, as documented in the OpenAI embeddings guide. Sentence segmentation uses sat-12l-sm, the best-scoring general -sm model in the wtpsplit model table.

EPUB

After reviewing the alignment JSON, build the EPUB:

uv run book-aligner epub books/CAdS/book-pair.json books/CAdS/alignment.json books/CAdS/CAdS.epub

The default layout is tuned for a Kindle text size 7 (roughly 16 lines by 40 characters). Source text uses the top of each page and translation uses the bottom, with a thin divider between them. Both languages share the same 16-line estimate, so one side may be longer when the pair still fits.

Paragraphs keep their original boundaries. A page may contain more than one paragraph, and a long paragraph or alignment group may continue on the next page. Each page starts the marker cycle ● ◆ ■ ▲; a fifth aligned span restarts at ●. The same marker identifies corresponding spans in both languages. ↪ marks text continued from the previous page.

Use --lines and --chars to recalibrate the estimator for a different Kindle setting.