Repository navigation
Built-in engines W29: e-book and markup writers, UYAP UDF writer, interactive whiteboards - #1723
Merged
Merged
Conversation
The built-in e-book engine now writes the older formats it already read, from the document model the other writers use: - PDB as PalmDOC: 4 KB PalmDOC-compressed text records, the PalmDOC header, the title as database name, a bookmark record per heading - TCR: a 256-entry dictionary (every byte used plus frequent pairs merged on a sample), the text coded by longest match - PML: ASCII with \aNNN / \UXXXX, chapters and sub-headings, styles, alignment, list markers, links and anchors, footnotes, PNG pictures in the <name>_img folder reserved through the output allocator - RB: the XHTML page in zlib chunks, the info page, PNG pictures - SNB: book, contents and chapter XML, bzip2 text blocks (own encoder), file table and block tail Text is Windows-1252 when it fits, else UTF-8; the RB reader now reads an info page in UTF-8 too. Each writer stays within its reader's limits, checks for cancellation and writes through a temporary file. The edges cost 19 like AZW3 / MOBI, so calibre ranked first still takes EPUB to these formats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- LatexWriter, AsciiDocWriter, TypstWriter and ManWriter on the shared MarkupTextWriter walk: headings, emphasis, nested and numbered lists, quotes, code, tables with merged cells, links, footnotes, pictures, rules, and every special character of each language escaped. - MarkupConverter.WriteOnly offers every built-in reader's format into tex, adoc, typ and man at cost 8, below Pandoc's 11; reading them stays Pandoc's. Output goes through a temporary file; the walk checks the conversion's cancellation between blocks. - TexMathWriter writes the equation tree (Word math, 한글 equations, MathML) as TeX for LaTeX, AsciiDoc and Markdown, so Markdown output no longer writes linear text for them; TypstMathWriter writes Typst math. Trees deeper than 256 levels keep their text. - Tests build their models in code; the hostile writer tests cover the four new writers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anything Filee reads (Word, ODT, RTF, HWP/HWPX, Markdown, HTML, PDF, e-books, ...) now converts to UDF, the UYAP Document Editor's file, in one step through the new udf-writer engine. The writer works from the document model like the RTF and ODT writers and writes content.xml the way the editor does: one CDATA text with the header, body and footer, runs and pictures by offset, the page format, paragraphs with alignment, indents, spacing, line spacing, tab stops and lists, runs with font, size, bold/italic/underline/strike and colours, tables with column widths, colspan, fill, alignment and cell borders, pictures as PNG or JPEG, page breaks, one header and one footer. - Text XML can't hold is left out without moving the offsets; "]]>" splits the CDATA block. - content.xml stays within what the reader takes (part size, elements, paragraphs, objects, inflate ratio); pictures past the limits are left out; a refusal, failure or cancellation leaves no file. - UDF becomes writable. The reader now reads lists (Numbered/Bulleted, ListId, ListLevel), header rows, row heights and the cells' align, fillColor, borderNone and borderSpec. - UDF tables go to XLSX, ODS and CSV through Document tables; pictures of the pages come from the Filee renderer's PDF. - UDF -> folder (Extract) is taken only from a .udf itself, not from a document written as UDF on the way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… ODP and PPTX (#1665) Filee's own reader of interactive whiteboard files (IEC 62874 Common File Format): the pages of content.xml's SVG (pageSet, also written pageset) become slides of the drawing's size, with what the svg holds outside the pageSet on every page and iwb:element invisible elements hidden. SMART Notebook files (a page per SVG file, in imsmanifest.xml's order) are read the same way. The SVG is read into the slide model by SvgSlideReader: shapes and paths with their transforms applied, fills, strokes, gradients, text and textArea, XHTML text in foreignObject, pictures from the package or data: URIs, SVG pictures, use and nested svg. The drawing engine draws them to PDF, SVG and PNG, writes ODP with the ODP writer and PPTX through the PDF slides engine. Videos, sounds and pictures it can't decode are named in the left-out notice. Package, XML, nesting, uses, visits and path points are bounded; hrefs never leave the package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tebook, #1665) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # docs/ENGINES.md
Where Pandoc is installed (CI), the document engines read LaTeX, AsciiDoc, Typst and man through Pandoc's AST, so a route to Markdown exists without the Pandoc converter. The test now checks what it means to: the built-in markup engine never reads these formats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 29 of the built-in engines: e-book and markup writers without calibre and Pandoc, a UYAP UDF writer, and interactive whiteboards.
E-book writers without calibre
Ebooks/Legacy/from the sharedBookmodel: PDB (PalmDOC, 4 KB records with the existing compressor, bookmark records for headings 1–3), TCR (dictionary built per book), PML (formatting, links, footnotes, pictures in a<name>_imgfolder), RB (zlib-chunked HTML, info page, PNG pictures), SNB (bzip2 text blocks with Filee's own encoder, zlib file table).RocketBookReaderreads the info page as UTF-8 when valid (was Latin-1).DocumentLimits(text, records, files), cancellation,WriterTempFile.Markup writers without Pandoc
MarkupTextWriterwalk: headings, emphasis, nested lists, quotes, code, tables with merged cells, links, footnotes, pictures (not in man), rules; full escaping per language.MarkupConverter.WriteOnly: every built-in reader's format → tex/adoc/typ/man at cost 8 (Pandoc 11). Reading these four still uses Pandoc.TexMathWriter(LaTeX, AsciiDoclatexmath, Markdown) andTypstMathWriter. Markdown now writes Word/한글/MathML equations as$…$/$$…$$and reads them back as the same equation.UYAP UDF writer (#1546, #1633)
UyapWriter/udf-writer: every built-in reader's format → UDF in one step at cost 8. Paragraphs (alignment, indents, spacing, tabs, lists), runs (font, size, styles, colour), tables (widths, colspan, fill, borders, header rows; vertical merges drawn with borders), pictures, one header and footer, page breaks, A4 portrait/landscape.]]>, invalid XML characters and\rdropped so offsets stay right; content checked against the reader's limits; very repetitive content stored to pass the reader's inflate-ratio check.UyapReadernow reads lists, cell alignment/fill/borders and header rows.udf → folderis DirectOnly, so a document isn't routed through UDF to be unpacked.Interactive whiteboards (#1665)
IwbReader(IEC 62874 / Becta IWB, OpenBoard spellings) andSmartNotebookReader(.notebook) translate the SVG pages into the slide model withSvgSlideReader: shapes, paths, gradients, text and text areas, XHTML text, pictures,use.OdpWriter(editable); PPTX through the deck's PDF andPdfSlidesConverter.usecount, visits, path points, objects and pages (error.document_too_complex); fuzz targetIwbFuzzTests..iwband.notebook. General grade count 109.Merge notes
docs/ENGINES.md: the E-books row from the e-book branch, the Drawings row from the whiteboard branch.Tests
Run locally on the merged branch:
dotnet format --verify-no-changes --severity warnclean on Engines, Core and their tests🤖 Generated with Claude Code