A WPF viewer for text files of tens of gigabytes. BigTextReader never loads the file into memory. It locates lines through a sparse index and reads them on demand. The ceiling is disk space, not RAM.
The index holds the byte offset of every thousandth line. For the 53.7 GB test file, that is about 2 MB of offsets. One offset for each of its 255,652,807 lines would need 1.9 GB instead. Reaching a line between two checkpoints means seeking to the preceding one and scanning forward from there.
Because the assignment rules out off-the-shelf editor components, the view is a custom control. It
implements IScrollInfo and draws only the lines that fit on screen, around 20 in a default-sized
window, which makes a repaint cost the same whatever the file size.
Scrolling works in pixel offsets rather than line numbers, so a line can sit half off the top edge.
Horizontal scrolling slices the line string, and a very long line stays as cheap to draw as a short
one.
BigTextReader reads UTF-8, with or without a byte order mark. ASCII is a subset of UTF-8 and works unchanged. The file extension doesn't matter.
LF and CRLF line endings both work. The decoder strips the carriage return before it draws a line.
| Input | Result |
|---|---|
| UTF-8, no mark | Read normally. |
| UTF-8 with a byte order mark | Read normally. The reader skips the 3-byte mark. |
| UTF-16 or UTF-32, either byte order | Refused. The status bar reads Unsupported BOM - only ASCII/UTF-8 is supported. |
| Another 8-bit encoding, such as windows-1252 | Opens, and every byte outside ASCII becomes a replacement character. Nothing in the file identifies these encodings. |
| Line breaks written as carriage returns alone | Opens as one line. The scanner splits on line feed only. |
| A binary file | Opens, and the decoder produces meaningless text. less and Notepad behave the same way. |
- The view never wraps a line. A long line runs off to the right and you scroll to follow it.
- The view cuts a line longer than 256 KiB and appends a truncation marker. The file itself stays untouched, and saving it copies every byte.
- A line longer than 4 MiB leaves the rest of its 1000-line block unreachable. Those lines display as empty.
- Search stops after 262,144 matches and then reports the count with a trailing
+. - Indexing a very large file takes minutes and there's no way to cancel it. The 53.7 GB test file took 7 minutes 51 seconds. Text and the line count appear while the scan runs, which keeps the window usable throughout, and opening another file replaces the scan in progress.
The File menu offers three sources. All three end up as a file on disk that the reader indexes, and they behave identically once open. Open File reads the file you pick. The other two write a temporary file first.
| Command | Shortcut | Action |
|---|---|---|
| Open File | Ctrl+O |
Opens a file you pick. |
| Open URL | Ctrl+U |
Downloads a page into a temporary file, then opens it. |
| Generate Text | Ctrl+G |
Writes up to 1,000,000 random lines, optionally prefixed with the line number, then opens the result. |
| Save File | Ctrl+S |
Copies the open file byte for byte to a path you pick. |
| Find | Ctrl+F |
Shows the search bar and selects whatever is in it. |
BigTextReader deletes the temporary files it created when you close it.
Searching scans the whole file. It starts when you press Enter or click Search, never while you type: a full pass over 53.7 GB takes minutes, and searching as you type would queue one of those per keystroke. The status bar shows the phase and a progress bar while a scan runs. Every match is highlighted, and the current one uses a second colour.
Enter runs the search. F3 and Shift+F3 step forward and back through the matches, and
Escape closes the search bar and hands focus back to the text. The arrow keys, Page Up,
Page Down, Home, End and the mouse wheel all scroll.
Wheel and page scrolling ease into place instead of jumping, and so does moving to a match within about three screens. Past that the view jumps. Home and End always jump, which is what other software does.
The 53.7 GB test file holds 255,652,807 lines. Indexing it took 7 minutes 51 seconds, limited by disk throughput. Afterwards, reading any line took 22 ms or less, including line 255,000,000.
Memory stays flat across the whole run: 169 MB of private bytes with the file open, against 53.7 GB on disk.
- Windows 10 or Windows 11, x64. WPF runs only on Windows.
- Nothing else. The executable carries the .NET runtime with it. There's no framework to install.
The first launch takes longer than later ones. The single file unpacks its native components into a temporary folder and caches them there.
- .NET SDK 10.0 or newer. Run
dotnet --versionto check. This build used 10.0.401. - Windows, for the same reason as above, though only the UI project needs it.
BigTextReader.Coretargets plainnet10.0, and the engine and its tests build anywhere. - Visual Studio, optionally. The solution file
TextReaderAssignment.slnxuses the newer.slnxformat, which an older Visual Studio may not open. Every command below runs from thedotnetCLI.
dotnet build
dotnet test
dotnet build builds all three projects.
dotnet publish BigTextReader.App/BigTextReader.App.csproj -c Release -r win-x64 --self-contained -p:PublishSingleFile=true -p:IncludeNativeLibrariesForSelfExtract=true -p:EnableCompressionInSingleFile=true -o publish
The result is publish/BigTextReader.App.exe, 61.9 MB.
| Project | Target | Contents |
|---|---|---|
BigTextReader.Core |
net10.0 |
Indexing, caching, search, encoding, file and URL loading. No UI dependency. |
BigTextReader.App |
net10.0-windows |
WPF shell. Holds the virtualised text view, the view model, and the dialogs. |
Tests |
net10.0 |
xUnit tests for BigTextReader.Core. |
- The filtered view. I ran out of time before starting it.
- Cancel button for ongoing operations.
- Blocks have a 4 MiB limit. Once a block hits it, the remaining lines in that block are omitted.
- Allow more encodings.