Skip to content

feat(sync): stream daily extract downloads through core - #195

Merged
btoron merged 2 commits into
btoron:masterfrom
WilliamK112:codex/sync-daily-extract-stream-160
Oct 18, 2026
Merged

btoron merged 2 commits into
btoron:masterfrom
WilliamK112:codex/sync-daily-extract-stream-160

Conversation

@WilliamK112

Copy link
Copy Markdown
Contributor

Daily extract downloads currently buffer the entire ZIP through response.content. Add core.get_daily_extract_file_stream(date, filename, *, timeout=None) as a response context manager matching the usage in #160. Callers choose their chunk size with stream.iter_content(chunk_size=8192) and can inspect response headers for progress. The successful body stays unread until consumption, and the response closes on full or partial consumption, consumer exceptions, and HTTP failures.

Keep the existing buffered method unchanged. OFSC JSON errors use the existing OFSAPIException; non-JSON errors and unexpected status codes raise requests.HTTPError. With auto_raise=False, callers receive the raw response and must inspect its status before writing it. Document the API and the optional connect/read timeout.

Validation: 464 credential-free tests pass, 8 skip. The 17 new cases include two real loopback HTTP checks (Basic and bearer auth), error/cleanup paths, and a logical 80 MiB body where a partial read consumes only one 8 KiB chunk. Sixteen new cases fail on the previous implementation because the API is absent; the buffered API control passes both versions. Ruff check and format pass. Pyright reports the same seven existing synchronous-core diagnostics as pristine master, with no new diagnostics. No live Oracle tenant testing was performed.

The separate additionalHeaders defect reported here is addressed by #193.

Closes #160

Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com>
@btoron
btoron merged commit be654b3 into btoron:master Oct 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add streaming download support for daily extract files

2 participants