Skip to content

feat(async): stream daily extract downloads through core - #192

Open
WilliamK112 wants to merge 2 commits into
btoron:masterfrom
WilliamK112:codex/async-daily-extract-stream-161
Open

WilliamK112 wants to merge 2 commits into
btoron:masterfrom
WilliamK112:codex/async-daily-extract-stream-161

Conversation

@WilliamK112

Copy link
Copy Markdown
Contributor

Add ofsc.core.get_daily_extract_file_stream(date, filename, chunk_size=8192) to the async client, as requested in #161. It is an async context manager yielding decoded byte chunks from the shared HTTP client's streaming response, so callers can consume large extracts incrementally.

Keep the response open during iteration and close it on completion, early exit, consumer exceptions, and cancellation. Preserve existing typed API errors by reading error responses, and translate connection/download read failures to OFSCNetworkError; errors from the caller's own upload or processing code retain their identity. Reject nonpositive chunk sizes before making a request. Add a README usage example.

Fixes #161.

Validation:

  • 19 tests use real httpx response/context lifecycles with MockTransport: lazy consumption of an 80 MiB logical response with one reusable chunk, logging enabled/disabled, custom chunk boundaries, empty responses, HTTP error details, connection/read errors, cancellation, and consumer errors.
  • Full mocked suite: 466 passed, 8 skipped on Python 3.13.
  • Full Ruff check, Ruff format check (76 files), and diff checks pass.
  • Pyright reports only the same two existing get_all_activities date-default diagnostics reproduced in a pristine master@56ded86 worktree; it reports no diagnostics in the new method.

Validation uses synthetic streams and no tenant credentials; a live OFS extract download has not been run.

WilliamK112 and others added 2 commits October 13, 2026 03:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add async streaming download support for daily extract files

2 participants