Skip to content

fix(iterator): keep exhausted object iterators from fetching again - #2224

Merged
dirkkul merged 1 commit into
weaviate:mainfrom
JackyHsu-X:fix/iterator-exhaustion
Oct 24, 2026
Merged

dirkkul merged 1 commit into
weaviate:mainfrom
JackyHsu-X:fix/iterator-exhaustion

Conversation

@JackyHsu-X

Copy link
Copy Markdown
Contributor

Description

The synchronous and asynchronous object iterators do not remember reaching an empty page. After next() / __anext__() signals exhaustion, another call sends another fetch request and can even yield newly added objects. Python's iterator protocol requires subsequent next() calls to keep raising StopIteration.

Remember exhaustion for the current traversal in both iterators. Explicit iter() / __aiter__() still resets the iterator and its initial after cursor, preserving the existing restart behavior.

Testing

  • Four regressions cover initially empty and populated sync/async iterators, repeated calls after exhaustion, cached objects, forwarded query options/cursors, and explicit restart. All four fail without the fix and pass with it.
  • python -m pytest test --ignore=test/test_timeout.py -q: 543 passed, 1 skipped.
  • Ruff check/format, flake8 (including the project's lint plugins), and Pyright 1.1.400 for the two changed files passed.

Three tests in test/test_timeout.py fail on this Windows/GBK machine while reading conftest.py without an explicit encoding; the same three fail on unmodified upstream code. A combined unit/mock run was interrupted when it stopped making progress after the unit failures, so no complete mock-suite pass is claimed. Live Weaviate integration tests were not run.

AI assistance: Codex investigated the issue, wrote the patch, tests and PR description, and ran the local checks.

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@weaviate-git-bot

Copy link
Copy Markdown

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

@JackyHsu-X

Copy link
Copy Markdown
Contributor Author

I have read the Weaviate Contributor License Agreement and agree to its terms.

@dirkkul
dirkkul merged commit 169a58b into weaviate:main Oct 24, 2026
126 of 128 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants