Skip to content

Handle tuples during set()/merge() instead of crashing - #205

Open
afonsojanu wants to merge 1 commit into
dpath-maintainers:masterfrom
afonsojanu:fix-merge-tuple-crash
Open

Handle tuples during set()/merge() instead of crashing#205
afonsojanu wants to merge 1 commit into
dpath-maintainers:masterfrom
afonsojanu:fix-merge-tuple-crash

Conversation

@afonsojanu

Copy link
Copy Markdown

Fixes #189.

merge() and view() both reconstruct a copy of the source tree by
walking every path and calling segments.set() on it, and set()
assumes every intermediate container it touches can be filled in
incrementally through item assignment and extend(). A tuple breaks
that assumption: walk() still descends into it since a tuple isn't
a leaf, but set() has no way to grow a tuple in place. In practice
that means a source dict with a tuple anywhere in it crashes:

import dpath
x = {"foo": []}
y = {"foo": [("bar", "baz")]}
dpath.merge(x, y)
# AttributeError: 'tuple' object has no attribute 'extend'

There are actually two separate ways to hit this, both patched here:

  • set()'s main loop can find that a tuple is already sitting at the
    segment it needs to descend into, left there by an earlier pass over
    a shallower path in the same walk. It now swaps that tuple for an
    equivalent, mutable list before continuing.
  • _default_creator can also be asked to build a brand new container
    from a type hint that happens to be tuple (or another immutable
    container such as frozenset), which used to instantiate the hint
    literally. It now falls back to dict/list for anything that
    isn't already a MutableMapping/MutableSequence, since there's no
    way to fill an immutable container in afterwards regardless of where
    the hint came from.

Either way, the reconstructed result ends up with a list wherever
the source had a tuple, since a tuple can't be built element by
element once dpath decides it needs to add a value one segment at a
time.

I noticed while poking at this that tests/test_segments.py already
draws its property-based fixtures from two separate strategies:
random_node (which includes tuples, used for read-only walks) and
random_mutable_node (list/dict only, used for anything that calls
set()). That split looks like it was already working around this
exact gap, so I left it alone rather than widening it as part of this
fix and just added focused regression tests in tests/test_merge.py
that reproduce the reported crash directly.

Ran the full suite locally with nose2 (82 tests, all passing) and
flake8 clean.

merge() and view() reconstruct a copy of the source tree by walking
every path with segments.set(), which assumes each intermediate
container it touches can be filled in incrementally through item
assignment and extend(). A tuple breaks that assumption: walk() still
descends into it since it isn't a leaf, but set() has no way to grow
a tuple in place, so anything with a tuple value anywhere in the tree
raised AttributeError: 'tuple' object has no attribute 'extend'.

set() now swaps a tuple sitting at an intermediate segment for a
plain list before continuing to fill it in, and the type-hinted
creator does the same when asked to build a fresh container for a
segment whose source value happened to be a tuple (or another
immutable container like frozenset). The reconstructed result ends up
with a list wherever the source had a tuple, since there is no way to
build a tuple element-by-element after the fact anyway.

Fixes dpath-maintainers#189
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] AttributeError: 'tuple' object has no attribute 'extend'

1 participant