Skip to content

locate misses a position on a decorator line and reports it as module scope #408

Description

@rahlk

Describe the bug

_LOCATE_QUERY tests plain span containment (cldk/analysis/python/neo4j/neo4j_backend.py:2066):

"AND c.start_line <= pos.line AND pos.line <= c.end_line "

PyCallable.start_line is the def line. codeanalyzer-python sets it from ast.FunctionDef.lineno (codeanalyzer/syntactic_analysis/symbol_table_builder.py:342), and Python's AST puts decorators outside that line number. A position on a decorator therefore sits above every callable span. _locate_result finds no best_row and returns the module_scope diagnostic (neo4j_backend.py:2128). Line 180 of a routed Odoo controller is not module scope. It is a decorator applied to get_note_details.

The local backend uses the same predicate. See _find_innermost (cldk/analysis/python/codeanalyzer/codeanalyzer.py:325). test_locate_parity_* forbids the two backends from diverging, so the fix belongs in both.

The graph already carries the answer. Measured on a merged Odoo graph of 6,081 modules: 10,241 PY_DECORATED_BY edges, and every one carries a start_line. get_note_details has c.start_line = 181, and its decorator edge has start_line = 180 with expression = http.route('/crm/notes/data/admin_note_details/<int:note_id>', type='jsonrpc', auth='user'). An analyzer that reports on the auth= argument fires on line 180.

The payload carries it too. A decorated method in analysis.json keeps the applied position:

start_line: 6   code_start_line: 7
decorators[0].name: functools.lru_cache
decorators[0].span: {"start": [5, 5], "end": [5, 35], "bytes": [33, 63]}

One disjunct answers the position exactly:

OR EXISTS { (c)-[r:PY_DECORATED_BY]->()
            WHERE r.start_line <= pos.line AND pos.line < c.start_line }

A consumer already works around the gap with a heuristic. osprey's _decorator_fallback takes the nearest callable starting after an unplaced line. It admits that callable only when the callable carries decorators, and it allows 1 + 3 * ndec lines of slack. Of 1,124 unplaced positions, 300 sat one line above a decorated callable, and none sat above an undecorated one. Both backends can read the same fact exactly.

To Reproduce

Not stated in the original issue.

Expected behavior

  • A fixture with a decorated method resolves every line from its first decorator through end_line to that method. The test names the exact expected callable signature.
  • The same fixture returns equal LocateResult.callable and LocateResult.type on both backends, added to test_locate_parity_*.
  • A decorated nested function resolves its decorator line to the nested callable, named exactly.
  • A blank line between two top-level callables still returns the module_scope diagnostic.
  • A position on a class decorator still returns module_scope, asserted so the out-of-scope decision is visible rather than assumed.
  • A graph without PY_DECORATED_BY.start_line returns exactly what it returns today.
  • The 40-position locate benchmark is measured before and after, and the number is in the source.

Additional context

Scope boundary

Widens the containment test in the two Python backends. Nothing else. Does NOT change LocateResult's shape, the innermost-callable tie-break, the body-node selection, or the module_scope and file_not_in_graph diagnostics for a position that really is outside every callable. Does NOT touch the Java or TypeScript backends. Does NOT add a nearest-callable fallback: a position with no containing span and no decorator over it still reports module scope. Does NOT cover a class decorator, because a result with type set and callable unset is a new shape and needs its own decision. Such a position keeps today's behaviour.

Goals

  • On the Neo4j backend, locate places a decorator-line position on the callable the decorator applies to, using PY_DECORATED_BY.start_line
  • On the local backend, locate does the same, using PyCallable.decorators[].span
  • A position between two decorators of one callable resolves to that callable
  • A decorator on a nested callable resolves to the nested callable, not to the enclosing one
  • diagnostics is empty for a resolved decorator position
  • A graph built by codeanalyzer-python 1.5.1 or earlier keeps today's behaviour, and raises nothing
  • test_locate_parity_* covers a decorator position, and both backends agree
  • The measured locate latency is recorded in the comment above _LOCATE_QUERY

Caveats and known risks

  • The Neo4j half depends on a property that arrived in codeanalyzer-python v1.5.2. PR fix(neo4j): the graph projection carries the facts analysis.json carries codeanalyzer-python#206 added **_span_props(decorator.span) to PY_DECORATED_BY in _project_decorator, and codeanalyzer/neo4j/schema.py declares it. Incorporate codeanalyzer-python v1.5.2 #396 tracks the pin. Graphs from 1.5.1 or earlier carry no span there, so the new disjunct matches nothing and behaviour is unchanged. Tolerate the absence. Do not require the property.
  • code_start_line is not an alternative. codeanalyzer sets it to child.body[0].lineno (symbol_table_builder.py:386), the first body statement. That is later than def, not earlier.
  • A decorator span starts at the decorator expression, not at the @. For @functools.lru_cache(maxsize=8) the span starts at column 5 on the @'s own line. A line-level test is unaffected. A column-level test would not be.
  • The new disjunct adds a relationship expansion per candidate callable. The note above _LOCATE_QUERY records 46 ms for 40 positions on the 1.4.1 graph. Re-measure, keep the :PySymbol prefix seek, and record the new number rather than assuming the cost is free.
  • A nested decorated function's decorator line already lies inside the enclosing callable's span. The existing width-then-signature tie-break picks the inner callable, because the inner span is narrower. Assert that, rather than trusting it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions