Skip to content

analysis.json stores declaration text per callable; python and typescript store it once per module #215

Description

@rahlk

Is your feature request related to a problem? Please describe.

The three analyzers disagree on where a declaration's text lives in analysis.json.

JSON Neo4j code
python module source once + span.bytes per node; no per-callable code derived at projection (neo4j/project.py::_span_code)
typescript module source once + span.bytes per node; no per-callable code derived at projection (matching python)
java Callable.code stored per callable (entities/Callable.java:64) carried straight through

Python and typescript store a module's text once and address every node into it by UTF-8 byte span. Java stores each declaration's text again on the declaration. For a file with N declarations, java's JSON carries the overlapping text N+1 times.

Python's projection documents the model (neo4j/project.py:697-704):

"A declaration's text: the owning module's source sliced by the node's utf-8 byte span. Schema v2 stores source once per module, so the graph's code property … is derived here at projection time (#104)."

typescript has just adopted the same approach for its graph, so java is now the only analyzer whose JSON duplicates declaration text.

Describe the solution you'd like

  • Decide whether java adopts store-once-plus-span, or python/typescript adopt store-per-node
  • If java adopts it: JModule (or the java equivalent) carries source; nodes carry span.bytes; Callable.code is dropped from the JSON model
  • The Neo4j projection derives code by byte-slicing, as python and typescript do
  • Measure the payload difference on a large repository before and after

Describe alternatives you've considered

Not stated in the original issue.

Additional context

Scope boundary

Where declaration text lives in analysis.json. Not in scope: the Neo4j code property on :JCallable, which should stay — all three analyzers agree the graph carries code directly, and this issue would only change where java derives it from.

Caveats and known risks

  • Breaking for analysis.json consumers. Anything reading Callable.code would need to slice instead. The java SDK and any downstream tooling are affected.
  • Byte spans, not character spans. Python and typescript slice UTF-8 bytes (source.encode()[lo:hi]). Java strings are UTF-16, so a naive substring would cut multi-byte characters. The span contract has to be explicitly byte-based.
  • Java may already carry line/column spans but not byte offsets — if so, the analyzer has to start emitting them, which is the bulk of the work.
  • This may not be worth doing. Storing text per node is redundant but simple and directly readable. The counter-argument is only that three analyzers emitting the same schema should not disagree about where text lives; if the duplication is acceptable, the better fix might be for python and typescript to document the divergence rather than for java to change.

Definition of done

Either all three analyzers store declaration text the same way in analysis.json, or the divergence is recorded as a deliberate, documented decision in the canonical schema rather than an accident.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions