Skip to content

Phase 5c: tracing and metrics — the tracing and metrics layer - #66

Merged
Wahbeh-Mohammad merged 4 commits into
mainfrom
20-phase-5c-tracing-and-metrics
Sep 18, 2026
Merged

Wahbeh-Mohammad merged 4 commits into
mainfrom
20-phase-5c-tracing-and-metrics

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Part of #20. Fourth PR of the nine-PR phase-5 stack — phase 5c's code. Targets phase 5a's docs PR below it; its tests are the next PR up and the phase record the one after.

Base note. Phase 5c was built and reviewed in parallel with phase 5a, both cut from main at 993c439, and its three branches were then rebased onto 5a's docs tip d5df0ca by a reconciliation agent: nine collisions — the entry file's two phase-5 blocks, kept in sub-phase order; the smoke suite's two layer pins; the manifest, regenerated on the combined tree to 856 = 730 + 84 + 42 rows and byte-identical to git's auto-merge; and six prose files, both lanes' content kept with every count re-derived — each resolved inside the 5c commit that caused it, commit messages unchanged. Every 5c-only file is byte-identical to the reviewed tip 093e825 (git diff 093e825 e1094a8 -- <file> is empty for all of them), and the three rebased tips were re-proven in full (all seventeen gates, the matrix on all four Rubies, 5c's independence test and 5a's hermeticity checks on the combined tree). Because 5c's scope.rb and tracing.rb require 5b's diagnostics.rb for the two diagnostic-context key names, this was the 5c-first execution the 5c plan's Global Constraints anticipate: 5c created that file holding exactly the three constants 5b's design fixes — Diagnostics::TRACE_ID = :"trace.id", SPAN_ID = :"span.id", DEFAULT_KEYS — and nothing else, and 5b's Task 6 extends it three PRs up (P5-71).

What lands

The tracing and metrics half of chapter 15 in dexpace-core — 25 files, +1,131 / −30: the span and scope protocols behind phase 4a's three published singletons, the current-span carrier and the log-correlation scope, OBS-27's trace-id generation, the HTTP-shaped tracer vocabulary with its ordering contract, and the metrics SPI with its no-op meter. OBS-21–OBS-23, OBS-25–OBS-33. The only plain require added is securerandom (allowlisted); the gemspec keeps zero add_dependency lines; no rescue exists in any 5c lib file (OBS-30's asymmetry — tracer and meter calls are never wrapped); no method in the segment takes a ** splat (OBS-25; phase 0's Dexpace/NoKeywordSplat cop, verified live).

  • Phase 4a's handshake, honoured item by item. NO_SPAN's private class gains exactly the seven protocol methods (#recording? false, #set_attribute/#add_event/#record_error returning self, #status=, #finish idempotent, #context → Bundle::NONE), every one working on the frozen receiver phase 4a published, so the object keeps its identity; NO_TRACER gains #start_span/#in_span handing back NO_SPAN; NoTracerFactory#tracer keeps its five-argument shape verbatim (P4-8). _Span and _Tracer are widened in place in 4a's sig/ files. Bundle.members is the same eight before and after, #valid? stays derived, Bundle gains #sampled? only (the low bit of trace_flags), TraceIdFlavour stays a Data with .of, NONE's sentinel unchanged — no second no-op span, tracer, factory or meter anywhere in lib/, pinned by the smoke suite's constant list. Phase 4a's no_span_test.rb/no_tracer_test.rb "responds to nothing" pins are re-pinned to the exact protocol here (a necessary consequence of widening; the two suites must be green at this tip).
  • TraceIdFlavour#generate_trace_id(generator = ::SecureRandom) — W3C draws #hex(16), DATADOG #random_number(1 << 64) rendered decimal, NONE returns the invalid sentinel by identity; a zero draw is coerced ("0"*31 + "1", 1) through the injected generator seam because the branch is unreachable by sampling. XCUT-21's CSPRNG path, deliberately a separate code path from CFG-32's PRNG (charter boundary 8). No span-id generator ships (P5-44).
  • Dexpace::Instrumentation::Scope and NO_SCOPE — a plain three-ivar class holding the previous span on the Ruby call stack, never a stack in fiber storage (R13, P5-46); NO_SCOPE is returned on an identity test (the span being activated is already current) and not on the recording flag (P5-47). Scope.build is public surface marked @api private (P5-74).
  • Dexpace::Instrumentation::Tracing (extend self) — .current_span, .activate, .with_span, .correlate, .with_correlated_span. correlate pushes the bundle's ids under Diagnostics::TRACE_ID/SPAN_ID into Fiber[] (never Thread.current[] or ContextStore — charter boundary 14; a child thread, fiber and enumerator all see the pushed keys), skips the push for a non-recording span or an invalid bundle and delegates to plain activation, and restores each key to its prior value as one branchless assignment in the same ensure that restores the span (R12, P5-49). The floor's residual is recorded, not worked around (P5-72): Fiber[:k] = nil deletes the key only from Ruby 3.3 — on 3.2.11 it retains a nil-valued key, and 3.2 offers no removal but the warned Fiber#storage=, which core never calls — so OBS-23's "remove it if previously unset" is a removal on 3.3+ and a present-and-nil key on the floor, unobservable at Fiber[] and at OBS-10's null-skipping fold; the suite asserts what each row can honour and pins the 3.3.0 boundary.
  • Dexpace::Instrumentation::HTTPTracer and ::NULL — a module of exactly eleven no-op methods in OBS-28's three groups with the arities phases 6 and 8 emit against pinned; NULL the frozen instance of a private class including it. CallableAdapter — §8.1's bus shape, forwarding each callback as its name plus a payload keyed by parameter name to one #call(name, payload) object; refuses a non-callable at construction (P5-75). OBS-29's ordering contract ships as the vocabulary plus an ordering test over a conformant emitter fake; nothing in phase 5 emits it — the per-attempt half waits on phase 6a's retry step (its Task 9), the rest on phase 10's inbound list (R14).
  • NO_METER — #create_counter/#create_histogram return one shared frozen instrument each; #add/#record return nil with no hot-path validation (OBS-33); the meter and both instruments are private_constant classes behind the public constant; _Meter/_Counter/_Histogram declared filled for 5b's step to type against.
  • sig/ mirrors lib/ one file per file (110 == 110); sig/dexpace.rbs unchanged. The surface manifest regenerated once (730 → 772, all 42 rows read against the object model; the private no-op classes appear only as the value type of their constants' rows). The core smoke suite's constant list is on this branch because the code change invalidates it.

Decisions taken in the open, against the plan's text

Ledger rows P5-71–P5-76 and the checklist's "Deviations from the plan" itemise them: diagnostics.rb shipped early (P5-71); the floor residual (P5-72); the four recording doubles namespaced Dexpace::Recording* against the tree's top-level convention, because 5b's plan consumes three of them by exactly those names (P5-73); Scope.build public (P5-74); CallableAdapter refusing a non-callable (P5-75); NO_TRACER#in_span without a block returning NO_SPAN, its RBS declaring the block optional on the private class while _Tracer keeps it required (P5-76). Tracing is extend self (Style/ModuleFunction), and its RBS types the span as _Span & Object so Steep accepts the identity comparison.

Layering

Each tip of the stack is green under every gate on its own tree. This branch is green on the SimpleCov floor too — 97.86% on 4.0.6 at the reviewed tip and 98.14% at this rebased tip (1,763 runs, 0 failures; the matrix set green on 3.2.11). The tests PR takes the same tree to 99.97%.

Verification

  • Independent reviewer, approve on round 0 — 0 blocking / 0 should_fix / 2 nits, having run the seventeen gates individually at the reviewed tip on 4.0.6 (honest RuboCop 317 files clean), the full run at the tests and docs tips, the matrix set on 3.2.11 (code and tests tips) and 3.4.10 (tests tip), three extra seeds on 4.0.6 and one on 3.2.11 with identical run counts. After the rebase onto 5a the reconciliation agent re-ran all seventeen gates individually at this tip (98.14% coverage; honest RuboCop 356 files clean) and the matrix set on 3.2.11.
  • Handshake in a fresh process, both interpreters: Bundle.members the same eight; Bundle::NONE.span.equal?(NO_SPAN) and .tracer_factory.equal?(NO_TRACER_FACTORY); every #tracer argument shape returns NO_TRACER; sentinels "0"*32 / "0"*16, valid? false; all singletons frozen; one no-op class of each kind.
  • Mutations. 26 run red by the implementer (11 on 3.2.11 too), 28 applied by the reviewer, every one caught: #recording? true; a mutator returning nil; #finish raising twice; a **attributes splat (caught by the cop and the allocation test); one allocation on the no-op path; the factory minting a fresh tracer; a fresh Scope for the current span; no restore on raise; Thread.current[] as carrier; a prior value not restored; an unset key left present; a push for a non-recording span; a rescue inside Tracing; #sampled? reading bit 2; W3C uppercase / all-zero, DATADOG "0" / above 2⁶⁴−1, NONE not the sentinel; a non-nil HTTPTracer default; a wrong adapter name; started-after-succeeded and exhausted-without-failed orderings; a counter #add allocating.
  • By experiment on both interpreters: OBS-25 as a two-loop GC.stat delta of exactly 0.0 per call over 22 no-op call shapes; nested RecordingSpan activation restoring after a raise; correlate's push, restore and removal (absent on 4.0.6, present-and-nil on 3.2.11, as recorded); 10,000 W3C and 10,000 DATADOG draws all valid and unique, 8 threads × 1,000 concurrent; a raising tracer and a raising meter propagating unwrapped; Task 11's independence subprocess loading the ten 5c files with Event, Logger, Configuration and Clock undefined.

Known follow-ups from the review (not blocking)

  • R0-2 — tracing_matrix_facts_test.rb calls Fiber#storage= in one fact-pin test, deliberately, wrapped in WarningCapture with the prior map restored; no call anywhere in lib/. Acceptable as a fact pin; could move into a subprocess if the letter of "never in test" is wanted.
  • P5-72's reach beyond 5c: 5b's union restore (P5-23) and 8b's ASYNC-9/ASYNC-11 restore are written on the 3.4.10-only "= nil deletes" fact; routed as a new docs/knowledge/notes/observability.md entry and the roadmap's thirty-ninth inbound bullet. On 3.2 a per-key restore returns a worker to a map of nil-valued keys, not an empty map.
  • Scope.build is NFR-4-locked public surface at the first tag (P5-74) — flagged for the human consolidating the ledger.
  • The brief's expectation that removing the diagnostics line from lib/dexpace.rb yields a LoadError did not hold: scope.rb and tracing.rb require_relative it themselves, so the entry-file line is the explicit-requires convention rather than load-bearing.
  • The design's keyword-splat finding is verified closed as built (phase 0's Dexpace/NoKeywordSplat, scoped to gems/*/lib); the tracer-factory name collision stays on phase 10's inbound list.

@Wahbeh-Mohammad
Wahbeh-Mohammad changed the base branch from 18-phase-5a-configuration-docs to main September 18, 2026 08:51
Phase 5c (issue #20), the first of four code units. A 5c-first
execution: phase 5b is not running and scope.rb and tracing.rb read
its two diagnostic-context key names, so this creates
lib/dexpace/instrumentation/diagnostics.rb with exactly the three
constants 5b's design fixes -- TRACE_ID = :"trace.id",
SPAN_ID = :"span.id" and DEFAULT_KEYS, OBS-10's "exactly
{trace.id, span.id}" -- and nothing else of 5b's; its Task 6 extends
the file and creates no second one (P5-71). Symbols, not Strings:
Fiber.current.storage hands keys back as Symbols, Fiber#storage=
refuses a String key on every supported Ruby, and on 3.2 and 3.3 the
per-key Fiber[]= refuses one too (measured 2026-09-17; the interning
the corpus records arrives at 3.4).

TraceIdFlavour#generate_trace_id(generator = ::SecureRandom) is
OBS-27's generation over phase 4a's flavour Data, one branch per
flavour on `case name` (P5-50): W3C draws #hex(16), 32 lowercase hex
by construction; DATADOG draws #random_number(1 << 64) rendered
decimal; NONE returns its sentinel. A zero draw is coerced to the
lowest non-zero value -- a substitution, not a redraw, because a
redraw against an injected always-zero generator never terminates and
OBS-27's word is "coerced". The generator is a test seam answering
SecureRandom's own two methods, which a seeded Random shares; the
draw is XCUT-21's CSPRNG path, kept apart from CFG-32's (boundary 8).
No span-id generator ships (P5-44). `require "securerandom"` is the
one plain require 5c adds, already on the allowlist.

Bundle#sampled? reads the low bit of trace_flags, the one method
phase 4a reserved for phase 5, over the member it stored and without a
ninth: Bundle.members is the same eight (boundary 10).
Phase 5c (issue #20), the second code unit: the protocols phase 4a
postponed, given to the classes behind its three published singletons
in place, so every object keeps the identity 4a gave it and the RBS
interfaces are filled where 4a declared them empty on purpose.

NoSpan gains OBS-21's seven methods -- #recording? false, the three
mutators inert and returning self, #status= returning its argument
with no reader, #finish nil on every call, #context Bundle::NONE --
and NoTracer gains OBS-25's #start_span and #in_span, both handing
back NO_SPAN; NoTracerFactory is untouched (P4-8, boundary 10). Every
method works on a frozen receiver and writes no ivar, which is what
OBS-25's no-allocation clause wants; every attributes parameter is a
named optional keyword and never a ** splat (P5-42). OBS-29's 1:1
clause is read as binding stateful tracers only (P5-43).

Scope is OBS-22's handle: a plain three-ivar class holding the
previous span and the two prior diagnostic values on the Ruby call
stack, never a stack in fiber storage, because copy-on-write protects
the slot and not the object in it (R13, P5-46); #close restores all
three as one branchless assignment each (P5-49). NO_SCOPE is the
cached singleton OBS-25 requires, returned on an identity test -- the
span is already current -- and never on the recording flag, which
would leave a recording span un-restored under a non-recording one
(P5-47). The current-span slot key is a private_constant of scope.rb.

Tracing is a function module over that one Fiber[] slot: .current_span
never returns nil; .activate and .with_span are OBS-22; .correlate and
.with_correlated_span are OBS-23, pushing 5b's two keys for a
recording span with a valid bundle and delegating to plain activation
otherwise -- the invalid-bundle guard keeps Bundle::NONE's all-zero
ids out of every log line (OBS-26). Nothing here rescues: OBS-20 and
OBS-30 make a throwing tracer the caller's, and the block forms
restore through ensure regardless. On the 3.2 floor `Fiber[:k] = nil`
retains the key with a nil value, and the floor offers no removal but
the warned whole-map setter core never calls; the residual is
unobservable at every reader and is P5-72.
Phase 5c (issue #20), the third code unit. HTTPTracer is OBS-28's
event vocabulary richer than start/end: a module of eleven no-op
methods in three groups -- the operation lifecycle, the per-attempt
events and the transport milestones -- with the arities phases 6 and 8
emit against, so an implementer includes it and overrides only what
it needs and the rest stay no-ops rather than NoMethodErrors in the
caller's request path (OBS-30). NULL is the frozen instance design
§8.1 names. OBS-29's ordering is a contract on an implementer's
emitter, stated in the module's YARD and asserted by the tests PR's
ordering suite over a conformant fake; nothing in phase 5 emits any
of it, which OBS-29's own last sentence anticipates (R14) -- the
per-attempt group is phase 6a's retry step, the milestones phase 8's,
the operation triple phase 10's inbound list. CallableAdapter is
§8.1's bus shape over one #call(name, payload) object, allocating a
payload Hash per event by construction, which is why it is offered
and not the default; it refuses a non-callable at construction and
returns nil from every callback (P5-75).

NO_METER is OBS-31's default: a frozen no-op meter whose
#create_counter and #create_histogram return one shared frozen
instrument each, whatever the name, with the meter and both
instruments private classes behind the public constant -- sharing is
assertable meter-to-meter under two names without naming either
instrument, the asymmetry with NO_SPAN the design argues. The counter
documents OBS-33's non-negative rule and validates nothing on the hot
path; the histogram discards NaN and either infinity. The interfaces
_Meter, _Counter and _Histogram are declared filled here and only
here, for 5b's step to type against. No instrument name, unit or
attribute set is fixed (OBS-32 is post-v1).
Phase 5c (issue #20), the fourth code unit: the three files a gate
reads at run time, which travel with the code so the code tip keeps
every gate but the coverage floor green.

The runtime surface manifest of dexpace-core grows from 730 to 772
lines through `rake surface:regenerate`, once, and every one of the 42
added rows was read against the design's object model before it was
accepted: Bundle#sampled?; CallableAdapter and its eleven; Diagnostics
and its three constants; HTTPTracer and its eleven; NO_METER,
NO_SCOPE and NULL, each typed by its private class; Scope with #close
and .build; TraceIdFlavour#generate_trace_id; and Tracing with its
five functions as `#` rows, the shape URL and PercentEncoding take
under `extend self`. No private constant has a row -- the six no-op
classes, the two instrument singletons and the current-span key are
absent -- and NO_SPAN, NO_TRACER and NO_TRACER_FACTORY still appear
only as the value type of their constant's row, so the methods added
to their classes add no row. Nothing removed.

The core smoke suite gains the tracing-and-metrics Layers case: the
thirteen public constants under Dexpace::Instrumentation, 5b's three
diagnostics constants among them (P5-71), and the privacy of the rest
by qualified reference. Phase 4a's no_span_test.rb and
no_tracer_test.rb pinned the empty surface phase 4 published
("responds to nothing beyond Object"), which the widened classes
falsify; the two cases are re-pinned here to the exact protocol --
seven methods on NoSpan, two on NoTracer, one on the factory -- and
the tests PR adds the behaviour suites beside them.
@Wahbeh-Mohammad
Wahbeh-Mohammad force-pushed the 20-phase-5c-tracing-and-metrics branch from e1094a8 to 5b63ef0 Compare September 18, 2026 08:55
@Wahbeh-Mohammad
Wahbeh-Mohammad merged commit eb17603 into main Sep 18, 2026
5 checks passed
@Wahbeh-Mohammad
Wahbeh-Mohammad deleted the 20-phase-5c-tracing-and-metrics branch September 18, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant