Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/en/changes/changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@
* Fix span-attached events of eBPF profiling overwriting each other. An event's storage ID was its segment ID, start time and name. Rover sends a Zipkin event with an empty segment ID, so two Zipkin events with the same name and start time, in any two traces, had one ID, and so could two SkyWalking events on different spans of one segment. Elasticsearch kept only the one written last, and JDBC, whose row key is the same ID, cannot hold both. The ID now also carries the trace ID and span ID.
* Fix JDBC trace profiling queries failing when a task has no profiled segments: the empty segment-ID list now returns an empty result without generating an invalid SQL `IN ()` clause.
* Add the `query-server` module, so the native query APIs can have their own port, apart from the ports agents report to. Set `SW_QUERY_SERVER_REST_PORT` to a port greater than 0, and GraphQL (`/graphql`), the AI agent conversation view and files (`/ai-agent/conversations/*`), and `/status/config/ttl` move from the core REST port (default `12800`) to it, with their own host, context path and TLS settings (`SW_QUERY_SERVER_REST_*`). Point the UI's `queryUrl`, `swctl --base-url` and other GraphQL clients at the new port. With the default `0`, nothing changes. Together with `receiver-sharing-server`, no agent or query traffic is left on the core ports.
* LAL: save the logs of the OpenTelemetry services that set `service.layer=OTLP_GENERAL` through the new `otlp-general` rule in `lal/default.yaml`; no rule matched that layer, so these logs were dropped. A LAL rule declaring an unregistered or missing layer (names are case-sensitive) is now skipped with a warning at boot and rejected when applied as a runtime rule, instead of loading silently under `UNDEFINED` and never running.
* LAL: a log without a layer that no `layer: auto` rule claims runs the new `fallback: true` auto rules, instead of the rules declared for `GENERAL`, and is dropped when none claims it either, so where such a log is filed is decided by a rule, not hard-coded. The new `default-fallback` rule in `lal/default.yaml` files it under `GENERAL`, as before; edit its extractor to file it under another layer. A fallback rule must set the layer, as any auto rule. A custom `layer: GENERAL` rule written for the logs without a layer must declare `layer: auto` and `fallback: true` and set `layer "GENERAL"` in its extractor; it then runs next to `default-fallback`, as it ran next to `default` before.
* LAL: handle the mini-program error logs with the `miniprogram-errors-wechat` and `miniprogram-errors-alipay` rules in `lal/miniprogram.yaml`, declared for the `WECHAT_MINI_PROGRAM` and `ALIPAY_MINI_PROGRAM` layers, replacing the `layer: auto` rule `miniprogram-errors`. The mini-program SDK v0.5.0 sets `service.layer`, which routes its error logs to these rules; it is now the minimum supported SDK version, since an older SDK's error logs carry no layer and are filed under `GENERAL` without the `error_count` metric.

#### UI

Expand Down
52 changes: 49 additions & 3 deletions docs/en/concepts-and-designs/lal.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,28 @@ Use `tag 'key': sourceAttribute("attr")` in the extractor to selectively persist
## Layer
Layer should be declared in the LAL script to represent the analysis scope of the logs.
LAL rules are routed by layer — only rules matching the incoming log's layer are evaluated.
A rule's `layer` must be a registered layer name (case-sensitive) or `auto`. A rule declaring an unregistered or
missing layer is skipped with a warning at boot, and rejected when applied as a runtime rule.

A log is routed as follows:

1. A log whose layer is set, by the agent's `LogData.layer` or the OTLP `service.layer` resource attribute,
runs the rules declared for that layer. A log whose layer is not registered is dropped.
2. A log without a layer runs the `layer: auto` rules, see [Auto layer](#auto-layer).
3. When no `layer: auto` rule claims it, it runs the `layer: auto` rules declared `fallback: true`.
4. When no fallback rule claims it either, it is dropped.

The shipped `lal/default.yaml` declares:

- `default` (`layer: GENERAL`), saving the logs that arrive with the `GENERAL` layer;
- `default-fallback` (`layer: auto`, `fallback: true`), saving the logs without a layer under `GENERAL`. Edit
its extractor to file them under another layer, for example by `sourceAttribute(...)` or `tag(...)`;
- `otlp-general` (`layer: OTLP_GENERAL`), saving the logs of the OpenTelemetry services that set
`service.layer=OTLP_GENERAL`, the layer their spans are analyzed under.

A rule written for the logs without a layer declares `layer: auto`, with `fallback: true` when it is to take
only what the other auto rules leave, and sets the layer in its extractor, e.g. `extractor { layer "GENERAL" }`.
A `layer: GENERAL` rule sees only the logs that arrive with the `GENERAL` layer.

### Inline layer declarations (`layerDefinitions:`)

Expand Down Expand Up @@ -75,6 +97,8 @@ the built-in static fields in `Layer.java` for distribution layers. See
[`Layer.java`](../../../oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java)
javadoc for the full picture.

### Auto layer

When `layer: auto` is declared, the rule matches logs where `service.layer` is absent (common for OTLP
sources that don't set this attribute). The script is expected to set the layer in the extractor:

Expand All @@ -98,15 +122,37 @@ rules:

In `layer: auto` mode, `abort` and `drop` have distinct meanings:

- **`abort {}`** — "not mine, let others try." If all auto rules abort, the log falls back to
`GENERAL` layer processing automatically.
- **`drop {}`** — "mine, but sampled out." The rule claimed the log (no GENERAL fallback) but
- **`abort {}`** — "not mine, let others try." If all auto rules abort, the log runs the fallback auto rules.
- **`drop {}`** — "mine, but sampled out." The rule claimed the log (the fallback auto rules do not run) but
chose not to persist it (e.g., sampling). Use this for rate limiting or conditional dropping
within a recognized log type.

If an auto rule claims the log (doesn't abort) but doesn't set a layer, the log is warned and
dropped at persistence time.

All the auto rules run on such a log, in no defined order, and each one that claims it saves a copy of
its own, so an auto rule should claim only the logs it recognizes and `abort {}` on the others.

A rule declaring `fallback: true` (allowed only with `layer: auto`) runs after the other auto rules, only when
none of them claimed the log. It is the catch-all for the logs without a layer: `default-fallback` in
`lal/default.yaml` files them under `GENERAL`. A fallback rule follows the same rules as any auto rule: it
must set the layer, and `abort {}` leaves the log unclaimed, which drops it. Auto and fallback rule names
share one name space.

```yaml
rules:
- name: default-fallback
layer: auto
fallback: true
dsl: |
filter {
extractor {
layer "GENERAL"
}
sink {}
}
```

## Filter

A filter is a group of [parser](#parser), [extractor](#extractor) and [sink](#sink). Users can use one or more filters
Expand Down
5 changes: 4 additions & 1 deletion docs/en/setup/backend/admin-api/dsl-debugging-lal.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,10 +210,13 @@ samples are the entry-point `input` and the terminal `output`.
# /tmp/lal-multi.yaml
rules:
- name: app-extractor
layer: GENERAL
# Agent logs carry no layer, so a rule for them is a fallback auto rule that sets one.
layer: auto
fallback: true
dsl: |
filter {
extractor {
layer "GENERAL"
tag stage: 'extractor'
tag emitter: 'demo'
tag rule: 'multi-statement'
Expand Down
15 changes: 10 additions & 5 deletions docs/en/setup/backend/backend-alipay-mini-program-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,12 @@ metrics on the dashboard come from the `miniprogram.request.duration` histogram'

## Error Logs

The LAL rule `lal/miniprogram.yaml` uses `layer: auto` mode and dispatches on the
`miniprogram.platform` resource attribute — one rule file produces both the
`WECHAT_MINI_PROGRAM` and `ALIPAY_MINI_PROGRAM` layers. For each error log, the extractor
emits a `miniprogram_error_count` counter sample, which the log-MAL rule in
`log-mal-rules/miniprogram.yaml` aggregates into the per-layer `error_count` metric.
The LAL rules in `lal/miniprogram.yaml` handle the error logs. The SDK sets the `service.layer`
resource attribute, so its logs reach the `miniprogram-errors-wechat` and `miniprogram-errors-alipay`
rules, declared for the `WECHAT_MINI_PROGRAM` and `ALIPAY_MINI_PROGRAM` layers. For each error log,
the extractor also emits a `miniprogram_error_count` counter sample, which the log-MAL rules in
`log-mal-rules/miniprogram-wechat.yaml` and `log-mal-rules/miniprogram-alipay.yaml` aggregate into the
per-layer `error_count` metric.

Error categories (`exception.type` tag):

Expand Down Expand Up @@ -147,6 +148,10 @@ value.
then handle absence differently: native segments produce a literal `-` instance entity; OTLP
logs and metrics create no instance entity at all. Per-instance dashboards are meaningful
only when the operator sets `serviceInstance`.
- **SDK ≥ v0.5.0 is required.** It sets the `service.layer` resource attribute, which routes its
error logs to the LAL rules of the mini-program layers. An older SDK sets none, so its error logs
are filed under `GENERAL` and produce no `error_count` metric. Conversely, an OAP older than this
version has no LAL rule for these layers and drops the error logs of SDK ≥ v0.5.0.
- The recommended pattern (SDK docs + e2e CI) is to set `serviceInstance` to a version-scoped
value.

Expand Down
19 changes: 12 additions & 7 deletions docs/en/setup/backend/backend-wechat-mini-program-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,12 @@ metrics on the dashboard come from the `miniprogram.request.duration` histogram'

## Error Logs

The LAL rule `lal/miniprogram.yaml` uses `layer: auto` mode. It dispatches on the
`miniprogram.platform` resource attribute — one rule file produces both the `WECHAT_MINI_PROGRAM`
and `ALIPAY_MINI_PROGRAM` layers. For each error log, the extractor also emits a
`miniprogram_error_count` counter sample, which the log-MAL rule in `log-mal-rules/miniprogram.yaml`
aggregates into the per-layer `error_count` metric.
The LAL rules in `lal/miniprogram.yaml` handle the error logs. The SDK sets the `service.layer`
resource attribute, so its logs reach the `miniprogram-errors-wechat` and `miniprogram-errors-alipay`
rules, declared for the `WECHAT_MINI_PROGRAM` and `ALIPAY_MINI_PROGRAM` layers. For each error log,
the extractor also emits a `miniprogram_error_count` counter sample, which the log-MAL rules in
`log-mal-rules/miniprogram-wechat.yaml` and `log-mal-rules/miniprogram-alipay.yaml` aggregate into the
per-layer `error_count` metric.

Error categories (`exception.type` tag):

Expand All @@ -140,15 +141,19 @@ no longer auto-generates a device id; operators set `serviceInstance` to a versi

- **SDK ≤ v0.2.x** emits `componentId = 10001` (ajax-inherited). Its segments resolve to
`Layer.GENERAL` and do not benefit from this integration's layer / topology work. OTLP
metrics + logs still flow through MAL / LAL because they key on the `miniprogram.platform`
resource attribute, which v0.2 already emits.
metrics still flow through MAL because it keys on the `miniprogram.platform` resource
attribute, which v0.2 already emits.
- **SDK ≤ v0.3.x** auto-generates `service.instance.id = mp-{random}` per session, creating
one OAP instance entity per device — usually undesirable. Operators on v0.3.x can avoid this
by passing `init({ serviceInstance: serviceVersion })` explicitly.
- **SDK ≥ v0.4.0** leaves `service.instance.id` unset by default. The three signal pipelines
then handle absence differently: native segments produce a literal `-` instance entity; OTLP
logs and metrics create no instance entity at all. Per-instance dashboards are meaningful
only when the operator sets `serviceInstance`.
- **SDK ≥ v0.5.0 is required.** It sets the `service.layer` resource attribute, which routes its
error logs to the LAL rules of the mini-program layers. An older SDK sets none, so its error logs
are filed under `GENERAL` and produce no `error_count` metric. Conversely, an OAP older than this
version has no LAL rule for these layers and drops the error logs of SDK ≥ v0.5.0.
- The recommended pattern (SDK docs + e2e CI) is to set `serviceInstance` to a version-scoped
value (mirroring `service.version` or a release tag). Then all three signal pipelines
aggregate under the same OAP instance entity.
Expand Down
30 changes: 24 additions & 6 deletions oap-server/analyzer/log-analyzer/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,12 +192,30 @@ The generator detects the parser type from the AST at compile time and generates

### `layer: auto` Mode

Rules with `layer: auto` match logs where `service.layer` is absent (empty). Stored in
`LogFilterListener.Factory.autoDsls` (separate from the layer-keyed `dsls` map). At runtime,
`LogAnalyzer.doAnalysis()` routes empty-layer logs to auto rules via `createAnalysisListeners(null)`.

The `autoLayerMode` flag on `ExecutionContext` triggers a layer check in `FilterSpec.doSink()`:
if the extractor didn't set a layer, the log is warned and dropped.
Rules with `layer: auto` match logs where `service.layer` is absent (empty). `LogAnalyzer.doAnalysis()`
routes an empty-layer log in up to two passes, stopping at the first pass in which a rule claims it
(`LogAnalysisListener.claimed()`: did not abort):

1. The regular auto rules.
2. The `fallback: true` auto rules. Before this pass the metadata layer is reset to the value received,
since the rules share one `LogMetadata` and an aborted auto rule may have set a layer on it.

Both groups live in one immutable `Factory.AutoRules` snapshot, replaced whole on every runtime change, so a
rule whose `fallback` flag flips is never in neither (or both). `LogAnalyzer` takes both passes' listeners
from one snapshot up front via `LogAnalysisListenerFactory.createAuto()` → `AutoListeners`.
3. Nothing claimed it: the log is dropped (debug log). There is no fallback to the `GENERAL` rules;
`lal/default.yaml` ships `default-fallback`, which sets `GENERAL`.

Auto and fallback rules share one name space and rule key (`ruleKey(null, name)` → `<auto>|name`), so
`find` / `contains` / `remove` / suspend / runtime-rule keys treat them alike; `addOrReplace` moves a rule
between the two groups when its `fallback` flag changes. `fallback: true` on a non-auto rule fails compile.

The `autoLayerMode` flag on `ExecutionContext` (set for both passes) triggers a layer check in
`FilterSpec.doSink()`: if the extractor didn't set a layer, the log is warned and dropped. Because of it,
`TrafficSinkListener` never sees an empty layer and has no `GENERAL` fallback.

A static rule whose `layer` is neither `auto` nor a registered layer name (case-sensitive) is skipped with
a warning at boot (`Factory.loadStaticRules`); a runtime rule declaring one is rejected (`resolveLayer`).

### inputType and LALSourceTypeProvider SPI

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ public class LALConfig {

private String layer;

/**
* Only with {@code layer: auto}: the rule runs on a log without a layer only when no other auto rule
* claimed it. A log no fallback rule claims either is dropped.
*/
private boolean fallback;

/**
* Fully qualified class name of the input type (the extra log proto/POJO)
* for compile-time {@code parsed.*} getter resolution.
Expand Down
Loading
Loading