Repository navigation
Decide the layer of a log without one by LAL fallback rules, and save OTLP_GENERAL logs - #14244
Merged
Merged
Conversation
… OTLP_GENERAL logs A log without a layer runs the `layer: auto` rules, then, when none claimed it, the new `fallback: true` auto rules, and is dropped when none claims it either. The routing to the GENERAL rules in LogAnalyzer and the GENERAL fallback in TrafficSinkListener are removed; the new `default-fallback` rule in lal/default.yaml files such a log under GENERAL, as before, and operators edit it to choose another layer. Fallback rules share the auto rules' name space and rule key, `fallback: true` is allowed only with `layer: auto`, and the fallback pass starts from the layer as received, since the rules share one LogMetadata. The new `otlp-general` rule saves the logs of the OpenTelemetry services that set `service.layer=OTLP_GENERAL`, which no rule matched, so they were dropped. A static LAL rule declaring an unregistered or missing layer is skipped with a warning at boot, and a runtime rule declaring one is rejected, instead of loading under UNDEFINED and never running. The mini-program error logs are handled by rules declared for WECHAT_MINI_PROGRAM and ALIPAY_MINI_PROGRAM, replacing the `layer: auto` rule; mini-program-monitor >= v0.5.0 sets service.layer (SkyAPM/mini-program-monitor#11). The e2e cases run its simulator images of that merge commit. LALExpressionExecutionTest looked for its cases in directories that no longer exist and ran none; it reads them from the module's test resources again, with its harness aligned to LALScriptExecutionTest.
There was a problem hiding this comment.
🟡 Changes recommended
The auto-to-fallback transition is not atomic, allowing concurrent readers to miss an existing rule and causing logs to lose extraction or metrics or be dropped.
1 open issue
What changed in this PR
Moves layer selection for logs without a layer into configurable LAL fallback rules and enables storage of OTLP_GENERAL logs in SkyWalking’s log analyzer.
Changes:
- Adds fallback routing, shared auto-rule lookup, and validation of declared layers.
- Splits mini-program rules by platform layer and requires SDK v0.5.0.
- Updates migration documentation, adds routing and log-storage coverage, and restores expression-test discovery.
| File | Description |
|---|---|
test/e2e-v2/cases/satellite/native-protocols/lal.yaml |
Migrates agent-log processing to fallback rules. |
test/e2e-v2/cases/runtime-rule/lal/seed-rules/lal-v2.yaml |
Migrates the updated runtime-rule fixture. |
test/e2e-v2/cases/runtime-rule/lal/seed-rules/lal-v1.yaml |
Migrates the initial runtime-rule fixture. |
test/e2e-v2/cases/otlp-traces/native/log-cases.yaml |
Adds OTLP_GENERAL log and service checks. |
test/e2e-v2/cases/otlp-traces/native/expected/services-otlp-general-log.yml |
Defines expected service-layer results. |
test/e2e-v2/cases/otlp-traces/native/expected/logs-otlp-general.yml |
Defines expected persisted-log results. |
test/e2e-v2/cases/miniprogram/wechat/e2e.yaml |
Updates the simulator image reference in a comment. |
test/e2e-v2/cases/miniprogram/wechat/docker-compose.yml |
Pins the updated WeChat simulator image. |
test/e2e-v2/cases/miniprogram/alipay/e2e.yaml |
Updates the simulator image reference in a comment. |
test/e2e-v2/cases/miniprogram/alipay/docker-compose.yml |
Pins the updated Alipay simulator image. |
test/e2e-v2/cases/log/lal.yaml |
Migrates log extraction to fallback mode. |
test/e2e-v2/cases/dsl-debugging/lal-statement/seed-rules/lal-multistatement.yaml |
Migrates the debugging fixture to fallback mode. |
oap-server/server-starter/src/main/resources/lal/miniprogram.yaml |
Splits error-log rules by platform layer. |
oap-server/server-starter/src/main/resources/lal/default.yaml |
Adds default fallback and OTLP_GENERAL rules. |
oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/oap-cases/miniprogram.yaml |
Adds platform-specific rule fixtures. |
oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/oap-cases/miniprogram.data.yaml |
Adds mini-program error and non-error inputs. |
oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/oap-cases/default.yaml |
Mirrors the new default rules in tests. |
oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/oap-cases/default.data.yaml |
Adds inputs for the new default rules. |
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzerRoutingTest.java |
Tests routing and fallback metadata reset. |
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogFilterListenerLayerTest.java |
Tests layer validation and rule-group management. |
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALExpressionExecutionTest.java |
Repairs discovery and aligns input/tag handling. |
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzer.java |
Implements auto-then-fallback routing. |
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/TrafficSinkListener.java |
Removes implicit GENERAL layer assignment. |
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogFilterListener.java |
Manages fallback rules and validates layers. |
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogAnalysisListenerFactory.java |
Adds the default fallback-listener method. |
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/LALConfig.java |
Adds the fallback configuration flag. |
oap-server/analyzer/log-analyzer/CLAUDE.md |
Documents routing and registry behavior. |
docs/en/setup/backend/backend-wechat-mini-program-monitoring.md |
Documents layer routing and SDK compatibility. |
docs/en/setup/backend/backend-alipay-mini-program-monitoring.md |
Documents layer routing and SDK compatibility. |
docs/en/setup/backend/admin-api/dsl-debugging-lal.md |
Migrates the agent-log debugging example. |
docs/en/concepts-and-designs/lal.md |
Explains fallback semantics and rule migration. |
docs/en/changes/changes.md |
Records behavior changes and upgrade requirements. |
Review effort: Balanced
A runtime replace that flipped a rule's fallback flag wrote the auto and the fallback rule maps one after the other, so a concurrent log could find the rule in neither group and lose its extraction, metrics or the log itself. Both groups now live in one immutable snapshot, replaced whole, and LogAnalyzer takes both passes of a log from one snapshot through the new LogAnalysisListenerFactory.createAuto().
wu-sheng
approved these changes
Nov 17, 2026
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Decide the layer of a log without one by LAL fallback rules, and save OTLP_GENERAL logs
Fallback auto rules. A log without a layer runs the
layer: autorules, then, when none claimed it, the newfallback: trueauto rules, and is dropped when none claims it either. The routing to theGENERALrules inLogAnalyzerand theGENERALfallback inTrafficSinkListenerare removed. The newdefault-fallbackrule inlal/default.yamlfiles such a log underGENERAL, as before, and an operator edits its extractor to choose another layer.<auto>|name), so runtime-rule keys, find/remove and suspend/resume treat them alike.fallback: trueis allowed only withlayer: auto, and a fallback rule must set the layer, as any auto rule.LogMetadata, and an aborted auto rule may have set a layer on it.createFallback()is adefaultmethod onLogAnalysisListenerFactory.OTLP_GENERALlogs. The newotlp-generalrule saves the logs of the OpenTelemetry services that setservice.layer=OTLP_GENERAL. No rule matched that layer, so these logs were silently dropped. New cases inotlp-traces/native/log-cases.yamlcover it.Rule layers. A static LAL rule declaring an unregistered or missing layer (names are case-sensitive) is skipped with a warning at boot, and a runtime rule declaring one is rejected, instead of loading under
UNDEFINEDand never running.Mini-program. The
layer: autoruleminiprogram-errorsis replaced byminiprogram-errors-wechatandminiprogram-errors-alipay, declared forWECHAT_MINI_PROGRAMandALIPAY_MINI_PROGRAM. mini-program-monitor v0.5.0 setsservice.layer(SkyAPM/mini-program-monitor#11, merged) and is now the minimum SDK version. The e2e cases run the simulator images of that merge commit,9f59eb574addad78ffd2b84b8fc1de99c3071224.Upgrade note. A custom
layer: GENERALrule written for the logs without a layer must declarelayer: autoandfallback: trueand setlayer "GENERAL"in its extractor. The five e2e LAL files for agent logs (log, satellite, runtime-rule, dsl-debugging) were migrated this way.Test fix.
LALExpressionExecutionTestlooked for its cases in directories that no longer exist and ran none. It reads them from the module's test resources again, with its harness aligned toLALScriptExecutionTest's (theotlp-span-idinput; an absent tag reads as"").Verification.
Unit tests: log-analyzer (300) and runtime-rule (157) pass; checkstyle clean.
E2E against a rebuilt image: log (banyandb), satellite native-protocols, runtime-rule LAL, dsl-debugging lal-statement, iOS, mini-program WeChat and Alipay (with the published simulator images), and otlp-traces native on banyandb, ES and MySQL (24/24 each).
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the
CHANGESlog.