Repository navigation
Do not run the connector on subrequests #394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tomsommer
wants to merge
4
commits into
owasp-modsecurity:master
Choose a base branch
from
tomsommer:fix/subrequest-filters
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
883d6ff
Do not run the connector on subrequests
0d15257
Merge remote-tracking branch 'upstream/master' into fix/subrequest-fi…
6efc199
Skip subrequests in get_module_ctx() instead of in each hook
3ab4c22
Keep subrequest output in memory for response body inspection
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| #!/usr/bin/perl | ||
|
|
||
| # Tests for ModSecurity-nginx connector (subrequests). | ||
|
|
||
| ############################################################################### | ||
|
|
||
| use warnings; | ||
| use strict; | ||
|
|
||
| use Test::More; | ||
|
|
||
| BEGIN { use FindBin; chdir($FindBin::Bin); } | ||
|
|
||
| use lib 'lib'; | ||
| use Test::Nginx; | ||
|
|
||
| ############################################################################### | ||
|
|
||
| select STDERR; $| = 1; | ||
| select STDOUT; $| = 1; | ||
|
|
||
| my $t = Test::Nginx->new()->has(qw/http auth_request ssi/); | ||
|
|
||
| $t->write_file_expand('nginx.conf', <<'EOF'); | ||
|
|
||
| %%TEST_GLOBALS%% | ||
|
|
||
| daemon off; | ||
|
|
||
| events { | ||
| } | ||
|
|
||
| http { | ||
| %%TEST_GLOBALS_HTTP%% | ||
|
|
||
| server { | ||
| listen 127.0.0.1:8080; | ||
| server_name localhost; | ||
|
|
||
| modsecurity on; | ||
| modsecurity_rules ' | ||
| SecRuleEngine On | ||
| SecRule RESPONSE_STATUS "@streq 204" "id:501,phase:3,log,auditlog,pass" | ||
| SecRule RESPONSE_HEADERS:X-Main-Marker "@streq main" "id:502,phase:3,log,auditlog,pass" | ||
| SecAuditEngine RelevantOnly | ||
| SecAuditLogParts ABH | ||
| SecAuditLog %%TESTDIR%%/audit-auth.txt | ||
| SecAuditLogType Serial | ||
| SecAuditLogStorageDir %%TESTDIR%%/ | ||
| '; | ||
|
|
||
| location = /auth { | ||
| return 204; | ||
| } | ||
|
|
||
| location / { | ||
| auth_request /auth; | ||
| add_header X-Main-Marker main; | ||
| } | ||
| } | ||
|
|
||
| server { | ||
| listen 127.0.0.1:8080; | ||
| server_name ssi; | ||
|
|
||
| sendfile on; | ||
| log_subrequest on; | ||
| default_type text/plain; | ||
|
|
||
| modsecurity on; | ||
| modsecurity_rules ' | ||
| SecRuleEngine On | ||
| SecResponseBodyAccess On | ||
| SecResponseBodyMimeType text/html | ||
| SecRule REQUEST_HEADERS:X-Test "@streq ssi" "id:500,phase:1,log,auditlog,pass" | ||
| SecRule RESPONSE_BODY "@contains INCLUDED-MARKER" "id:503,phase:4,log,auditlog,pass" | ||
| SecAuditEngine RelevantOnly | ||
| SecAuditLogParts ABH | ||
| SecAuditLog %%TESTDIR%%/audit-ssi.txt | ||
| SecAuditLogType Serial | ||
| SecAuditLogStorageDir %%TESTDIR%%/ | ||
| '; | ||
|
|
||
| location = /ssi.html { | ||
| default_type text/html; | ||
| ssi on; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| EOF | ||
|
|
||
| $t->write_file('index.html', 'MAIN PAGE'); | ||
| $t->write_file('ssi.html', | ||
| 'BEFORE <!--# include virtual="/static.txt" --> AFTER'); | ||
| $t->write_file('static.txt', 'INCLUDED-MARKER'); | ||
| $t->run(); | ||
| $t->plan(7); | ||
|
|
||
| ############################################################################### | ||
|
|
||
| my $r = http_get('/index.html'); | ||
|
|
||
| like($r, qr/^HTTP.*200/, 'request with auth_request passes'); | ||
| like($r, qr/MAIN PAGE/, 'main body delivered'); | ||
|
|
||
| my $audit = $t->read_file('audit-auth.txt'); | ||
|
|
||
| like($audit, qr/\[id "502"\]/, 'main response headers inspected'); | ||
| unlike($audit, qr/\[id "501"\]/, 'auth subrequest response not inspected as main response'); | ||
|
|
||
| $r = http(<<EOF); | ||
| GET /ssi.html HTTP/1.0 | ||
| Host: ssi | ||
| X-Test: ssi | ||
|
|
||
| EOF | ||
|
|
||
| like($r, qr/BEFORE INCLUDED-MARKER AFTER/, 'ssi include delivered'); | ||
|
|
||
| $audit = $t->read_file('audit-ssi.txt'); | ||
|
|
||
| like($audit, qr/\[id "503"\]/, 'included file inspected in phase 4'); | ||
|
|
||
| # Each serial audit log record starts with a part-A section header of the | ||
| # form "---<boundary>---A--" on a line of its own. The SSI subrequest is | ||
| # created while the main response body is sent, after every ACCESS-phase | ||
| # handler has run, so rule 500 has made the transaction relevant by then and | ||
| # a phase 5 run for the logged subrequest would show up as an extra record. | ||
|
|
||
| my $records = () = $audit =~ /^---\w+---A--$/mg; | ||
|
|
||
| is($records, 1, 'phase 5 ran once for the transaction'); | ||
|
|
||
| ############################################################################### |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Subrequest bodies can skip response-body inspection when served from files. This was already true before this PR, and this early return is the natural place to fix it.
SSI/addition subrequest output does reach the body filter, through the postpone filter as
r->main. Butfilter_need_in_memoryis only ever set on the main request (L467), so the subrequest's copy filter does not have to load file buffers into memory. Withsendfile on, an<!--# include virtual="/static.txt" -->(oradd_after_body) of a static file arrives as anin_filebuf with an emptypos..last.msc_append_response_body()then gets 0 bytes, and the included content reaches the client without phase-4 inspection.On master the subrequest also returned before L467 (
ctx->processedwas already set), so this is not a regression. Since this branch now handles subrequests explicitly, settingr->filter_need_in_memory = 1here (when the main request has a ctx) would close the gap.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in 3ab4c22. The subrequest branch of the header filter now sets
r->filter_need_in_memory = 1whenngx_http_modsecurity_get_module_ctx(r->main)is non-NULL, then passes the subrequest on. That is the flagngx_http_copy_filter()reads for the subrequest (r->main_filter_need_in_memory || r->filter_need_in_memory). I set it per subrequest rather than settingmain_filter_need_in_memoryon the main request, becausengx_http_subrequest()only copies that into subrequests created after the main header filter has run.New test in
modsecurity-subrequest.t:sendfile on, an SSI<!--# include virtual="/static.txt" -->of atext/plainfile, and a phase-4RESPONSE_BODYrule matching a marker that is only present in the included file. It fails against master and against a build of this branch with only thefilter_need_in_memoryline removed (the audit record has rule 500 but not 503), and it passes with the change.