Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions src/ngx_http_modsecurity_header_filter.c
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,18 @@ ngx_http_modsecurity_header_filter(ngx_http_request_t *r)

/* XXX: if NOT_MODIFIED, do we need to process it at all? see xslt_header_filter() */

/*
* subrequest output reaches the body filter as part of the main
* response; keep it in memory so that file buffers are inspected too
*/
if (r != r->main) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subrequest bodies can skip response-body inspection when served from files. This was already true before this PR, and this early return is the natural place to fix it.

SSI/addition subrequest output does reach the body filter, through the postpone filter as r->main. But filter_need_in_memory is only ever set on the main request (L467), so the subrequest's copy filter does not have to load file buffers into memory. With sendfile on, an <!--# include virtual="/static.txt" --> (or add_after_body) of a static file arrives as an in_file buf with an empty pos..last. msc_append_response_body() then gets 0 bytes, and the included content reaches the client without phase-4 inspection.

On master the subrequest also returned before L467 (ctx->processed was already set), so this is not a regression. Since this branch now handles subrequests explicitly, setting r->filter_need_in_memory = 1 here (when the main request has a ctx) would close the gap.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 3ab4c22. The subrequest branch of the header filter now sets r->filter_need_in_memory = 1 when ngx_http_modsecurity_get_module_ctx(r->main) is non-NULL, then passes the subrequest on. That is the flag ngx_http_copy_filter() reads for the subrequest (r->main_filter_need_in_memory || r->filter_need_in_memory). I set it per subrequest rather than setting main_filter_need_in_memory on the main request, because ngx_http_subrequest() only copies that into subrequests created after the main header filter has run.

New test in modsecurity-subrequest.t: sendfile on, an SSI <!--# include virtual="/static.txt" --> of a text/plain file, and a phase-4 RESPONSE_BODY rule matching a marker that is only present in the included file. It fails against master and against a build of this branch with only the filter_need_in_memory line removed (the audit record has rule 500 but not 503), and it passes with the change.

if (ngx_http_modsecurity_get_module_ctx(r->main) != NULL) {
r->filter_need_in_memory = 1;
}

return ngx_http_next_header_filter(r);
}

ctx = ngx_http_modsecurity_get_module_ctx(r);

dd("header filter, recovering ctx: %p", ctx);
Expand Down
6 changes: 6 additions & 0 deletions src/ngx_http_modsecurity_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,12 @@ ngx_inline ngx_http_modsecurity_ctx_t *
ngx_http_modsecurity_get_module_ctx(ngx_http_request_t *r)
{
ngx_http_modsecurity_ctx_t *ctx;

/* subrequests share r->pool; the cleanup walk would find the parent's ctx */
if (r != r->main) {
return NULL;
}

ctx = ngx_http_get_module_ctx(r, ngx_http_modsecurity_module);
if (ctx == NULL) {
/*
Expand Down
135 changes: 135 additions & 0 deletions tests/modsecurity-subrequest.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
#!/usr/bin/perl

# Tests for ModSecurity-nginx connector (subrequests).

###############################################################################

use warnings;
use strict;

use Test::More;

BEGIN { use FindBin; chdir($FindBin::Bin); }

use lib 'lib';
use Test::Nginx;

###############################################################################

select STDERR; $| = 1;
select STDOUT; $| = 1;

my $t = Test::Nginx->new()->has(qw/http auth_request ssi/);

$t->write_file_expand('nginx.conf', <<'EOF');

%%TEST_GLOBALS%%

daemon off;

events {
}

http {
%%TEST_GLOBALS_HTTP%%

server {
listen 127.0.0.1:8080;
server_name localhost;

modsecurity on;
modsecurity_rules '
SecRuleEngine On
SecRule RESPONSE_STATUS "@streq 204" "id:501,phase:3,log,auditlog,pass"
SecRule RESPONSE_HEADERS:X-Main-Marker "@streq main" "id:502,phase:3,log,auditlog,pass"
SecAuditEngine RelevantOnly
SecAuditLogParts ABH
SecAuditLog %%TESTDIR%%/audit-auth.txt
SecAuditLogType Serial
SecAuditLogStorageDir %%TESTDIR%%/
';

location = /auth {
return 204;
}

location / {
auth_request /auth;
add_header X-Main-Marker main;
}
}

server {
listen 127.0.0.1:8080;
server_name ssi;

sendfile on;
log_subrequest on;
default_type text/plain;

modsecurity on;
modsecurity_rules '
SecRuleEngine On
SecResponseBodyAccess On
SecResponseBodyMimeType text/html
SecRule REQUEST_HEADERS:X-Test "@streq ssi" "id:500,phase:1,log,auditlog,pass"
SecRule RESPONSE_BODY "@contains INCLUDED-MARKER" "id:503,phase:4,log,auditlog,pass"
SecAuditEngine RelevantOnly
SecAuditLogParts ABH
SecAuditLog %%TESTDIR%%/audit-ssi.txt
SecAuditLogType Serial
SecAuditLogStorageDir %%TESTDIR%%/
';

location = /ssi.html {
default_type text/html;
ssi on;
}
}
}

EOF

$t->write_file('index.html', 'MAIN PAGE');
$t->write_file('ssi.html',
'BEFORE <!--# include virtual="/static.txt" --> AFTER');
$t->write_file('static.txt', 'INCLUDED-MARKER');
$t->run();
$t->plan(7);

###############################################################################

my $r = http_get('/index.html');

like($r, qr/^HTTP.*200/, 'request with auth_request passes');
like($r, qr/MAIN PAGE/, 'main body delivered');

my $audit = $t->read_file('audit-auth.txt');

like($audit, qr/\[id "502"\]/, 'main response headers inspected');
unlike($audit, qr/\[id "501"\]/, 'auth subrequest response not inspected as main response');

$r = http(<<EOF);
GET /ssi.html HTTP/1.0
Host: ssi
X-Test: ssi

EOF

like($r, qr/BEFORE INCLUDED-MARKER AFTER/, 'ssi include delivered');

$audit = $t->read_file('audit-ssi.txt');

like($audit, qr/\[id "503"\]/, 'included file inspected in phase 4');

# Each serial audit log record starts with a part-A section header of the
# form "---<boundary>---A--" on a line of its own. The SSI subrequest is
# created while the main response body is sent, after every ACCESS-phase
# handler has run, so rule 500 has made the transaction relevant by then and
# a phase 5 run for the logged subrequest would show up as an extra record.

my $records = () = $audit =~ /^---\w+---A--$/mg;

is($records, 1, 'phase 5 ran once for the transaction');

###############################################################################
Loading