Skip to content

Commit afb2921

Browse files
committed
Let the sanitizers see pool allocations, and their reports
Two things kept this run from reporting what it is for. ngx_palloc and ngx_pnalloc hand out slices of a pool block that the allocator sees as one piece, so a write past an allocation lands inside memory ASan handed over whole and no report is made. nginx keeps a switch for exactly this case and sends every allocation straight to malloc when NGX_DEBUG_PALLOC is set. And the reports were being lost. nginx redirects its own stderr onto the error log file as soon as error_log names one, which Test::Nginx always does, so a grep over the suite's output saw ASan only indirectly -- through the worker dying -- and missed every UBSan report outright: halt_on_error keeps the process alive, and "runtime error:" carries none of the levels Test::Nginx looks for. With log_path the reports land in files of their own, and the findings step reads both sources. The options are comma separated with log_path in the middle, because Test::Nginx appends the name of each test block to it and cuts the value out with /log_path=([^,]+)/: a colon after the path is read as part of it, and the option behind it ends up inside the file name, which ASan refuses outright.
1 parent 112d948 commit afb2921

2 files changed

Lines changed: 53 additions & 12 deletions

File tree

‎.github/workflows/sanitizers.yml‎

Lines changed: 49 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,35 @@ jobs:
2020
timeout-minutes: 15
2121

2222
env:
23-
# nginx runs the little programs its configure compiles. With leak
24-
# detection on, each of them reports at exit and returns non-zero,
25-
# and every feature probe comes out "found but is not working". So
26-
# leaks are valgrind's job here, not this job's.
27-
ASAN_OPTIONS: detect_leaks=0
23+
# detect_leaks=0: nginx runs the little programs its configure
24+
# compiles, and with leak detection on each of them reports at exit
25+
# and returns non-zero, so every feature probe comes out "found but
26+
# is not working".
27+
#
28+
# log_path, because nginx redirects its own stderr onto the error
29+
# log file and a report written to stderr would be lost there; see
30+
# the findings step below.
31+
#
32+
# Comma separated, and log_path in the middle, both on purpose.
33+
# Test::Nginx appends the name of each test block to log_path so
34+
# that every block gets a file of its own, and it cuts the value out
35+
# with /log_path=([^,]+)/: a colon after the path is read as part of
36+
# it, and whatever option followed ends up inside the file name,
37+
# which ASan rejects with "Invalid value for bool option". That is
38+
# also why abort_on_error sits here and not on the step's command
39+
# line.
40+
ASAN_OPTIONS: detect_leaks=0,log_path=${{ github.workspace }}/sanlog/asan,abort_on_error=0
2841
# The suppression file names one finding in nginx's own startup
2942
# path and says why; see ci/ubsan.suppress. The path has to be
3043
# absolute, the suite runs nginx from a directory of its own.
31-
UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=0:suppressions=${{ github.workspace }}/ci/ubsan.suppress
32-
CI_CC_OPT: '-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1'
44+
UBSAN_OPTIONS: print_stacktrace=1,halt_on_error=0,suppressions=${{ github.workspace }}/ci/ubsan.suppress,log_path=${{ github.workspace }}/sanlog/ubsan
45+
# -DNGX_DEBUG_PALLOC=1 is what makes this run able to see the thing
46+
# it is for. Without it ngx_palloc and ngx_pnalloc hand out slices
47+
# of a pool block, so a write past an allocation lands inside a
48+
# block ASan handed over whole and no report is made; nginx keeps a
49+
# switch for exactly this and sends every allocation straight to
50+
# malloc when it is set.
51+
CI_CC_OPT: '-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1 -DNGX_DEBUG_PALLOC=1'
3352
CI_LD_OPT: '-fsanitize=address,undefined'
3453

3554
steps:
@@ -47,8 +66,8 @@ jobs:
4766
- name: Run the test suite under the sanitizers
4867
run: |
4968
set -o pipefail
50-
ASAN_OPTIONS="$ASAN_OPTIONS:abort_on_error=0" \
51-
TEST_NGINX_NO_SHUFFLE=1 \
69+
mkdir -p sanlog
70+
TEST_NGINX_NO_SHUFFLE=1 \
5271
TEST_NGINX_BINARY="$HOME/nginx/sbin/nginx" \
5372
prove -r t/ 2>&1 | tee sanitizer.log
5473
@@ -58,13 +77,31 @@ jobs:
5877
# t/ only reaches the configuration parser and one request. This
5978
# is the part that walks the module at run time, which is what the
6079
# sanitizers are here for.
61-
ASAN_OPTIONS="$ASAN_OPTIONS:abort_on_error=0" \
6280
ci/link.sh "$HOME/nginx/sbin/nginx" 2>&1 | tee -a sanitizer.log
6381
6482
- name: No findings allowed
6583
run: |
66-
if grep -nE "AddressSanitizer|runtime error:" sanitizer.log; then
67-
echo "a sanitizer reported the lines above" >&2
84+
# Both sources, because the sanitizers write where log_path
85+
# points and nginx writes its stderr to the error log file.
86+
# nginx redirects stderr onto that file as soon as error_log
87+
# names one, which Test::Nginx always does, so a grep over the
88+
# suite's output alone saw ASan only indirectly -- through the
89+
# worker dying -- and missed every UBSan report: halt_on_error
90+
# keeps the process alive and "runtime error:" carries none of
91+
# the levels Test::Nginx looks for.
92+
cat sanitizer.log > sanfindings.txt
93+
for f in sanlog/*; do
94+
[ -e "$f" ] || continue
95+
echo "== $f ==" >> sanfindings.txt
96+
cat "$f" >> sanfindings.txt
97+
done
98+
99+
if grep -nE "AddressSanitizer|runtime error:" sanfindings.txt |
100+
grep -q .
101+
then
102+
echo "--- everything the sanitizers wrote ---" >&2
103+
cat sanfindings.txt >&2
104+
echo "a sanitizer reported findings, see above" >&2
68105
exit 1
69106
fi
70107
echo "the sanitizers found nothing"

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@ t/servroot/
33

44
# created by ci/build.sh, ci/reload.sh and ci/link.sh
55
ci-work/
6+
# created by the sanitizer workflow
7+
sanlog/
8+
sanitizer.log
9+
sanfindings.txt
610

711
# leftovers from patch(1)
812
*.orig

0 commit comments

Comments
 (0)