Skip to content

experiment: use a lazy bytestring hasql 'unknown'#2333

Closed
robx wants to merge 2 commits into
PostgREST:mainfrom
robx:optimize
Closed

experiment: use a lazy bytestring hasql 'unknown'#2333
robx wants to merge 2 commits into
PostgREST:mainfrom
robx:optimize

Conversation

@robx
Copy link
Copy Markdown
Contributor

@robx robx commented Jun 16, 2022

compare #2261

@robx
Copy link
Copy Markdown
Contributor Author

robx commented Jun 16, 2022

Load test has no significant change. Memory use in mem-tests seems more or less significantly decreased:

< ok 1 - POST /rpc/leak?columns=blob: with a json key of 1M the memory usage(15,605,584 bytes) is less than 16M
> ok 1 - POST /rpc/leak?columns=blob: with a json key of 1M the memory usage(14,662,232 bytes) is less than 16M
< ok 2 - POST /leak?columns=blob: with a json key of 1M the memory usage(15,538,336 bytes) is less than 16M
> ok 2 - POST /leak?columns=blob: with a json key of 1M the memory usage(14,541,408 bytes) is less than 16M
< ok 3 - PATCH /leak?id=eq.1&columns=blob: with a json key of 1M the memory usage(15,585,384 bytes) is less than 16M
> ok 3 - PATCH /leak?id=eq.1&columns=blob: with a json key of 1M the memory usage(14,601,496 bytes) is less than 16M
< ok 4 - POST /rpc/leak?columns=blob: with a json key of 10M the memory usage(42,927,272 bytes) is less than 44M
> ok 4 - POST /rpc/leak?columns=blob: with a json key of 10M the memory usage(32,960,632 bytes) is less than 44M
< ok 5 - POST /leak?columns=blob: with a json key of 10M the memory usage(42,847,232 bytes) is less than 44M
> ok 5 - POST /leak?columns=blob: with a json key of 10M the memory usage(32,893,504 bytes) is less than 44M
< ok 6 - PATCH /leak?id=eq.1&columns=blob: with a json key of 10M the memory usage(42,900,040 bytes) is less than 44M
> ok 6 - PATCH /leak?id=eq.1&columns=blob: with a json key of 10M the memory usage(32,947,056 bytes) is less than 44M
< ok 7 - POST /rpc/leak?columns=blob: with a json key of 50M the memory usage(164,339,792 bytes) is less than 172M
> ok 7 - POST /rpc/leak?columns=blob: with a json key of 50M the memory usage(114,496,360 bytes) is less than 172M
< ok 8 - POST /leak?columns=blob: with a json key of 50M the memory usage(164,227,224 bytes) is less than 172M
> ok 8 - POST /leak?columns=blob: with a json key of 50M the memory usage(114,421,840 bytes) is less than 172M
< ok 9 - PATCH /leak?id=eq.1&columns=blob: with a json key of 50M the memory usage(164,282,832 bytes) is less than 172M
> ok 9 - PATCH /leak?id=eq.1&columns=blob: with a json key of 50M the memory usage(114,479,848 bytes) is less than 172M
< ok 10 - POST /perf_articles?columns=id,body: with a json payload of 32K that has 1000 array values the memory usage(12,605,744 bytes) is less than 14M
> ok 10 - POST /perf_articles?columns=id,body: with a json payload of 32K that has 1000 array values the memory usage(12,574,616 bytes) is less than 14M
< ok 11 - POST /perf_articles?columns=id,body: with a json payload of 329K that has 10000 array values the memory usage(13,508,120 bytes) is less than 14M
> ok 11 - POST /perf_articles?columns=id,body: with a json payload of 329K that has 10000 array values the memory usage(13,182,160 bytes) is less than 14M
< ok 12 - POST /perf_articles?columns=id,body: with a json payload of 3.4M that has 100000 array values the memory usage(22,804,512 bytes) is less than 24M
> ok 12 - POST /perf_articles?columns=id,body: with a json payload of 3.4M that has 100000 array values the memory usage(19,428,296 bytes) is less than 24M

@steve-chavez
Copy link
Copy Markdown
Member

Memory use in mem-tests seems more or less significantly decreased:

Awesome! And this is by removing just one copy from what I understand, so it should be even better when removing more copies.

To ensure we don't increase on memory usage later, it'd be good to update the following tests:

jsonKeyTest "1M" "POST" "/rpc/leak?columns=blob" "16M"
jsonKeyTest "1M" "POST" "/leak?columns=blob" "16M"
jsonKeyTest "1M" "PATCH" "/leak?id=eq.1&columns=blob" "16M"
jsonKeyTest "10M" "POST" "/rpc/leak?columns=blob" "44M"
jsonKeyTest "10M" "POST" "/leak?columns=blob" "44M"
jsonKeyTest "10M" "PATCH" "/leak?id=eq.1&columns=blob" "44M"
jsonKeyTest "50M" "POST" "/rpc/leak?columns=blob" "172M"
jsonKeyTest "50M" "POST" "/leak?columns=blob" "172M"
jsonKeyTest "50M" "PATCH" "/leak?id=eq.1&columns=blob" "172M"
postJsonArrayTest "1000" "/perf_articles?columns=id,body" "14M"
postJsonArrayTest "10000" "/perf_articles?columns=id,body" "14M"
postJsonArrayTest "100000" "/perf_articles?columns=id,body" "24M"

Load test has no significant change.

Hmm, I expected this to be affected too. Once this PR is merged I'll make some bench tests.

@robx
Copy link
Copy Markdown
Contributor Author

robx commented Aug 10, 2022

Closing this one in favour of #2349 (which includes the same change via jsonBytes instead of unknown)

@robx robx closed this Aug 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants