Skip to content

Document behavior of antispam_mem_table_size and antispam_base_table_… #880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jiemingwang wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Document behavior of antispam_mem_table_size and antispam_base_table_… #880

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5a7b877
Document behavior of antispam_mem_table_size and antispam_base_table_…
jiemingwang Jun 18, 2026
eaad7e9
Merge pull request #1 from transparency-dev/main
jiemingwang Jun 29, 2026
0bc8ce1
Reduce defauls for antispam_mem_table_size and antispam_base_table_size
jiemingwang Jun 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions cmd/tesseract/posix/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,14 @@ To ensure service stability, we strongly recommend setting the `GOMEMLIMIT` envi
> - **Recommended setting**: 80-90% of the container's total memory limit (e.g., `GOMEMLIMIT=1.8GiB` for a 2GiB container).


## antispam_mem_table_size & antispam_base_table_size

These parameters set the upper bound on antispam table size. Increasing them results in larger individual .sst files and fewer total .sst files. They should be tuned in conjunction with the compactor count and available CPU resources. On systems with excess CPU capacity, a configuration with smaller tables and more compactors is often preferable.

> [!IMPORTANT]
> - **These parameters must not be reduced after initial configuration (defaults are antispam_mem_table_size=256<<18 and antispam_base_table_size=16<<18). Lowering them will cause the POSIX process to fail during BadgerDB initialization and remain stuck, typically with an error similar to: “Arena too small, toWrite:40 newTotal:174483044 limit:174483027”.


## Witnessing

> [!WARNING]
Expand Down
12 changes: 6 additions & 6 deletions cmd/tesseract/posix/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,8 @@ var (
antispamIndexCacheSize = flag.String("antispam_index_cache_size", "768MB", "Amount of RAM to allocate for antispam index cache, set to zero for unlimited.")
antispamCompactionInterval = flag.Duration("antispam_compaction_interval", tposix_as.DefaultCompactionInterval, "Interval between GC/compaction runs on the antispam index.")
antispamNumCompactors = flag.Int("antispam_num_compactors", 2, "Number of BadgerDB compactors to run.")
antispamMemTableSize = flag.Int64("antispam_mem_table_size", 256<<20, "Size of BadgerDB memtables.")
antispamBaseTableSize = flag.Int64("antispam_base_table_size", 16<<20, "Size of BadgerDB base tables.")
antispamMemTableSize = flag.Int64("antispam_mem_table_size", 256<<18, "Size of BadgerDB memtables.")
antispamBaseTableSize = flag.Int64("antispam_base_table_size", 16<<18, "Size of BadgerDB base tables.")
awaiterPollInterval = flag.Duration("awaiter_poll_interval", storage.DefaultAwaiterPollInterval, "Interval between two checkpoint polls by the awaiter. Used for antispam, and if enable_publication_awaiter is set, to block add-* requests responses. Must be strictly positive or defaults to DefaultAwaiterPollInterval.")

// Infrastructure setup flags
Expand Down Expand Up @@ -249,10 +249,10 @@ func newStorage(ctx context.Context, signer note.Signer) (st *storage.CTStorage,
CompactionInterval: *antispamCompactionInterval,
BadgerOptions: func(o badger.Options) badger.Options {
return o.
WithCompression(options.None). // Off as this appears to cause memory issues when compacting large indices.
WithMemTableSize(*antispamMemTableSize). // Default tunes memtables for high write throughput
WithBaseTableSize(*antispamBaseTableSize). // Default tunes to reduce file count
WithNumCompactors(*antispamNumCompactors). // Default tunes to be able keep up with high throughput of LSM merges
WithCompression(options.None). // Off as this appears to cause memory issues when compacting large indices.
WithMemTableSize(*antispamMemTableSize). // Default tunes memtables for high write throughput
WithBaseTableSize(*antispamBaseTableSize). // Default tunes to reduce file count
WithNumCompactors(*antispamNumCompactors). // Default tunes to be able keep up with high throughput of LSM merges
WithIndexCacheSize(int64(antispamIndexCacheBytes)).
WithBlockCacheSize(int64(antispamBlockCacheBytes))
},
Expand Down