TOOLS-4100 Pin an unreleased 9.0 DSC Server build via mongodb-downloader

[autarch]

TOOLS-4100 Pin an unreleased 9.0 DSC Server build via mongodb-downloader

Adds a mongodb-downloader config pinning mongo commit d7c2447c for a disagg
(disaggregated-storage) 9.0 build on ubuntu2204-x86_64, cached in the
c2c-testing-server-artifacts bucket, plus the generated artifacts file.

Because the entry sets disagg: true, the tarball also carries the four
buildscripts/modules/atlas files that mongodb-runner needs to drive a DSC
cluster, read from the same commit as the binaries. That is what lets the
launch script added in a later commit work without a mongo repo checkout.

Verified after upload: the tarball is present in S3 (219 MiB), check passes,
the extracted mongod reports v9.0.0-alpha0-d7c2447c, it accepts
--setParameter disaggregatedStorageEnabled=true (so it is genuinely a DSC
build, not a stock one), and the shipped manifest.json pins SLS commit
665e99c1ceea92bf4dd27e32e194548375400fe9.

• etc/mongodb-downloader-config.yaml - the bucket and the 9.0-dsc pin
• etc/mongodb-downloader-s3-artifacts.json - generated; records the tarball and commit

TOOLS-4100 Add a script to authenticate docker to the SLS ECR registry

The SLS container images backing a DSC cluster live in a private ECR registry
(account 664315256653), so docker must be logged in before mongodb-runner can
pull them.

• scripts/authenticate-sls-ecr.sh - new; logs docker in to the SLS registry

TOOLS-4100 Add scripts to start and stop a local DSC cluster

scripts/start-dsc-cluster.sh downloads the pinned disagg Server build, reads
the SLS compose file and pinned SLS image tag out of that same tarball, and
starts a 2-node DSC replica set via mongodb-runner, printing the connection
string to export as TOOLS_TESTING_MONGOD.

Reading the compose file and image tag from the tarball rather than from a
mongo repo checkout means they cannot drift from the Server binaries, and
means no mongo checkout is needed to start a cluster.

The script starts a cluster and stops; it does not run tests. Startup pays for
a docker compose project and, on a first run, several minutes of image pulls,
which should not be re-paid per test iteration.

Two things this had to account for, both found by running it:

• The S3 bucket holding the Server tarball and the ECR registry holding the
  SLS images are in different AWS accounts, and no single profile can reach
  both. So the ECR login takes its own profile via SLS_ECR_AWS_PROFILE while
  AWS_PROFILE stays with the downloader. Note that a profile can log in to the
  registry successfully and still lack permission to pull, because minting a
  token only needs access in the caller's own account.
• mongodb-runner spawns mongod from its own working directory, so a relative
  --binDir fails with ENOENT even when the binary is present. All paths handed
  to the runner are absolute.

Verified: a 2-node DSC replica set starts and reports PRIMARY,SECONDARY with
disaggregatedStorageEnabled: true on server 9.0.0-alpha0-d7c2447c.

mongodb-runner DSC support is still an unmerged draft (mongodb-js/devtools-shared#822),
so DEVTOOLS_SHARED points at a local checkout of it.

• scripts/start-dsc-cluster.sh - new; downloads the build and starts the cluster
• scripts/stop-dsc-cluster.sh - new; tears down the cluster and its compose project
• scripts/authenticate-sls-ecr.sh - take the profile from SLS_ECR_AWS_PROFILE
• .gitignore - ignore the dsc-cluster/ working directory

TOOLS-4100 Address review findings in the DSC cluster scripts

The important one: start-dsc-cluster.sh could strand a live cluster. Once
mongodb-runner start succeeds the cluster is ours to clean up, but a later
failure -- notably failing to find a connection string in the output -- exited
non-zero leaving a running 2-node cluster and its compose project behind, with
no hint about how to remove them. An EXIT trap now tears the cluster down on
any failure after that point, and is cleared once the cluster is confirmed
usable, since leaving it running is the whole point of the script.

The rest are smaller:

• stop-dsc-cluster.sh removed ./dsc-cluster/connection-string relative to the
  caller's working directory, so running it from anywhere but the repo root
  left a stale file. It now derives the repo root the same way the start
  script does. Verified by running it from /tmp.
• python3 and the pinned_sls_commit key were undeclared prerequisites, so a
  missing python3 produced a bare "command not found" rather than one of the
  named up-front errors the script already uses for docker and the runner.
• authenticate-sls-ecr.sh was missing set -o nounset, which both other
  scripts set, and applied ":?" guards to values assigned literally one line
  above, where they can never fire and wrongly imply an external input.
• Corrected two misleading comments: "starts a cluster and stops" read as the
  cluster stopping, and the note about stdout/stderr described the tee
  backwards. One comment also referred to realpath, which is not used.

Re-verified after the changes: a 2-node DSC cluster starts, reports
PRIMARY,SECONDARY with disaggregatedStorageEnabled true, survives the script
exiting, and tears down cleanly with no compose project left behind.

TOOLS-4100 Make SLS_ECR_AWS_PROFILE optional

Whether one AWS profile can both read the Server tarball from S3 and pull the
SLS images from ECR depends on how a given developer's access is set up, so
requiring the split is wrong. SLS_ECR_AWS_PROFILE now runs just the ECR login
under its own profile when set, and falls back to AWS_PROFILE when unset --
which is what authenticate-sls-ecr.sh already did on its own.

Verified both ways: with the two profiles split, and with a single profile and
SLS_ECR_AWS_PROFILE unset. Both produce a 2-node cluster reporting
PRIMARY,SECONDARY with disaggregatedStorageEnabled true.

Worth knowing for the single-profile case: mongodb-downloader download skips
S3 entirely once the binaries are already extracted at the recorded commit, so
after the first download a profile with only ECR access is enough to start a
cluster.

TOOLS-4100 Skip profile-level-2 tests under disaggregated storage

Three integration tests enable database profiling at level 2 and then read
system.profile to assert the query the tool actually issued -- for instance
that TOOLS-1952's $natural hint reaches the server. Disaggregated storage
rejects profile: 2 outright:

(CommandNotSupported) Profile level 2 is not supported in this storage
mode: Disaggregated Storage

so these tests cannot observe what they exist to observe, and failed before
exercising any dump or export behavior. Skip them there.

The new testutil.SkipForDisaggregatedStorage keys off DSC rather than off the
profiling command failing, so an unexpected profiling failure on any other
configuration still fails loudly instead of quietly skipping. It sits beside
SkipForAtlasCluster and SkipIfFCVLessThan, which serve the same purpose for
other environments.

These three were the only DSC-specific failures in the suite. Two further
failures, TestCount and TestMongoDumpViews, were downstream fallout: both
passed in isolation and failed only in a full run, because the profiling
failures aborted before their deferred cleanup and left mongodump_test_db
populated for the next test. Fixing these three clears those two as well.

Verified against a 2-node DSC cluster and against a non-DSC replica set built
from the same 9.0 binaries:

• On DSC, all three skip with the reason above, and the suite drops from 18 to
  13 failing tests -- exactly the 13 that also fail on the non-DSC baseline. No
  DSC-only failure remains.

• On non-DSC, all three still run and pass, so the skip is scoped to DSC and
  does not silently disable coverage elsewhere.

• common/testutil/testutil.go - add SkipForDisaggregatedStorage

• mongodump/mongodump_test.go - skip TestMongoDumpTOOLS1952 and TestMongoDumpViewsAsCollections

• mongoexport/mongoexport_test.go - skip TestMongoExportTOOLS1952

[autarch]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• TOOLS-4100 Pin an unreleased 9.0 DSC Server build via mongodb-downloader #1118 👈 (View in Graphite)
• TOOLS-4100 Always connect to the test cluster using the same shared code #1119 : 1 other dependent PR (#1120 )
• TOOLS-4100 Fix a panic that can happen when TOOLS_TESTING_MONGOD is set #1117
• Add more .gitignore entries for test files created during test runs #1116
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.