repo: import doublezero-offchain and doublezero-solana with their history by bgm-malbeclabs · Pull Request #4240 · malbeclabs/doublezero

bgm-malbeclabs · 2026-08-27T23:36:59Z

Step 2 of the monorepo migration, per docs/superpowers/specs/2026-08-27-monorepo-migration-design.md. Imports malbeclabs/doublezero-offchain and malbeclabs/doublezero-solana into this repo with their history and their release tags.

Summary

Both source trees are relocated with git filter-repo --to-subdirectory-filter and merged with --allow-unrelated-histories, so every path from each source repo lands under a new top-level offchain/ or solana/ and nothing that was already here moves. Zero conflicts in either merge.
All 550 commits and all 126 tags come across. Tags stay unprefixed, so each component's version line stays continuous: the next contributor-rewards release picks up from v0.6.1 rather than restarting.
Imported commit messages are rewritten in the same filter-repo pass. A bare (#N) in a source repo meant a pull request there; here it would resolve to an unrelated one. 451 subjects carried one, across 377 distinct numbers, and every number sampled exists in this repo. They now read malbeclabs/doublezero-offchain#N and malbeclabs/doublezero-solana#N. References that were already qualified are untouched.
offchain and solana join the root workspace exclude list. Both keep their own Cargo.toml, Cargo.lock and rust-toolchain.toml and build as nested workspaces, exactly as they did in their own repos. The root Cargo.lock does not change.
.github/dependabot.yml enumerates its cargo directories instead of globbing **/*, which would otherwise have found both imported manifests. Nothing here builds either tree, so a grouped update would have edited their lockfiles, merged green with nothing compiled, and diverged them from the repos that still release them. The five listed directories are every one with its own lockfile, which is what the glob resolved to before the import.
The imported .github/workflows/ are not promoted to the repo root, so none of the eleven workflows runs. GitHub Actions reads workflows only from the root, so the nested copies are inert, and leaving them in place keeps each imported tree an exact copy of its source and gives step 5 its source material in-tree.
The source repos are still the ones that release. Nothing about how this repo builds, tests or releases changes.

The orphan sentinel/v0.6.1 tag was deleted first, per D5. It had no release and no goreleaser config behind it, and it sat above offchain's live sentinel line, which reaches v0.2.6. The collision check was re-run immediately before the import and still prints exactly sentinel; doublezero-solana collides with nothing.

Merge instructions

This must be merged with "Create a merge commit", not squash. A squash collapses 550 commits into one and destroys the history this pull request exists to preserve.

Two rules on main block that and both need lifting for the merge, then restoring:

required_linear_history, which refuses merge commits outright.
required_signatures. git filter-repo rewrites every tree, which invalidates the signatures the source commits carried, so all 550 arrive unsigned. Nothing can re-sign another author's commits.

The pull_request rule is separate and stays on throughout, so nobody can push to main directly during the window.

The tags are pushed after this merges, so they point at commits that are ancestors of main.

Size

424 files and roughly 589,000 insertions, far above this repo's 500-line guideline. It cannot be broken up: an import of two repositories is one operation, and splitting it would leave main holding a half-imported tree. Note that 82% of the insertions are five JSON files, two of which are contributor-rewards test snapshots totalling 483,000 lines. The Rust is 74,000 lines across 292 files, and all of it is unchanged from the source repos. The reviewable surface of this pull request is the three files outside the imported trees; the checks below establish that everything else is a copy.

Testing Verification

git diff between each imported tree and its filtered source is empty. Both trees are byte-identical to what git filter-repo produced from the source main.
Outside offchain/ and solana/, exactly three files change: Cargo.toml, CHANGELOG.md and .github/dependabot.yml. No existing job definition is touched.
All 126 source tags are present and each points at the same commit as in its filtered source. 110 from offchain, 16 from solana.
No bare #N reference remains anywhere in the imported history, in subjects or bodies.
The root Cargo.lock is unchanged, and the root workspace resolves with the two new directories excluded.
Both nested workspaces resolve standalone from their own directories.
Neither imported tree contains Go sources, so the root go build ./... scope is unaffected.
None of the 16 imported tag prefixes matches a tag trigger in any of the 49 root workflows, and neither source repo has a bare v*.*.* tag, so pushing the tags fires no release.

Follow-ups, deliberately not in this pull request

Two pre-existing findings in the imported code, filed as malbeclabs/infra#2395: a missing SOLANA_RPC boots the scheduler clean and fails later inside a worker, and DZ_LEDGER_RPC is read into config that nothing consumes.
The root .dockerignore does not exclude the new trees, so the release.docker.core.yml build context grows by about 31 MB. That workflow fires only on bare v*.*.* tags. Step 5 merges the release setup and is the place to decide whether the root images need either tree.

Adds PR template same as doublezero repo

…n#116) ## Summary of Changes This PR adds prom metrics to validator-debt. The metrics endpoint is specified with an env var and if it's not set, then metrics aren't captured. Only a few things have been instrumented which are the most important and more might be added as needed. Closes #1723. ## Testing Verification Ran the validator-debt app locally and in a separate tab ran `curl` against the metrics endpoint to verify that metrics were being properly surfaced. ``` > curl localhost:9090/metrics # TYPE doublezero_validator_debt_overlapping_epochs counter doublezero_validator_debt_overlapping_epochs{dz_epoch="45",solana_epoch="871"} 1 # TYPE doublezero_validator_debt_build_info gauge doublezero_validator_debt_build_info{version="0.1.0-rc4",commit="UNKNOWN",date="UNKNOWN",pkg_version="0.1.0-rc4"} ```

…/doublezero-offchain#181) ## Summary of Changes This PR moves validator-debt in line with the [group_imports](https://rust-lang.github.io/rustfmt/?version=v1.8.0&search=#group_imports) that are on unstable rust. Unfortunately, this can't be accomplished with adding the directive to rustfmt because it's only on unstable so this will have to be something that is monitored through code review or another mechanism. Closes #2087. ## Testing Verification * Build still builds

…eclabs/doublezero-offchain#180) ## Summary of Changes This PR adds a command called `harvest-2z`, which requests a quote and swap instructions from Jupiter endpoints to atomically swap SOL into 2Z in order to convert SOL to 2Z for the Revenue Distribution program (by atomically buying SOL in conjunction with the Jupiter route). The round-trip SOL -> 2Z -> SOL pathway is protected by using the swap rate fetched by the SOL conversion oracle. There must be enough SOL in the user's wallet in order for this command to succeed. The command itself is gasless, where your SOL balance should be unchanged after execution. Once executed, the user will have harvested some amount of 2Z tokens. If the user does not currently have a 2Z associated token account, creating this token account will result in less 2Z harvested, but will still preserve your SOL balance after execution. To execute, run the following: ```sh doublezero-solana revenue-distribution harvest-2z ``` Output will resemble: ```sh Harvested 5.98151278 2Z tokens with 1.000000000 SOL ``` The command can also be simulated with the `--dry-run` argument. Dry-run will produce program logs and an output at the end resembling: ```sh Simulated harvesting 5.98151278 2Z tokens with 1.000000000 SOL ``` ## Testing Verification Performed the command with dry-run mode.

…` argument (malbeclabs/doublezero-offchain#182) ## Summary of Changes This PR adds more view modes for distribution accounting. These views help give a better view of all of the accounting that goes into the lifecycle of a distribution. These views fetch the records posted by the protocol's accountant processes and exposes how the distribution tracks this data (e.g. how much a contributor is rewarded and whether its rewards were distributed). - Add `--view` argument with the following values: - `summary` -- Default view. Shows aggregate information about fee configuration, debt and rewards. - `validator-debt` -- Record of all Solana validator debt owed and whether it has been processed. - `unprocessed-validator-debt` -- Similar to `validator-debt` but only shows unprocessed debt. - `rewards` -- Record of all network contributors reward distributions. - Cleaned up default fetch (now `summary` view) for distributions. Examples of these views will be posted as comments. This change also affected public interfaces to the `doublezero-solana-validator-debt` and `doublezero-contributor-rewards` crates, which is why I also updated these CHANGELOGs. Closes #2035. ## Testing Verification * Tested views locally (examples will be in the comments).

…ero-offchain#184) # Summary Fix https://github.com/malbeclabs/infra/issues/346 Fixes connection reset retry logic that was added in malbeclabs/doublezero-offchain#177 but didn't work properly. - The previous fix attempted to detect "connection reset" errors by checking if `reqwest_err.to_string().contains("connection reset")`, but this never matched because `Display` for `Kind::Request` only outputs `"error sending request"` without the nested source details - This caused connection reset errors to bypass retry logic and immediately log at ERROR level, triggering Grafana alerts - This implementation walks the error source chain (following the same pattern as `reqwest::Error::is_timeout()`, which you can refer [here](https://github.com/seanmonstar/reqwest/blob/a97e1956dd14a79b0207082e327098331519bf2b/src/error.rs#L116)) to find the underlying `io::Error` and check for `io::ErrorKind::ConnectionReset`

…bin (malbeclabs/doublezero-offchain#185) ## Summary of Changes * Change installation directory for binaries from /usr/local/bin/ to /usr/bin/ * Linux distributions require that packages do not install files into /usr/local/: * https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs * https://docs.fedoraproject.org/en-US/packaging-guidelines/ Closes #2083. ## Testing Verification * Built packages locally with `goreleaser --snapshot --clean`

…clabs/doublezero-offchain#187) ## Summary of Changes * Binaries were moved from /usr/local/bin/ to /usr/bin/ in a previous PR * This PR updates changelogs and systemd units to reflect this change

…albeclabs/doublezero-offchain#186) # Summary Adds grace period checking before posting contributor rewards merkle roots to prevent on-chain transaction failures. The scheduler now waits for the Revenue Distribution program's grace period (default: 1 hour) to expire before attempting to post calculations. Fixes #2118 ## Changes - Add `check_calculation_allowed()` to validate grace period based on Distribution timestamp - Add `wait_for_grace_period()` to poll every 60s until grace period expires - Update `post_rewards_merkle_root()` to wait for grace period before posting - Add `grace_period_max_wait_seconds` configuration (default: 6 hours) - Update all tests to include new configuration field

…a-validator-debt (malbeclabs/doublezero-offchain#188) ## Summary of Changes * Release new versions of contributor-rewards, sentinel, and solana-validator-debt

…clabs/doublezero-offchain#189) ## Summary of Changes - Remove the default Jupiter route to allow any route from SOL -> 2Z. - Add `--specific-dex` optional argument to allow the user to specify one DEX program label. For example: ```sh doublezero-solana revenue-distribution harvest-2z --specific-dex HumidiFi ``` See https://lite-api.jup.ag/swap/v1/program-id-to-label for reference. ## Testing Verification Tested command with `--dry-run`.

…rvest-2z` (malbeclabs/doublezero-offchain#190) ## Summary of Changes The default limit price was not factoring the discounted price. This change adds this, which makes the atomic swap in `harvest-2z` safer. Also added a check in `harvest-2z` in the simulation to ensure that the token balance change increases. ## Testing Verification Executed `convert-2z` and `harvest-2z` with `--dry-run`.

## Summary of Changes Uptick crate version to 0.2.2 to prepare for release. ## Testing Verification N/A

…ublezero-solana#86) We do not want the system to finalize a null rewards root if there is revenue to distribute for a given epoch. Presumably there is revenue because users are using the network, so there should be contributors to reward. Closes #2098.

…/doublezero-offchain#195) ## Summary of Changes Companion PR to malbeclabs/doublezero-solana#87. Whenever the Revenue Distribution program is upgraded to take this change, admin needs to call the migration instruction after the upgrade, which will fix the Journal state. This command needs to reflect the new way the migration instruction must be called. Closes #2144. ## Testing Verification CI in malbeclabs/doublezero-solana#87 forks mainnet to test this CLI command.

…ero-solana#87) This change requires performing a migration that fixes the Journal's balance after an onchain upgrade. Closes #2097.

…lezero-offchain#183) ## Summary of Changes This PR adds a new app called `scheduler` to manage the debt lifecycle processes. It's written in Elixir which is a new language and adds some risk as only Rahul and I know Elixir well. However, this is done not simply to increase complexity but because it provides a lightweight and reliable scheduling mechanism through Erlang's process supervision. This could be written in Rust but it would take longer and be less reliable. It might be worth rewriting in Rust at some point. It'd be pretty simple as the surface area is intentionally kept as small as possible. It uses the same entrypoint as the CLI so there aren't competing implementations. Rust is interfaced through a NIF (native implemented function) which is like foreign function interface (FFI) in other languages. There's a thin layer written in Rust that calls into the `validator-debt` crate et al. The way this works is that there is an application [supervisor](https://hexdocs.pm/elixir/Supervisor.html) that manages processes. Any of the lifecycle processes will be managed through a [GenServer](https://hexdocs.pm/elixir/1.19.2/GenServer.html) which basically runs in a loop executing some component of the debt lifecycle process at some interval. Intervals are configured using cron syntax. `CHANGELOG.MD` added. Note there is some followup work to extract some of the logic from the CLI into a shared place, perhaps `solana-tools` but that will be done in a subsequent PR. Closes #2029 ## Testing Verification This has been tested running every two hours with results being posted to slack as seen [here](https://malbeclabs.slack.com/archives/C09LES1Q127). The app was run from my local machine for more than an day without interruption or errors.

…bs/doublezero-offchain#196) # Summary This PR replaces the point-in-time access pass validator selection approach with the canonical time-series approach that matches the Python script used for [fees](https://github.com/doublezerofoundation/fees/). Fetches validator pubkeys from S3 hourly Parquet snapshots and applies the 12-hour connection rule (validators must appear in >12 hourly snapshots to qualify). Fixes #2039 ## Changes - **New `s3_fetcher` module** - Fetches hourly Parquet snapshots from S3 for a given Solana epoch - Merges gossip, validators, users, and devices datasets - Applies 12-hour connection rule (>12 appearances required) - Supports mainnet-beta and testnet (probably just remove testnet?) - Implements consecutive failure tracking (default 12, configurable) - **Updated `worker.rs`** - Integrated S3 fetcher to replace access pass approach - Added CSV validation logic for testing (strict 100% match required) - Removed deprecated access pass code - Now uses S3-fetched validators for reward calculations - **New `export-validators` command** - CLI: `solana-validator-debt export-validators --epoch <EPOCH> [--output <CSV>]` - Exports validator pubkeys per epoch to CSV format - Mostly for validation and debugging - **Required Env Vars:** - `VALIDATOR_DEBT_AWS_ACCESS_KEY_ID` - AWS access key - `VALIDATOR_DEBT_AWS_SECRET_ACCESS_KEY` - AWS secret key - **Optional Env Vars:** - `VALIDATOR_DEBT_S3_BUCKET` (default: "malbeclabs-data-metrics-dev") - `VALIDATOR_DEBT_AWS_REGION` (default: "us-east-1") - `VALIDATOR_DEBT_S3_MAX_CONSECUTIVE_FAILURES` (default: 12) - `VALIDATOR_DEBT_S3_ENDPOINT` (for MinIO compatibility) ## Testing ``` +---------+----------+--------+----------+-------------+---------------+ | Epoch | Status | Rust | Python | Rust Only | Python Only | |---------+----------+--------+----------+-------------+---------------| | 859 | ✓ | 380 | 380 | 0 | 0 | | 860 | ✓ | 385 | 385 | 0 | 0 | | 861 | ✓ | 390 | 390 | 0 | 0 | | 862 | ✓ | 397 | 397 | 0 | 0 | | 863 | ✓ | 397 | 397 | 0 | 0 | | 864 | ✓ | 401 | 401 | 0 | 0 | | 865 | ✓ | 407 | 407 | 0 | 0 | | 866 | ✓ | 411 | 411 | 0 | 0 | | 867 | ✓ | 411 | 411 | 0 | 0 | | 868 | ✓ | 410 | 410 | 0 | 0 | | 869 | ✓ | 406 | 406 | 0 | 0 | | 870 | ✓ | 399 | 399 | 0 | 0 | | 871 | ✓ | 400 | 400 | 0 | 0 | | 872 | ✓ | 403 | 403 | 0 | 0 | | 873 | ✓ | 403 | 403 | 0 | 0 | | 874 | ✓ | 382 | 382 | 0 | 0 | | 875 | ✓ | 385 | 385 | 0 | 0 | | 876 | ✓ | 387 | 387 | 0 | 0 | | 877 | ✓ | 387 | 387 | 0 | 0 | | 878 | ✓ | 385 | 385 | 0 | 0 | +---------+----------+--------+----------+-------------+---------------+ ``` ## TODO - [x] Validate results against more CSV files from [fees](https://github.com/doublezerofoundation/fees) - [x] Remove debugging - [x] Remove CSV validation logic - [x] Remove CSV path parameter from `calculate_validator_debt` - [x] Perhaps remove "access pass blocked" bail entirely --------- Co-authored-by: Ben Marx <bgm@malbeclabs.com>

…ler (malbeclabs/doublezero-offchain#197) ## Summary of Changes This PR adds in a worker to manage initializing distributions automatically. It runs every two minutes. This PR also moved some of the logic for initializing a distribution into the validator-debt worker so that both the scheduler and the CLI have a common, minimal interface into the underlying code. Closes #2141. ## Testing Verification The current distribution has already been initialized but here's evidence of the worker calling the underlying rust code: ``` 14:02:00.650 [debug] Scheduling job for execution 14:02:00.658 [debug] Task for job started on node 14:02:00.658 [debug] Execute started for job 14:02:00.760 [debug] Execution ended for job 14:02:01.328 [error] initialize_distribution: received error: "Last completed DZ epoch 52 != program's epoch 53" 14:02:01.331 [error] GenServer Scheduler.Worker.InitializeDistribution terminating ** (stop) "initialize_distribution shutting down" Last message: {:continue, :initialize_distribution} ```

…/doublezero-offchain#198) # Summary contributor-rewards scheduler was entering a permanent failure state after 10 consecutive RPC failures, requiring manual intervention to recover. This PR removes the hard halt and implements self-healing retry logic with monitoring alerts. Fixes https://github.com/malbeclabs/infra/issues/368 ## Changes - Remove hard halt: Scheduler now retries indefinitely instead of halting after max failures - Add RPC retry with backoff: Wrap get_epoch_info() with backoff - Add periodic alerting: Emit ERROR log every 10 consecutive failures for monitoring - Remove max_consecutive_failures config: No longer needed since hard halt is removed

…albeclabs/doublezero-solana#88) Open for comments. Closes #2093.

…ro-offchain#199) ## Summary of Changes This PR adds in a worker to automatically run calculate_distribution. It follows the existing pattern of having a light interface into the underlying validator-debt crate and runs every two hours. It works by getting the current dz_epoch, then subtracting one epoch to get the most recently completed epoch. If the epoch is already finalized, the worker just shuts down. Otherwise it continues its run as usual. In a subsequent PR, we'll add a counter to this that must be reached before the debt is finalized. Note that due to the differences in floating point math (how "society" rounds - round 5 up) and scalar math (safer with integers - floor) there can be a 1 lamport difference between Nihar's CSV and the calculations provided by the `mul_scalar/1` method that Karl created. We've verified for every epoch that there's no greater than 1 lamport difference between the csv and the calculations in the validator-debt crate. Closes #2027 ## Testing Verification This has been tested and results posted in [#tmp-validator-debt](https://malbeclabs.slack.com/archives/C09LES1Q127/p1763490621115539)

…lezero-offchain#200) ## Summary of Changes This PR updates the calculate_distribution worker to add in a counter that increments each time the calculate_distribution function successfully completes. When the counter reaches three, it moves to the finalize_distribution callback where the finalize_distribution NIF is called. If any of the calls fail or the computed_debt differs from the ledger debt record, the process is stopped and the counter is reset to zero. Note that only the third successful run is posted to slack. Closes #2028 ## Testing Verification * This was run and posted to slack as evidenced [here](https://malbeclabs.slack.com/archives/C09LES1Q127/p1763585298695259)

…mmand (malbeclabs/doublezero-offchain#201) ## Summary of Changes Add Solana validator debts view to `doublezero-solana` CLI. There are two `--view` modes: `outstanding` and `node`. Outstanding has an optional `--node-id` parameter to calculate the total outstanding debt for a given validator node ID. If unspecified, the command will show all validators with outstanding debt. The node view shows all debt statuses for every relevant Solana epoch for a given `--node-id`. Other changes include: - Remove `fetch journal` from CLI. - Remove `--dz-ledger-url` where the specific DoubleZero Ledger connection can be derived from the Solana connection. - Better account fetching and account data processing in Solana tools. - Refactor CLI based on tools changes. - Add `debt_record_key` to `doublezero-solana-validator-debt` crate. - Specify minor versions of zero-versioned dependencies and removed unused dependencies from workspace. Closes #2045. ## Testing Verification Ran new and existing fetch commands locally.

…otency (malbeclabs/doublezero-offchain#202) # Summary Fixes #2199 The idempotency check only verified on-chain records (steps 1-4) but didn't check if the merkle root was posted to the Distribution account (step 5). This caused epochs to be incorrectly marked as processed when only partial work was completed. Changes: - Add check_distribution_merkle_root() method to verify Distribution account - Update rewards_exist_for_epoch() to require both records AND merkle root - Add retry logic with exponential backoff for Distribution account fetch The scheduler will now retry epochs indefinitely until ALL conditions are met: - Records exist (shapley output or reward input) - Distribution account exists for that epoch - Distribution.rewards_merkle_root != Hash::default() (non-zero) - Distribution.total_contributors > 0

…clabs/doublezero-offchain#203) # Summary Add 5 individual skip flags to the calculate-rewards command to allow selective execution of write operations, providing more flexibility than the all-or-nothing --dry-run flag. Fixes: #2206 This enables scenarios like: - Reposting only merkle root without rewriting telemetry - Updating specific ledger records while preserving others - Testing individual write steps in isolation - Recovery from partial failures with selective re-execution ## Changes - Add `WriteConfig` struct with tests - Add `--skip-device-telemetry` flag for device telemetry write - Add `--skip-internet-telemetry` flag for internet telemetry write - Add `--skip-reward-input` flag for reward input write - Add `--skip-shapley-output` flag for shapley output storage write - Add `--skip-merkle-root` flag for merkle root posting to Solana

…albeclabs/doublezero-offchain#204) ## Summary of Changes ## Testing Verification * Ran it locally with [act](https://github.com/nektos/act) * Doing it live

…albeclabs/doublezero-offchain#206) # Summary This PR adds Slack webhook integration to notify when contributor rewards cycles complete, with detailed tracking of all write operations and their on-chain identifiers. Fixes: #2211 ## Changes - Add `SlackSettings` to configuration with webhook URL and channel ID - Extend `WriteSummary` to track identifiers (record addresses/transaction signatures) for each write operation - Update `post_rewards_merkle_root()` to return transaction signature instead of () - Update `calculate_rewards()` to return `WriteSummary` with all operation results - Add `slack-notifier::contributor_rewards` module with markdown table formatting - Integrate Slack notifications into scheduler worker (automatic after successful completion) - Add `--slack-notify` CLI flag for testing notifications with `calculate-rewards` command - Add Slack settings validation (webhook URL format, required fields)

…o-offchain#2193) (malbeclabs/doublezero-offchain#205) ## Summary of Changes This PR adds goreleaser and actions so that the scheduler can be deployed to remote environments Closes #2193 ## Testing Verification * This has been deployed to both testnet and mainnet-beta and the binary is in [cloudsmith](https://cloudsmith.io/~malbeclabs/repos/doublezero/packages/detail/deb/doublezero-offchain-scheduler/0.1.0-1/a=amd64;xc=main;d=any-distro%252Fany-version;t=binary/)

…-solana#89) I spent a good few hours trying to run the program tests on this suite. Turns out, all I needed to do was run the Makefile. Please update the README to reflect this change.

…albeclabs/doublezero-offchain#408) ## Summary - Add a `squads` module with Squads Protocol v4 vault support: the `SquadsArgs` and `OptionalSquadsArgs` clap argument structs, vault derivation and multisig verification, loader-v3 authority decoding, and base58 encoding of vault transactions for import into the Squads UI - Gate it behind a `squads` feature, on by default, so consumers can opt out with `default-features = false` - Take `Message` from `solana-message` and `UpgradeableLoaderState` from `solana-loader-v3-interface` rather than through `solana-sdk`, which deprecates the former re-export and does not carry the latter - Add a crate `README.md` covering the new module, with a placeholder for the rest of the crate ## Testing Eight unit tests cover the vault index rule across every `NetworkEnvironment`, the inspector URL encoding (including an endpoint that carries its own query string, where unescaped separators would otherwise read as parameters of the explorer link), the endpoint sharing warning, and loader-v3 authority decoding across the present, absent, and wrong-account-kind cases. Vault derivation is pinned by a known-answer test against a real devnet multisig and its vault, and the encoded payload is decoded back to assert the vault is fee payer and sole signer with a zeroed blockhash.

…quads vaults (malbeclabs/doublezero-offchain#409) Closes #4184 ## Summary - Add `vault_transaction_payload_budget`, reporting the bytes a payload of a given instruction count may serialize to, for a caller that grows a single instruction until it stops fitting - Add `try_encode_vault_transaction`, which measures the payload and refuses one that cannot work. `print_vault_transaction` becomes `try_print_vault_transaction` and routes through it - Derive the reserve as const arithmetic rather than in a comment, one term per line, from the `vault_transaction_create` envelope plus the compute budget pair, `proposal_create`, and `proposal_approve` the Squads app bundles into that same transaction. Compile-time assertions pin each block's subtotal and the 384 total, so a changed term fails the build instead of leaving a stale literal. A comment records the three app behaviors the derivation assumes away and what each costs - Refuse a payload naming a signer other than the vault, and one carrying more than 48 instructions. Both import and collect approvals before failing at execute, unlike an oversized payload, which fails at import - Check the `vault_transaction_execute` transaction as well, at 329 bytes plus 33 per payload account key. The issue specifies 277, which omits the compute budget pair the app bundles there too - Record the conventions this change was written under in `CLAUDE.md`, covering numeric literals, prose in comments and documentation, the `try_` prefix, and where workspace dependency features belong ## Testing - Hand-computed payload lengths at 3, 127 and 128 instruction-data bytes, spanning the legacy length prefix boundary - A payload landing exactly on the budget and one byte over it, at one instruction and at two - The wrapper Squads builds around a budget-sized payload, assembled and measured at 1,229 bytes of the 1,232 available - The widest payload the budget accepts, at 24 account keys and 1,121 bytes to execute - Refusals for an empty payload, a foreign signer, and 49 instructions - A downstream caller's vault print path against devnet, run before and after the change, byte-identical

…eclabs/doublezero-offchain#410) Closes #4183 ## Summary #### solana-sdk - Add `ValidatorClientRewards` as a `Pod` mirror of the onchain struct, with a `PrecomputedDiscriminator` impl, a `Default` impl matching the program's, a `checked_short_description` accessor, and a compile-time assertion pinning the account at 184 bytes - Delete the five `VCR_*_OFFSET` constants, `VCR_SHORT_DESCRIPTION_LEN`, `VCR_ACCOUNT_DATA_LEN`, the standalone discriminator constant, `parse_validator_client_rewards`, and `ValidatorClientRewardsInfo`. None of them shipped, so the changelog bullet that introduced them is rewritten rather than paired with a removal entry - Fold the new mirror into the existing mirrored-layout section alongside `ShredRewardToken` and `ValidatorPublisherRewards` rather than opening a second one #### solana-cli - Read the account through the mirror in `show`, `claim`, and `init-holding`. All three decode the whole struct, so they require at least the 184 bytes the program allocates - `claim` takes its post-transaction count from `SolanaConnection::try_fetch_zero_copy_data_with_commitment`, which warns on a missing or undecodable account where the previous code printed `(unavailable)` silently #### solana-fork - Build the synthetic account from `ValidatorClientRewards::default()` plus field assignment instead of copying bytes to hand-written offsets, and size the rent exemption from the built buffer. The parse-back round trip and its three `ensure!` checks go with them, being tautological against a struct the same function builds - Rename `--synthetic-vcr-manager` to `--synthetic-validator-client-rewards-manager`, along with the fork test script and the local-validator workflow #### repo - Record the conventions this change was written under in `CLAUDE.md`: the `_key` suffix for pubkey bindings, and which helper to reach for when reading a zero-copy account ## Testing - Unit tests across the three crates, including three cases for `checked_short_description` and an exact-match assertion pinning the rendered `show` summary byte for byte, which is what guards the format string's line continuations against a dropped escape - The `local-validator` workflow drives `show`, `init-holding`, and `claim` against a synthetic account baked into the fork at genesis, covering the renamed flag and the mirror's field offsets end to end

…wed (malbeclabs/doublezero-solana#123) ## Summary of Changes * `FinalizeDistributionRewards` now rejects a null rewards root whenever 2Z is owed to contributors — nonzero collected 2Z or an uncollected registered integration — not just when SOL debt is nonzero. * `CollectIntegrationRewards` now rejects an integration registered after the distribution's snapshot. * Why: finalize is permissionless and one-way. A null root latched while 2Z sits in the distribution strands that 2Z permanently; the old guard only covered SOL debt. Fixes malbeclabs/infra#1868. ## Testing Verification * New regression tests cover both guards. Full revenue-distribution SBF suite passes locally with the pinned Solana v3.0.12 toolchain under both feature sets; fmt and clippy are clean; refreshed sha256sums validated by reproducing the committed passport hashes.

…0ms slot times (malbeclabs/doublezero-offchain#411) ## Summary of Changes Five call sites divided wall clock by a hardcoded 400ms slot duration. They get three independent fixes rather than one shared helper, because they want different answers: the two accuracy-critical sites stop depending on a slot-duration constant at all, and the two that keep one now share a single value. * **`contributor-rewards`** — `find_epoch_at_timestamp` verifies its estimate against real block times. The old estimate drifted ~30k slots per day of lookback and no fixed constant survives the SIMD-0525 rollout. That epoch picks the leader schedule rewards are computed against, so the search errors rather than guessing. * **`solana-cli`** — the duplicate constants in `shreds pay` and `prepare-offchain-message` collapse into one 350ms `NOMINAL_SLOT_DURATION`, deliberately cluster-independent: a `~` prefixed estimate and a deadline slot the CLI and operator must both compute want reproducibility over accuracy. * **`validator-debt`** — deletes two dead timestamp-to-epoch paths and both constants. `rpc.rs` already does this mapping correctly for production; the deleted code had no callers and contained an unsigned subtraction that panicked in debug and wrapped in release. ### Before merging * **`--valid-for` windows lengthen.** `--valid-for 1h` is now 10,285 slots, not 9,000. Mainnet reaches 350ms at epoch 1020 (2026-08-21); until then the flag grants ~68 minutes. Testnet runs at 200ms, so ~34 minutes. The help text states the conversion rate. * **The demand path can now fail where it used to be wrong.** `ingestor/demand.rs` propagates a leader-schedule error, so a backfill older than the endpoint's ledger retention errors instead of silently mis-estimating. The snapshot paths warn and continue, but `snapshot` now validates before writing rather than leaving an unusable file behind. * **350ms needs one more bump** when SIMD-0525 finishes stepping mainnet to 200ms. Nothing enforces it; the constants' comments record the schedule. ## Testing Verification * Pure-function tests for the search's per-step decision (both bounds, the unbounded current-epoch case, the unstarted-epoch case), the absent-block classifier (each of the three shapes `getBlockTime` uses, plus a pruned ledger failing closed), and the epoch-boundary skip budget (exclusive edge, a history gap, and a short-epoch cluster). No RPC mock, and no new dependency. * CLI assertions recomputed by hand from the 350ms derivation, with the arithmetic in each test comment. The deletions are checked by `clippy -Dwarnings`, which catches every orphaned import and unused constant.

…s org (malbeclabs/doublezero-offchain#412) Ahead of moving `doublezero-offchain` and `doublezero-solana` from the `doublezerofoundation` org to `malbeclabs`. ## ⚠️ Do not merge before the transfer CI on this PR fails, and that is expected. `malbeclabs/doublezero-solana` does not exist yet, so cargo cannot resolve the new dep URL. The same reason makes the local pre-commit clippy hook fail, so the commit skipped it. Merge order: transfer both repos, then merge this, then re-run CI. ## What changed | File | Change | Why | |---|---|---| | `release/.goreleaser.*.yaml` (5) | `owner: doublezerofoundation` → `malbeclabs` | **The one real breakage.** GoReleaser writes the GitHub release to the owner named here. The new repo's `GITHUB_TOKEN` has no write access to the old org, so every release workflow fails until this lands. | | `Cargo.toml` | 3 `doublezero-solana` git deps | Tag `revenue-distribution/v0.3.7` is unchanged, so the same commit resolves from the new URL. | | `Cargo.lock` | 3 matching `source =` lines | URL swap only. The pinned sha `4368da2c` does not change, so no re-resolution. | | `Cargo.toml` | `repository`, `homepage` | Crate metadata. | | `scripts/install-doublezero-solana.sh` | `REPO=` and 4 curl examples | Public install path validators run. | | `CONTRIBUTING.md` | `doublezero-rewarder` → `doublezero-offchain` | That old name already redirects here. Left alone it would take two hops after the transfer. | ## Deliberately left alone - **`network-shapley-rs`** stays on `doublezerofoundation`. That repo is not moving and it is public, so the dep keeps working. Possible follow-up. - **CHANGELOG.md link footers** (~14 files). GitHub keeps redirects for old PR and release URLs, so these stay correct. ## Not a code change, still needed - Re-add the four repo Actions secrets if the transfer drops them: `CLOUDSMITH_TOKEN`, `GORELEASER_KEY`, `SLACK_BOTS_WEBHOOK`, `MALBECLABS_DOUBLEZERO_SSH`. The SSH one looks unused, every git dep is public. - Check the `main` ruleset and the `copilot` environment survived. - Cloudsmith is unaffected. Those repos (`doublezero`, `doublezero-testnet`, `doublezero-devnet`) live on Cloudsmith, not GitHub, and the apt package names do not change.

…clabs (malbeclabs/doublezero-solana#125) Ahead of moving `doublezero-solana` and `doublezero-offchain` from the `doublezerofoundation` org to `malbeclabs`. ## ⚠️ Do not merge before the transfer `malbeclabs/doublezero-offchain` does not exist yet, so the `local-validator` workflow fails until both repos have moved. Merge order: transfer both repos, then merge this. ## What changed | File | Change | |---|---| | `.github/workflows/local-validator.yml` | `DOUBLEZERO_OFFCHAIN_GIT_INSTALL` — the `cargo install --git` URL for the offchain repo | | `programs/revenue-distribution/src/state/program_config/mod.rs` | RFC-0002 doc link | That is everything outside the changelogs. This repo is clean otherwise: no submodules, no crates.io publishing (`program-tools` is `publish = false`), no container images, and `verify-build.yml` only compares checksums, so no verified build is bound to the repo URL. ## Deliberately left alone **CHANGELOG.md link footers** (3 files, ~70 links to old PRs and tags). GitHub keeps redirects for those. ## Not a code change, still needed `.github/CODEOWNERS` names `@karl-dz` on every rule. A CODEOWNERS entry goes **silently dead** if the user lacks write access on the repo, with no warning. Add them to the `malbeclabs` org with write access on this repo before or with the transfer, otherwise review assignment stops working and nobody notices. Also worth checking after the move: the `main` ruleset and the `copilot` environment. This repo has no Actions secrets, so nothing to re-add. The wiki (enabled here) and the 5 forks travel with the repo.

…no leader schedule (malbeclabs/doublezero-offchain#413) ## Summary of Changes * The scheduler no longer writes a snapshot it cannot use: a failed leader-schedule fetch propagates instead of becoming `leader_schedule: null`, and `create_epoch_snapshot` validates before saving. * A failed fetch was one `warn!`, then an unusable snapshot uploaded over the epoch's canonical S3 key; the tick then failed reading it back with `Missing leader schedule`, a symptom whose cause survived only in the log. malbeclabs/doublezero-offchain#411 fixed this for the `snapshot` CLI command but not for the scheduler, which is the path that runs in production. * Scheduler failures now log the full cause chain (`{:#}`). Plain `{}` prints only the outermost context, so the newly propagated reason would still have been dropped. * **Security:** `EpochFinder`'s retry logs printed `reqwest`'s error verbatim, which includes the request URL. That URL carries the read endpoint's API key on mainnet-beta and journald ships to Loki, so those logs now strip it. This matters because the companion PR points mainnet-beta's reads at the keyed endpoint. * **Behavior change:** a `--dry-run` tick that previously marked the epoch processed with an unusable snapshot now fails and retries — nothing validated on that path, since `calculate_rewards` never ran. Both deployed environments run with dry-run off. ## Testing Verification * New `tests/test_snapshot_validate.rs` builds a `CompleteSnapshot` from the existing `testnet_snapshot.json` and `leader-schedule-epoch-89.json` fixtures: `validate()` is `Ok` on a complete snapshot, and names the right issue when the leader schedule is missing or empty.

…abs/doublezero-offchain#414)

Copilot

Pull request overview

This pull request imports the doublezero-offchain and doublezero-solana repositories into this monorepo under offchain/ and solana/ while keeping their histories and tags intact. It supports the monorepo migration plan by bringing both codebases in-tree without changing the existing workspace build behavior.

Changes:

Add the offchain/ and solana/ source trees as nested workspaces.
Exclude offchain/ and solana/ from the root Cargo workspace so the root build remains unchanged.
Add a root CHANGELOG.md entry that documents the import.

Reviewed changes

Copilot reviewed 83 out of 424 changed files in this pull request and generated 6 comments.

File	Description
`Cargo.toml`	Excludes `offchain/` and `solana/` from the root workspace to avoid changing root builds.
`CHANGELOG.md`	Documents the repository import in the root changelog.
`offchain/…`	Imported offchain tools and services, including the scheduler and Rust crates.
`solana/…`	Imported Solana programs, mocks, and shared crates as a nested workspace.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

ben-dz

A few below that seem easier to solve before merging:

Merging is blocked by required_linear_history AND required_signatures, not just the linear history. Both need to be lifted to merge, as GitHub reports the 550 imported commits as unverified.
Cargo.toml:25 — Nothing in this repo builds or tests either imported tree, but Dependabot still edits them. These exclude entries keep offchain/ and solana/ out of every root job, while .github/dependabot.yml's cargo directories: "**/*" picks up their manifests: the next grouped rust-minor-patch PR — #4237 currently spans 5 directories, and they land weekly, not monthly — will carry offchain/Cargo.lock and solana/Cargo.lock to an all-green merge with nothing compiled, diverging the trees from the repos that still release them. The generate-fixtures parallel does not hold: sdk-fixture-drift runs cargo run in all four of those. directories has no exclusion syntax, so the fix is enumerating the cargo directories. The same hole swallows a human edit to either tree.
CHANGELOG.md:19 — 451 of the imported commit subjects end in a bare (#N) that meant a PR in the source repo but resolves to an unrelated one here — 377 distinct numbers, and every sampled one exists in this repo. Once they land on main each renders as a link to the wrong pull request permanently, and GitHub posts cross-reference backlinks on those threads. Only fixable before the merge: git filter-repo --message-callback can rewrite them to (malbeclabs/doublezero-offchain#N) in the same pass that relocates the tree.
CHANGELOG.md:19 — Nine tags do not come across. The sources hold 126 but only 117 are reachable from their main, so contributor-rewards/v0.2.2, offchain-scheduler/v0.1.0, doublezero-solana/v0.0.1 and six rc tags are dropped — six of the nine have published GitHub Releases. Same orphan class as the sentinel/v0.6.1 handled under D5, but the entry reads as if all tags survive.

Relocated with git-filter-repo --to-subdirectory-filter offchain, so every path from the source repo lands under offchain/ and nothing existing moves. All 110 tags come across unprefixed, which keeps each component's version line continuous. Commit messages are rewritten in the same pass so that a bare (#N), which meant a pull request in the source repo, becomes malbeclabs/doublezero-offchain#N. Left alone, 451 of them would resolve to unrelated pull requests in this repo and post cross-reference backlinks on those threads. Step 2 of the monorepo migration.

Relocated with git-filter-repo --to-subdirectory-filter solana. The tree keeps its own workspace, lockfile and rust-toolchain.toml, so it builds exactly as it does today. Commit messages are rewritten so a bare (#N) becomes malbeclabs/doublezero-solana#N. Step 2 of the monorepo migration.

Both trees keep their own Cargo.toml, Cargo.lock and rust-toolchain.toml and build as nested workspaces, exactly as they did in their own repos. Step 4 of the monorepo migration folds offchain into the root workspace. The solana tree stays excluded because 62 of its 96 closure crates resolve differently under a shared lockfile. The cargo entry in dependabot.yml globbed "**/*", which would have found both imported manifests. No job here builds either tree, so a grouped update would have edited their lockfiles, merged green with nothing compiled, and diverged them from the repos that still release them. The five directories listed are every one that has its own lockfile, which is what the glob resolved to before the import.

) Step 3 of the monorepo migration. **Stacked on #4240** and based on that branch, so the diff shown here is step 3 alone. It retargets to `main` once #4240 merges. ## Summary Ten git dependencies in `offchain/Cargo.toml` become path dependencies: - Seven on `malbeclabs/doublezero`, pinned to `client/v0.31.0`, now point into `crates/`, `config/` and `smartcontract/`. - Three on `malbeclabs/doublezero-solana`, pinned to `revenue-distribution/v0.3.7`, now point into `solana/`. **All ten pins stop existing.** That removes the failure of 2026-08-26 as a class: cargo treats a git dependency's URL and revision as part of the crate identity, so two consumers naming the same crate differently build two copies whose types do not unify. It surfaced as 194 errors about methods that plainly existed, with nothing in the error text pointing at the cause. Both trees are still excluded nested workspaces with their own lockfiles. A path dependency may point outside its own workspace, so this lands before the workspaces merge in step 4. `network-shapley-rs` stays a git dependency, since it lives in another organization and is out of scope. ## The one code change Flipping the pins moves offchain from `client/v0.31.0` to current `main`: 80 commits and about 150 changed files across the seven packages. Exactly one thing broke, and the `contributor-rewards` golden tests from #414 are what caught it. `User.feed_pk: Pubkey` became `feed_pks: Vec<Pubkey>` in malbeclabs/infra#2114, and the committed snapshot predates the change. `apply_serviceability_json_compat_migrations` already backfilled the singular field, so it now folds it into the plural one. The subtlety is that this list serializes as a single comma-separated string rather than a JSON array, the same as its `publishers` and `subscribers` neighbors, so an absent feed is the empty string and not an empty array. **The goldens then matched exactly.** Moving to current `main` leaves every computed reward value unchanged, which is the assurance step 1 existed to provide. ## Testing Verification - No `git+` source for `malbeclabs/doublezero` or `malbeclabs/doublezero-solana` remains in `offchain/Cargo.lock`. The only git source left is `network-shapley-rs`, unchanged. - The `contributor-rewards` goldens pass and match, so the reward values this repo computes are identical before and after. - The offchain workspace passes its full suite, zero failures. - The solana tree is untouched by this pull request and still builds standalone. Its one failing test, `test_configure_program`, needs `make build-sbf` to have run first, as `solana/README.md` documents, and fails the same way on the base branch. - `cargo fmt --all --check` and `cargo clippy --workspace --all-targets` are clean on offchain. ## One thing to know for step 4 `rust-toolchain.toml` is directory-scoped, not workspace-scoped. Building from `offchain/` now compiles the root crates with offchain's **1.92.0** toolchain rather than the root's 1.97.1. It works today, and this pull request deliberately leaves it alone. Step 4 resolves it by deleting `offchain/rust-toolchain.toml` when the workspaces merge, along with the edition trap: all 14 offchain crates declare `edition.workspace = true` and would silently inherit the root's 2021 instead of their 2024.

`doublezero-offchain` merged #415, #416 and #417 after the import in #4240 captured its main at #414, so the imported tree was three commits behind. This replays them into `offchain/`, keeping each commit's author, date and message, with the bare pull request references rewritten to name the source repository exactly as the import rewrote its own 451. `offchain/Cargo.lock` is excluded from the replay, because step 4 deleted it. The version bump it carried lands in the root lockfile instead, which is where the crate resolves now. `solana/` needs no sync: its upstream tip is the tip that was imported.

bgm-malbeclabs and others added 30 commits November 4, 2025 16:00

add pr template (malbeclabs/doublezero-offchain#179)

58f0a07

Adds PR template same as doublezero repo

move binaries to /usr/bin: update changelogs and systemd units (malbe…

542c34c

…clabs/doublezero-offchain#187) ## Summary of Changes * Binaries were moved from /usr/local/bin/ to /usr/bin/ in a previous PR * This PR updates changelogs and systemd units to reflect this change

fix(workspace): version bump contributor-rewards, sentinel, and solan…

436cf6b

…a-validator-debt (malbeclabs/doublezero-offchain#188) ## Summary of Changes * Release new versions of contributor-rewards, sentinel, and solana-validator-debt

solana-cli: uptick to 0.2.2 (malbeclabs/doublezero-offchain#191)

fc0b351

## Summary of Changes Uptick crate version to 0.2.2 to prepare for release. ## Testing Verification N/A

revenue-distribution: fix swap balance in journal (malbeclabs/doublez…

56259a5

…ero-solana#87) This change requires performing a migration that fixes the Journal's balance after an onchain upgrade. Closes #2097.

docs/rfc: add 0001_ALLOW_WRITING_OFF_DEBT_FOR_SAME_DISTRIBUTION.md (m…

606beff

…albeclabs/doublezero-solana#88) Open for comments. Closes #2093.

add action to deploy scheduler (malbeclabs/doublezero-offchain#2193) (m…

7b0eabc

…albeclabs/doublezero-offchain#204) ## Summary of Changes ## Testing Verification * Ran it locally with [act](https://github.com/nektos/act) * Doing it live

docs(README): change build and test directions (malbeclabs/doublezero…

fc0845a

…-solana#89) I spent a good few hours trying to run the program tests on this suite. Turns out, all I needed to do was run the Makefile. Please update the README to reflect this change.

karl-dz and others added 10 commits August 10, 2026 18:14

remove CODEOWNERS (malbeclabs/doublezero-solana#124)

b15ff97

contributor-rewards: add golden tests covering reward values (malbecl…

64f5d13

…abs/doublezero-offchain#414)

bgm-malbeclabs requested review from a team and a lite review from Copilot August 27, 2026 23:37

Copilot started reviewing on behalf of bgm-malbeclabs August 27, 2026 23:37 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

ben-dz reviewed Aug 28, 2026

View reviewed changes

bgm-malbeclabs added 3 commits August 28, 2026 11:43

bgm-malbeclabs force-pushed the migrate/import-offchain-and-solana branch from 4e1ef8e to 8bb4791 Compare August 28, 2026 19:32

bgm-malbeclabs mentioned this pull request Aug 29, 2026

offchain: replace the ten git dependencies with path dependencies #4245

Merged

repo: merge main to pick up the six-shard e2e matrix

8d64f70

bgm-malbeclabs mentioned this pull request Sep 1, 2026

offchain/scheduler: fail the boot when SOLANA_RPC is unset #4254

Open

martinsander00 approved these changes Sep 1, 2026

View reviewed changes

ben-dz approved these changes Sep 1, 2026

View reviewed changes

bgm-malbeclabs mentioned this pull request Sep 1, 2026

offchain: sync the three commits that landed after the import #4257

Open

bgm-malbeclabs merged commit e0381d8 into main Sep 1, 2026
37 checks passed

bgm-malbeclabs deleted the migrate/import-offchain-and-solana branch September 1, 2026 22:06

Navigation Menu

Uh oh!

repo: import doublezero-offchain and doublezero-solana with their history - #4240

repo: import doublezero-offchain and doublezero-solana with their history#4240
bgm-malbeclabs merged 554 commits into
mainfrom
migrate/import-offchain-and-solana

bgm-malbeclabs commented Aug 27, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ben-dz left a comment •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Uh oh!

Conversation

bgm-malbeclabs commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Merge instructions

Size

Testing Verification

Follow-ups, deliberately not in this pull request

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ben-dz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

bgm-malbeclabs commented Aug 27, 2026 •

edited

Loading

ben-dz left a comment •

edited

Loading