Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions .agents/context/crates/magicblock-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,9 @@ Caveats:
5. Standalone/primary nodes reset the bank and, for primaries, send a replication `Message::Reset`.
6. The intent execution service starts only after replay/reset and handles pending commit intent recovery before its normal accept loop.
7. Standalone nodes switch the scheduler to `SchedulerMode::Primary`; primary/replica modes spawn the replication service instead.
8. Standalone ephemeral nodes start background base-layer setup: funding check, magic fee vault init/delegation, optional startup fee claim, and optional domain registration.
9. The undelegation request service, claim-fees periodic task, intent execution service, ledger truncator, and primary-only task scheduler are started.
8. Non-replica validators start `UndelegationRequestService`; it consumes Chainlink observed requests and the configured polling backfill. Replicas keep Chainlink disabled and do not scan DLP requests.
9. Standalone ephemeral nodes start background base-layer setup: funding check, magic fee vault init/delegation, optional startup fee claim, and optional domain registration.
10. Claim-fees periodic task, intent execution service, ledger truncator, and primary-only task scheduler are started.

### Scheduled intent service flow

Expand Down Expand Up @@ -219,9 +220,10 @@ The ledger stores a validator keypair file beside the blockstore parent. With `v
5. Ephemeral vault initialization must preserve the ephemeral flag and Magic Program owner.
6. Validator identity must remain privileged in local AccountsDb.
7. Committor service wiring must stay available before Magic Program execution can fetch commit nonces or schedule settlement work.
8. Do not add blocking RPC, slow I/O, or expensive serialization to scheduler/executor hot paths from this crate; keep such work in startup/background services where possible.
9. Shutdown must cancel/stop services before flushing AccountsDb and ledger.
10. Operator-facing config keys and base-layer registration/fee-vault behavior are compatibility-sensitive.
8. Owner-program DLP undelegation request polling must stay in the background processor path, not the slot ticker or transaction-execution hot path.
9. Do not add blocking RPC, slow I/O, or expensive serialization to scheduler/executor hot paths from this crate; keep such work in startup/background services where possible.
10. Shutdown must cancel/stop services before flushing AccountsDb and ledger.
11. Operator-facing config keys and base-layer registration/fee-vault behavior are compatibility-sensitive.

## Common change areas and what to inspect

Expand Down
12 changes: 11 additions & 1 deletion .agents/context/crates/magicblock-chainlink.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ Important methods:
- `fetch_accounts(pubkeys, fetch_origin)`: ensures accounts and then reads them from the local bank.
- `accounts_delegated_on_base_and_er(pubkeys, fetch_origin)`: checks that each account is DLP-owned on base and represented as delegated/DLP-owned locally.
- `undelegation_requested(pubkey)`: called by committor/account flows before an account is undelegated so Chainlink keeps watching for base-layer completion.
- `fetch_undelegation_requests()`: scans base-layer Delegation Program accounts for active `UndelegationRequest` PDAs using filtered `getProgramAccounts` and returns decoded `ObservedUndelegationRequest`s for `magicblock-accounts`.
- `fetch_count()` / `is_watching()`: mainly observability/testing helpers.

Disabled replication mode is intentionally conservative:
Expand Down Expand Up @@ -274,6 +275,15 @@ Key behavior:
- Non-advancing updates are ignored unless they represent a same-slot delegated refresh needed for undelegate/redelegate recovery.
- Delegated updates cause direct subscription cleanup; undelegation-completion updates retain/directly ensure subscriptions as appropriate and release `UndelegationTracking` ownership.

### DLP undelegation request scanning

Owner-program undelegation requests are discovered in two ways:

- Live updates: DLP-owned `UndelegationRequest` account subscription/program-subscription updates are decoded in `FetchCloner::process_subscription_update` and broadcast as `ObservedUndelegationRequest`.
- Backfill scans: `FetchCloner::fetch_undelegation_requests` calls `getProgramAccounts` for `dlp_api::id()` with a `DataSize(UndelegationRequest::size_with_discriminator())` filter and a discriminator `memcmp` at offset `0`, then decodes each returned account with `UndelegationRequest::try_from_bytes_with_discriminator`.

The scan uses Base64Zstd account encoding and gets a nearby base-chain slot for `observed_slot`. Malformed matching accounts are logged and skipped; a bad account must not abort the whole scan. Polling cadence is controlled by `chainlink.undelegation-request-poll-interval` in `magicblock-config` and consumed by `magicblock-accounts`.

### Greedy discovery

If a subscription update discovers a DLP-owned account absent from the bank, Chainlink may greedily fetch and clone it if the delegation record says it belongs to this validator (or is confined). This is especially important for delegated eATA discovery and owner-program subscriptions.
Expand Down Expand Up @@ -368,7 +378,7 @@ Changing SubMux behavior can affect ordering, duplicate clone submissions, and p

## Lifecycle mode and configuration

`ChainlinkConfig` wraps `RemoteAccountProviderConfig` and currently includes `remove_confined_accounts`.
`ChainlinkConfig` wraps `RemoteAccountProviderConfig` and includes settings such as `remove_confined_accounts`, allowed program filters, resubscription delay, Range risk checks, and `undelegation_request_poll_interval` for the DLP request backfill consumer in `magicblock-accounts`.

`RemoteAccountProviderConfig` includes:

Expand Down
5 changes: 4 additions & 1 deletion .agents/context/crates/magicblock-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ Most structs use `rename_all = "kebab-case"`; environment variables are upper sn
- `ledger.block-time` -> `MBV_LEDGER__BLOCK_TIME`
- `task-scheduler.failed-task-cleanup-interval` -> `MBV_TASK_SCHEDULER__FAILED_TASK_CLEANUP_INTERVAL`
- `chainlink.risk.request-timeout` -> `MBV_CHAINLINK__RISK__REQUEST_TIMEOUT`
- `chainlink.undelegation-request-poll-interval` -> `MBV_CHAINLINK__UNDELEGATION_REQUEST_POLL_INTERVAL`

Do not introduce aliases casually. Operator docs, `config.example.toml`, integration tooling, and deployment configs depend on stable keys.

Expand All @@ -182,7 +183,7 @@ Most config structs use `deny_unknown_fields`. This catches typos and stale conf

### Defaults are operational behavior

Defaults are not merely test conveniences. They set devnet remotes, local storage, development validator keypair, base fee, commit compute unit price, account DB size, ledger timing, metrics cadence, Chainlink monitoring capacity, Range risk defaults, task scheduler timings, and gRPC stream limits. Changing defaults can affect local developer flows, integration tests, startup performance, storage usage, and network traffic.
Defaults are not merely test conveniences. They set devnet remotes, local storage, development validator keypair, base fee, commit compute unit price, account DB size, ledger timing, metrics cadence, Chainlink monitoring capacity, DLP undelegation-request polling cadence, Range risk defaults, task scheduler timings, and gRPC stream limits. Changing defaults can affect local developer flows, integration tests, startup performance, storage usage, and network traffic.

### Config example is tested

Expand Down Expand Up @@ -277,12 +278,14 @@ Inspect first:
- `magicblock-config/src/config/chain.rs` and `grpc.rs`;
- `magicblock-api/src/magic_validator.rs::init_chainlink`;
- `magicblock-chainlink/src/remote_account_provider/`;
- `magicblock-services/src/undelegation_request_service.rs` for DLP undelegation-request polling;
- `magicblock-aml` Range risk usage;
- allowed-program filtering tests in Chainlink.

Risks:

- subscription limits and resubscription delay affect account sync throughput and provider load;
- undelegation-request polling adds periodic base-layer RPC load and should stay disabled with `0s` only when subscription backfill is not needed;
- allowed-program semantics treat `None` and `Some(vec![])` as unrestricted in current Chainlink code;
- risk config may add external I/O and should remain explicitly disabled by default.

Expand Down
10 changes: 9 additions & 1 deletion .agents/specs/validator-specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,13 @@ owner, rent payer, and commit nonce information must come from the delegation
record/metadata. Validators must still verify that the request PDA matches the
delegated account before scheduling.

The validator observes owner-program `UndelegationRequest` PDAs through
Chainlink and `magicblock-accounts`. Live request-account subscription updates
are the low-latency path. A background backfill loop also scans DLP-owned
accounts with a filtered `getProgramAccounts` every configured interval
(`chainlink.undelegation-request-poll-interval`, default `5m`) and feeds the
decoded requests into the same `ScheduleCommitAndUndelegate` scheduling path.

The current Delegation Program does not undelegate from finalize instructions.
Commit/finalize-style instructions only commit/finalize state and record or
preserve `DelegationMetadata.undelegation_requester`. The validator-side
Expand All @@ -278,7 +285,8 @@ and request account cleanup. When metadata says `Validator` or is still `None`
at task-build time, no request account is passed. `None` can be valid for
validator-requested undelegation because commit and finalize task lists are
built before the commit-stage transaction records the validator requester on
base.
base. `AlreadyUndelegated` remains a conflict for validator-requested
undelegation state, not a reason to drop an owner-program request.

### Callback discriminator

Expand Down
6 changes: 6 additions & 0 deletions config.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,12 @@ remove-confined-accounts = false
# Env: MBV_CHAINLINK__RESUBSCRIPTION_DELAY
# resubscription-delay = "50ms"

# Period for polling DLP-owned UndelegationRequest accounts as a backfill for
# missed subscription updates. Set to "0s" to disable polling.
# Default: "5m"
# Env: MBV_CHAINLINK__UNDELEGATION_REQUEST_POLL_INTERVAL
# undelegation-request-poll-interval = "5m"

# ------------------------------------------------------------------------------
# Optional: Range Risk API validation for post-delegation actions signers
# ------------------------------------------------------------------------------
Expand Down
1 change: 1 addition & 0 deletions magicblock-api/src/magic_validator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,7 @@ impl MagicValidator {
dispatch.transaction_scheduler.clone(),
identity_keypair.insecure_clone(),
ledger.latest_block().clone(),
config.chainlink.undelegation_request_poll_interval,
))
});

Expand Down
62 changes: 61 additions & 1 deletion magicblock-chainlink/src/chainlink/fetch_cloner/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ use std::{

use dlp_api::{
pda::delegation_record_pda_from_delegated_account,
state::{DelegationRecord, UndelegationRequest},
state::{
discriminator::AccountDiscriminator, DelegationRecord,
UndelegationRequest,
},
};
use lru::LruCache;
use magicblock_accounts_db::traits::AccountsBank;
Expand All @@ -30,8 +33,13 @@ use magicblock_metrics::metrics::{
use parking_lot::Mutex as PlMutex;
use scc::HashMap;
use solana_account::{AccountSharedData, ReadableAccount};
use solana_account_decoder_client_types::UiAccountEncoding;
use solana_keypair::Keypair;
use solana_pubkey::Pubkey;
use solana_rpc_client_api::{
config::{RpcAccountInfoConfig, RpcProgramAccountsConfig},
filter::{Memcmp, RpcFilterType},
};
use solana_sdk_ids::system_program;
use solana_signature::Signature;
use solana_signer::Signer;
Expand Down Expand Up @@ -334,6 +342,58 @@ where
.await?)
}

pub async fn fetch_undelegation_requests(
&self,
) -> ChainlinkResult<Vec<ObservedUndelegationRequest>> {
let observed_slot = self.remote_account_provider.get_slot().await?;
let config = RpcProgramAccountsConfig {
filters: Some(vec![
RpcFilterType::DataSize(
UndelegationRequest::size_with_discriminator() as u64,
),
RpcFilterType::Memcmp(Memcmp::new_raw_bytes(
0,
AccountDiscriminator::UndelegationRequest
.to_bytes()
.to_vec(),
)),
]),
account_config: RpcAccountInfoConfig {
encoding: Some(UiAccountEncoding::Base64Zstd),
..Default::default()
},
..Default::default()
};
let accounts = self
.remote_account_provider
.get_program_accounts_with_config(&dlp_api::id(), config)
.await?;

let mut requests = Vec::with_capacity(accounts.len());
for (request_pda, account) in accounts {
let Ok(request) =
UndelegationRequest::try_from_bytes_with_discriminator(
&account.data,
)
else {
warn!(
request_pda = %request_pda,
data_len = account.data.len(),
"Skipping malformed DLP undelegation request account"
);
continue;
};
requests.push(ObservedUndelegationRequest {
request_pda,
delegated_account: request.delegated_account,
expires_at_slot: request.expires_at_slot,
observed_slot,
});
}

Ok(requests)
}

Comment thread
snawaz marked this conversation as resolved.
pub fn cloner(&self) -> &Arc<C> {
&self.cloner
}
Expand Down
74 changes: 73 additions & 1 deletion magicblock-chainlink/src/chainlink/fetch_cloner/tests.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
use std::{collections::HashMap, sync::Arc, time::Duration};

use dlp_api::state::DelegationRecord;
use dlp_api::{
pda::undelegation_request_pda_from_delegated_account,
state::{DelegationRecord, UndelegationRequest},
};
use magicblock_metrics::metrics::{
chainlink_pending_fetch_accounts_value,
chainlink_pending_fetch_waiters_gauge_value,
Expand Down Expand Up @@ -220,6 +223,29 @@ where
}
}

fn undelegation_request_data(
delegated_account: Pubkey,
expires_at_slot: u64,
) -> Vec<u8> {
let request = UndelegationRequest {
delegated_account,
expires_at_slot,
};
let mut data = vec![0; UndelegationRequest::size_with_discriminator()];
request.to_bytes_with_discriminator(&mut data).unwrap();
data
}

fn dlp_account(data: Vec<u8>) -> Account {
Account {
lamports: 1_000_000,
data,
owner: dlp_api::id(),
executable: false,
rent_epoch: 0,
}
}

fn insert_plain_ata_in_bank(
accounts_bank: &Arc<AccountsBankStub>,
ata_pubkey: Pubkey,
Expand Down Expand Up @@ -577,6 +603,52 @@ fn pending_waiters_gauge_value() -> i64 {
)
}

#[tokio::test]
async fn fetch_undelegation_requests_scans_filtered_dlp_accounts() {
let delegated_account = random_pubkey();
let request_pda =
undelegation_request_pda_from_delegated_account(&delegated_account);
let request_data = undelegation_request_data(delegated_account, 20);

let ctx = setup(
vec![
(request_pda, dlp_account(request_data.clone())),
(
random_pubkey(),
dlp_account(vec![
0;
UndelegationRequest::size_with_discriminator()
]),
),
(
random_pubkey(),
Account {
owner: system_program::id(),
data: request_data,
..Default::default()
},
),
],
42,
Keypair::new(),
)
.await;

let requests = ctx
.fetch_cloner
.fetch_undelegation_requests()
.await
.unwrap();

assert_eq!(ctx.rpc_client.program_account_fetches(), 1);
assert_eq!(requests.len(), 1);
let request = &requests[0];
assert_eq!(request.request_pda, request_pda);
assert_eq!(request.delegated_account, delegated_account);
assert_eq!(request.expires_at_slot, 20);
assert_eq!(request.observed_slot, 42);
}

// -----------------
// Single Account Tests
// -----------------
Expand Down
20 changes: 20 additions & 0 deletions magicblock-chainlink/src/chainlink/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,17 @@ impl<T: ChainRpcClient, U: ChainPubsubClient, V: AccountsBank, C: Cloner>
}
}

pub async fn fetch_undelegation_requests(
&self,
) -> ChainlinkResult<Vec<ObservedUndelegationRequest>> {
match self {
Self::Enabled(chainlink) => {
chainlink.fetch_undelegation_requests().await
}
Self::Disabled => Ok(Vec::new()),
}
}

pub fn fetch_count(&self) -> Option<u64> {
match self {
Self::Enabled(chainlink) => chainlink.fetch_count(),
Expand Down Expand Up @@ -627,6 +638,15 @@ impl<T: ChainRpcClient, U: ChainPubsubClient, V: AccountsBank, C: Cloner>
Ok(())
}

pub async fn fetch_undelegation_requests(
&self,
) -> ChainlinkResult<Vec<ObservedUndelegationRequest>> {
let Some(fetch_cloner) = self.fetch_cloner() else {
return Ok(Vec::new());
};
fetch_cloner.fetch_undelegation_requests().await
}
Comment thread
snawaz marked this conversation as resolved.

pub fn fetch_cloner(&self) -> Option<&Arc<FetchCloner<T, U, V, C>>> {
self.fetch_cloner.as_ref()
}
Expand Down
Loading
Loading