Skip to content

scrimlet-reconcilers: Add dpd port reconciliation#10449

Open
jgallagher wants to merge 5 commits into
john/feature-sled-agent-scrimlet-reconcilersfrom
john/scrimlet-reconcilers-dpd-ports
Open

scrimlet-reconcilers: Add dpd port reconciliation#10449
jgallagher wants to merge 5 commits into
john/feature-sled-agent-scrimlet-reconcilersfrom
john/scrimlet-reconcilers-dpd-ports

Conversation

@jgallagher
Copy link
Copy Markdown
Contributor

This PR looks big but over 60% of it is tests; the production code that needs careful review is the new sled-agent/scrimlet-reconcilers/src/dpd_reconciler/port_reconciler.rs module. It contains a port of what sync_switch_reconciler does to configure QSFP ports, with several changes:

  1. We read the desired settings from RackNetworkConfig instead of the db
  2. We compute diffs via daft
  3. Includes a workaround for a dpd issue where ports that change speed or FEC have to be removed before reapplying. @internet-diglett is working on a dpd change that would make this unnecessary, but for now it's easy enough to work around on our side.

@jgallagher jgallagher added the networking Related to the networking. label May 15, 2026
@internet-diglett
Copy link
Copy Markdown
Contributor

Created oxidecomputer/dendrite#272 for testing

Copy link
Copy Markdown
Contributor

@internet-diglett internet-diglett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.addresses
.iter()
.filter_map(|a| {
// TODO we're discarding any vlan_id - is that okay?
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the vlan configuration is handled by uplinkd

Comment on lines +488 to +491
// TODO We're discarding the `ip_net.prefix()` here
// and only using the IP address; at some point we
// probably need to give the full CIDR to dendrite?
Some(ip_net.addr())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all of the mechanisms that rely on the net mask / full CIDR are also handled via uplinkd.

Comment on lines +422 to +429
// Workaround for dpd limitation: the speed and fec of a link
// require us to remove it first. Any other change can be
// applied in place. TODO: link to dendrite issue / PR
if leaf.before().speed != leaf.after().speed
|| leaf.before().fec != leaf.after().fec
{
to_clear.insert(port_id.clone());
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this workaround now

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Will remove this once #10497 merges and I can pull it into this branch.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, although it looks to do the same update that I've already done here :D

Comment on lines +224 to +227
let are_port_settings_clear = {
let DpdPortSettings { links } = &settings;
links.is_empty()
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just something I was curious about, is there a reason we are doing this instead of something like:

Suggested change
let are_port_settings_clear = {
let DpdPortSettings { links } = &settings;
links.is_empty()
};
let are_port_settings_clear = settings.links.is_empty();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've started being pretty aggressive with adding full destructurings like this to be defensive against fields being added in the future. Suppose a new version of DpdPortSettings adds another field (interfaces or whatever). If we had this:

let are_port_settings_clear = settings.links.is_empty();

that will still compile, and we have to rely on memory / tests / whatever to know whether it's okay that we're ignoring a new field. But this:

let DpdPortSettings { links } = &settings;

will fail to compile because it doesn't include the new field, which guarantees a person will look at it and decide what to do about the new field.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. That makes a lot of sense and that's a pattern I should also keep in mind for use in the future!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

networking Related to the networking.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants