Electra upgrade by claravanstaden · Pull Request #1283 · Snowfork/snowbridge

claravanstaden · 2024-09-06T14:00:39Z

Changes

changes beacon config to include fork versions instead of just a deneb fork value.
adds Electra modified and new beacon structures

Testing

Beacon client updates for Electra updates work
Live switch-over from Deneb to Electra works
Smoke tests work

Resolves SNO-1140.

Companion: Snowfork/polkadot-sdk#178

Geth version: lightclients/geth
Lodestar version: 1.24.0

claravanstaden · 2024-09-10T09:55:51Z

relayer/relays/beacon/header/syncer/api/api_deneb.go

 	}, nil
 }

-func DenebJsonExecutionPayloadHeaderToScale(e *beaconjson.FullExecutionPayloadHeaderJson) (scale.ExecutionPayloadHeaderDeneb, error) {


Unused method.

# Conflicts: # relayer/contracts/gateway.go # relayer/relays/beacon/header/syncer/syncer_test.go # relayer/relays/beacon/store/datastore_test.go

yrong · 2025-01-20T14:13:10Z

web/packages/test/scripts/electra-deploy-ethereum.sh

+        --rest.namespace="*" \
+        --jwt-secret $config_dir/jwtsecret \
+        --chain.archiveStateEpochFrequency 1 \
+        >"$output_dir/lodestar.log" 2>&1 &


should it be output_electra_dir?

No, it outputs to /tmp/snowbridge, which I think should be fine. :)

yrong · 2025-01-20T14:17:17Z

web/packages/test/scripts/set-env.sh

+if [ "$is_electra" == "true" ]; then
+    HOST=$(ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1')


This may not work for machines with multi-network interfaces, my PC, e.g.

➜ test git:(electra) ✗ ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1' 192.168.0.127 26.26.26.1 192.168.64.1

You may be able to force bind docker to 127.0.0.1 when forwarding ports from host to container. But also maybe we should build from source to avoid docker.

Build from source: 23eb81d

yrong · 2025-01-20T14:20:11Z

web/packages/test/scripts/electra-deploy-ethereum.sh

+    docker run --rm \
+      -v "${output_electra_dir}:/mnt" \
+      docker.io/ethpandaops/geth:lightclient-prague-devnet-4 \


I would suggest not introducing docker for the local setup, can maybe build from source if necessary like lodestar.

Build geth from source: 23eb81d

yrong · 2025-01-20T14:31:24Z

relayer/relays/beacon/header/syncer/syncer.go

+	retries := 5
+	bootstrap, err := s.getCheckpoint()
 	if err != nil {
-		return scale.BeaconCheckpoint{}, fmt.Errorf("get finalized checkpoint: %w", err)
+		for retries > 0 {
+			retries = retries - 1
+			bootstrap, err = s.getCheckpoint()
+			if err != nil {
+				log.WithError(err).Info("retry bootstrap, sleeping")
+				time.Sleep(10 * time.Second)
+				continue
+			}
+			break


Just curious is the retry necessary here?

I got intermittent failures from Lodestar (on the Mekong network) where a bootstrap payload was not available for the provided finalized checkpoint, so I added a retry, instead of having to retry the whole e2e script. :)

yrong · 2025-01-20T14:34:03Z

relayer/relays/beacon/header/syncer/syncer.go

+	} else if forkVersion == protocol.Deneb {
 		beaconState = &state.BeaconStateDenebMainnet{}
 	} else {
 		beaconState = &state.BeaconStateCapellaMainnet{}


The Capella branch can be deleted?

relayer/relays/beacon/header/syncer/syncer.go

yrong · 2025-01-20T14:44:05Z

relayer/relays/beacon/header/syncer/syncer.go

 	} else {
-		executionPayloadScale, err := api.CapellaExecutionPayloadToScale(sszBlock.ExecutionPayloadCapella())
+		executionPayloadScale, err := api.CapellaExecutionPayloadToScale(beaconBlock.ExecutionPayloadCapella())


Can just remove the Capella branch?

Good idea, ac52035.

yrong

Cool!

# Conflicts: # relayer/templates/inbound-fixtures.mustache

# Description Adds support for the upcoming Ethereum Electra upgrade, while maintaining backwards compatibility for the current Deneb fork. ## Integration Relayers should be updated to send updated Electra consensus data structures. ## Review Notes The [Ethereum Electra hard-fork consensus](https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/light-client/sync-protocol.md) changes affecting the Ethereum light client are mainly isolated to the [Generalized Indexes](https://github.com/protolambda/eth2.0-ssz/blob/master/specs/navigation/generalized_indices.md) of data structures changing. Before Electra, these values were hardcoded in config. For Electra, these values change and needed to the updated. Methods were added to return the correct g-index for the current fork version. Data structures used by the Ethereum client did not change in this hard-fork. The BeaconState container has been updated with additional changes, but because the on-chain code does not reference the BeaconState directly (only indirectly through merkle proofs), it is not a concern. Off-chain relayers use the BeaconState to generate proofs, and so the relayer code has been updated accordingly. ### Companion PR for off-chain relayers Snowfork/snowbridge#1283 --------- Co-authored-by: claravanstaden <Cats 4 life!> Co-authored-by: Ron <yrong1997@gmail.com> Co-authored-by: Vincent Geddes <vincent@snowfork.com> Co-authored-by: Alistair Singh <alistair.singh7@gmail.com> Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>

# Description Adds support for the upcoming Ethereum Electra upgrade, while maintaining backwards compatibility for the current Deneb fork. ## Integration Relayers should be updated to send updated Electra consensus data structures. ## Review Notes The [Ethereum Electra hard-fork consensus](https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/light-client/sync-protocol.md) changes affecting the Ethereum light client are mainly isolated to the [Generalized Indexes](https://github.com/protolambda/eth2.0-ssz/blob/master/specs/navigation/generalized_indices.md) of data structures changing. Before Electra, these values were hardcoded in config. For Electra, these values change and needed to the updated. Methods were added to return the correct g-index for the current fork version. Data structures used by the Ethereum client did not change in this hard-fork. The BeaconState container has been updated with additional changes, but because the on-chain code does not reference the BeaconState directly (only indirectly through merkle proofs), it is not a concern. Off-chain relayers use the BeaconState to generate proofs, and so the relayer code has been updated accordingly. ### Companion PR for off-chain relayers Snowfork/snowbridge#1283 --------- Co-authored-by: claravanstaden <Cats 4 life!> Co-authored-by: Ron <yrong1997@gmail.com> Co-authored-by: Vincent Geddes <vincent@snowfork.com> Co-authored-by: Alistair Singh <alistair.singh7@gmail.com> Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com> (cherry picked from commit dd7562a)

Adds support for the upcoming Ethereum Electra upgrade, while maintaining backwards compatibility for the current Deneb fork. Relayers should be updated to send updated Electra consensus data structures. The [Ethereum Electra hard-fork consensus](https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/light-client/sync-protocol.md) changes affecting the Ethereum light client are mainly isolated to the [Generalized Indexes](https://github.com/protolambda/eth2.0-ssz/blob/master/specs/navigation/generalized_indices.md) of data structures changing. Before Electra, these values were hardcoded in config. For Electra, these values change and needed to the updated. Methods were added to return the correct g-index for the current fork version. Data structures used by the Ethereum client did not change in this hard-fork. The BeaconState container has been updated with additional changes, but because the on-chain code does not reference the BeaconState directly (only indirectly through merkle proofs), it is not a concern. Off-chain relayers use the BeaconState to generate proofs, and so the relayer code has been updated accordingly. Snowfork/snowbridge#1283 --------- Co-authored-by: claravanstaden <Cats 4 life!> Co-authored-by: Ron <yrong1997@gmail.com> Co-authored-by: Vincent Geddes <vincent@snowfork.com> Co-authored-by: Alistair Singh <alistair.singh7@gmail.com> Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>

* electra upgrade * relayer progress * more electra work * fix config * fixes * fix tests * update go version * containers * electra changes * try testing lodestar + electra * revert local script * mekong network setup * electra updates * fixes * fixes * fix tests * electra local net * fix config * cleanup * fixes * progress * fix execution header proof * revert testing changes * adds tests * cleanup * cleanup * uses geth binary instead of docker * removes all traces of capella * devnet6 fixes * update bindings

Adds support for the upcoming Ethereum Electra upgrade, while maintaining backwards compatibility for the current Deneb fork. Relayers should be updated to send updated Electra consensus data structures. The [Ethereum Electra hard-fork consensus](https://github.com/ethereum/consensus-specs/blob/dev/specs/electra/light-client/sync-protocol.md) changes affecting the Ethereum light client are mainly isolated to the [Generalized Indexes](https://github.com/protolambda/eth2.0-ssz/blob/master/specs/navigation/generalized_indices.md) of data structures changing. Before Electra, these values were hardcoded in config. For Electra, these values change and needed to the updated. Methods were added to return the correct g-index for the current fork version. Data structures used by the Ethereum client did not change in this hard-fork. The BeaconState container has been updated with additional changes, but because the on-chain code does not reference the BeaconState directly (only indirectly through merkle proofs), it is not a concern. Off-chain relayers use the BeaconState to generate proofs, and so the relayer code has been updated accordingly. Snowfork/snowbridge#1283 --------- Co-authored-by: claravanstaden <Cats 4 life!> Co-authored-by: Ron <yrong1997@gmail.com> Co-authored-by: Vincent Geddes <vincent@snowfork.com> Co-authored-by: Alistair Singh <alistair.singh7@gmail.com> Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>

* electra upgrade * relayer progress * more electra work * fix config * fixes * fix tests * update go version * containers * electra changes * try testing lodestar + electra * revert local script * mekong network setup * electra updates * fixes * fixes * fix tests * electra local net * fix config * cleanup * fixes * progress * fix execution header proof * revert testing changes * adds tests * cleanup * cleanup * uses geth binary instead of docker * removes all traces of capella * devnet6 fixes * update bindings

claravanstaden added 4 commits September 6, 2024 16:00

electra upgrade

a1e12b9

relayer progress

934973e

more electra work

d64e224

fix config

aaf566b

claravanstaden commented Sep 10, 2024

View reviewed changes

fixes

7cfd412

claravanstaden mentioned this pull request Sep 10, 2024

Electra upgrade Snowfork/polkadot-sdk#178

Draft

claravanstaden added 8 commits September 10, 2024 13:58

fix tests

5d345b7

update go version

51aa8de

containers

3a25d81

Merge branch 'main' into electra

d5fd515

Merge branch 'main' into electra

5de9c0e

electra changes

b2379ac

try testing lodestar + electra

c1b3e9c

Merge branch 'main' into electra

3b5d3ca

# Conflicts: # relayer/contracts/gateway.go # relayer/relays/beacon/header/syncer/syncer_test.go # relayer/relays/beacon/store/datastore_test.go

github-actions bot added Component: Relayer Component: Web labels Dec 19, 2024

claravanstaden added 6 commits December 19, 2024 10:26

revert local script

b940cc2

mekong network setup

29e83b5

electra updates

fb78a95

fixes

c71fb87

fixes

647666c

fix tests

1dc8c16

claravanstaden mentioned this pull request Jan 8, 2025

Snowbridge - Ethereum Electra Upgrade Support paritytech/polkadot-sdk#7075

Merged

claravanstaden added 6 commits January 8, 2025 13:37

electra local net

0c382c3

fix config

fb4ed25

cleanup

3fddb4c

fixes

9fd0f18

progress

8fc7910

fix execution header proof

6104059

claravanstaden added 4 commits January 15, 2025 14:22

revert testing changes

d6ce7cf

adds tests

b121a6c

cleanup

53e71c2

cleanup

8271bb8

claravanstaden marked this pull request as ready for review January 16, 2025 10:10

yrong reviewed Jan 20, 2025

View reviewed changes

relayer/relays/beacon/header/syncer/syncer.go Show resolved Hide resolved

yrong reviewed Jan 20, 2025

View reviewed changes

claravanstaden added 2 commits January 22, 2025 10:17

uses geth binary instead of docker

23eb81d

removes all traces of capella

ac52035

claravanstaden requested a review from yrong January 22, 2025 08:53

yrong approved these changes Jan 22, 2025

View reviewed changes

claravanstaden added 2 commits February 7, 2025 15:37

devnet6 fixes

3fda85a

Merge branch 'main' into electra

9d2bfe0

# Conflicts: # relayer/templates/inbound-fixtures.mustache

update bindings

590fe32

claravanstaden merged commit 247f735 into main Feb 24, 2025

claravanstaden deleted the electra branch February 24, 2025 08:12

		if [ "$is_electra" == "true" ]; then
		HOST=$(ifconfig \| grep -Eo 'inet (addr:)?([0-9]\.){3}[0-9]' \| grep -Eo '([0-9]\.){3}[0-9]' \| grep -v '127.0.0.1')

Conversation

claravanstaden commented Sep 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

claravanstaden Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yrong left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

claravanstaden commented Sep 6, 2024 •

edited

Loading

claravanstaden Jan 22, 2025 •

edited

Loading