feat: L3 support #437

ocdbytes · 2024-12-20T16:30:19Z

Added support for L3s. Starknet client is added in this PR and tests are added for several client functions. for now we are assuming that the l3 gas prices will be zero only.

Pull Request type

Please add the labels corresponding to the type of changes your PR introduces:

Feature
Testing

What is the current behavior?

Currently we didn't support the L3 appchain spec.

What is the new behavior?

Included the starknet client and updated the cli arguments for running madara in appchain mode. For now for all the gas prices and metrics we are still using the l1 modules. We can cover this refactoring in another PR.

Key Changes

Initially we had only one client for settlement (crates/madara/client/eth) but now the folder structure is little bit changed in order to add two clients. crates/madara/client/eth is renamed to crates/madara/client/settlement_client :

├── Cargo.toml
├── README.md
└── src /
    ├── client.rs
    ├── error.rs
    ├── eth /
    │   ├── event.rs
    │   ├── mod.rs
    │   └── starknet_core.json
    ├── gas_price.rs
    ├── lib.rs
    ├── messaging.rs
    ├── starknet /
    │   ├── event.rs
    │   ├── mod.rs
    │   └── utils.rs
    ├── state_update.rs
    ├── sync.rs
    └── utils.rs

Added automock for client testing. automock in client
Also Added seperate tests based on mocking the client for messaging and state update listeners.
In this PR wrappers for event stream is added for ETH which basically takes the alloy eth event stream and puts a stream wrapper implemented in order to make event processing easier. Stream Wrapper.Here the stream wrapper is not generalised and is hard coded to return the Messaging stream in another PR we should make the event stream generalsie din order to be used with any event listener from alloy or any other lib we use in future.
We have implemented separate event streams for eth and starknet, as starknet does not have .watch() function for listening to events we have added a custom stream implementation for starknet (Starknet Stream Implementation)
We have also changes the input type of eth_core_contract_address and eth_gps_statement_verifier in chain config in order to be more accomodating to other types we might use in future and we are currently using.

Important

For now as the PR is itself very long and connot be breaked into components because of inter component and e2e messaging tests. All the variables which are related to l1 are not renamed rn as it will be more time consuming and has no impact on functionality as of now.

Simple Diagram for messaging compoenents

Does this introduce a breaking change?

No

.gitignore

crates/client/settlement_client/Cargo.toml

crates/client/settlement_client/src/client.rs

crates/client/settlement_client/src/eth/mod.rs

crates/client/settlement_client/src/starknet/utils.rs

crates/client/settlement_client/src/starknet/mod.rs

crates/node/src/service/l1.rs

…ts 1

crates/client/settlement_client/src/starknet/mod.rs

crates/client/settlement_client/src/messaging/mod.rs

crates/client/settlement_client/src/starknet/mod.rs

crates/client/settlement_client/src/messaging/sync.rs

crates/client/settlement_client/src/client.rs

crates/client/settlement_client/src/messaging/sync.rs

apoorvsadana · 2025-01-07T20:47:18Z

crates/client/settlement_client/src/starknet/event.rs

+                while !page_indicator {
+                    let events = provider
+                        .get_events(
+                            EventFilter {
+                                from_block: filter.from_block,
+                                to_block: filter.to_block,
+                                address: filter.address,
+                                keys: filter.keys.clone(),
+                            },
+                            continuation_token.clone(),
+                            1000,
+                        )
+                        .await?;
+
+                    event_vec.extend(events.events);
+                    if let Some(token) = events.continuation_token {
+                        continuation_token = Some(token);
+                    } else {
+                        page_indicator = true;
+                    }
+                }


this code exists in get_events in mod.rs as well, can we use that?

That cannot be used unfortunately as there are some constraints in stream implementation.
So when I used the function it kills the stream when no event is returned. So there are no new calls to the stream.
I am checking more on this behaviour on why this is happening.

crates/client/settlement_client/src/starknet/event.rs

Mohiiit

I need some more context in order to understand the event structure a bit better, by structure I mean, how we are fetching for both cases and how is cancelling of messages is working.

crates/client/settlement_client/Cargo.toml

crates/client/settlement_client/src/client.rs

Mohiiit · 2025-01-09T11:16:01Z

crates/client/settlement_client/src/eth/mod.rs

        let block_number = self.provider.get_block_number().await?.as_u64();
        Ok(block_number)
    }

    /// Get the block number of the last occurrence of a given event.
-    pub async fn get_last_event_block_number<T: SolEvent>(&self) -> anyhow::Result<u64> {


why have we removed this? ideally this function should be able to filter out on any event right? if not let's update it's name and comment accordingly?

This is not removed get_last_event_block_number function still exists although it is not used anywhere, but it is still there in the client trait implementation.

my bad for wrong comment, I meant why we removed <T: SolEvent>?

So basically It was removed to make the ClientTrait more generalised.

ahh I see, so can we change the name of the function here? get_last_state_update_event_block_number, given it was generic earlier?

crates/client/settlement_client/src/eth/mod.rs

crates/client/settlement_client/src/messaging/sync.rs

Mohiiit · 2025-01-09T12:09:24Z

crates/client/settlement_client/src/state_update.rs

+
+        // Initialize database service
+        let db = Arc::new(
+            DatabaseService::new(&base_path, backup_dir, false, chain_info.clone(), Default::default())


open_for_testing can be used instead of new

crates/client/settlement_client/src/state_update.rs

crates/node/src/cli/chain_config_overrides.rs

crates/node/src/main.rs

test-artifacts/devnet.yaml

crates/node/src/service/l1.rs

apoorvsadana · 2025-01-14T13:17:06Z

crates/node/src/service/l1.rs

+        devnet: bool,
+        mempool: Arc<Mempool>,
+    ) -> anyhow::Result<Box<dyn Service>> {
+        match config.settlement_layer {


should we add the if !config.l1_sync_disabled && (config.l1_endpoint.is_some() || !devnet) else {} part that is common in eth and starknet here? we can have this match statement inside the if part and it will return the client

as discussed this common logic should be in the create function

crates/client/settlement_client/src/eth/mod.rs

crates/client/settlement_client/src/starknet/mod.rs

Mohiiit · 2025-01-15T04:24:22Z

crates/madara/client/settlement_client/src/client.rs

+    // Get the latest block number
+    async fn get_latest_block_number(&self) -> anyhow::Result<u64>;
+
+    // Get the block number of the last occurrence of a specific event


let's change the comment, given it's not generic now

Suggested change

// Get the block number of the last occurrence of a specific event

// Get the block number of the last occurrence of the state update event

Mohiiit · 2025-01-15T04:52:44Z

crates/madara/client/settlement_client/src/eth/event.rs

+            Ok(event_data) => {
+                assert_eq!(event_data.block_number, 100);
+                assert_eq!(event_data.event_index, Some(0u64));
+                // Add more assertions as needed


is this WIP?

nop removed missed it used claude for generating the tests.

Mohiiit · 2025-01-15T05:01:31Z

crates/madara/client/settlement_client/src/messaging/mod.rs

+async fn process_message(
+    backend: &MadaraBackend,
+    event: &CommonMessagingEventData,
+    _chain_id: &ChainId,


let's remove this? given it's not required

So this param was there in the eth implementation also so I thought maybe it is left for some future functionality. Should I remove this ?

Mohiiit · 2025-01-15T05:22:25Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+                // Block number management in case of pending block number events.
+                match event.block_number {
+                    Some(block_number) => Ok(block_number),
+                    None => Ok(self.get_latest_block_number().await? + 1),


in starknet, we get events as soon as the transactions are executed? and that's why we might get last event from the pending block?

Mohiiit · 2025-01-15T05:36:48Z

crates/madara/client/settlement_client/src/state_update.rs

@@ -1,37 +1,32 @@
 use std::sync::Arc;

+<<<<<<<< HEAD:crates/madara/client/settlement_client/src/state_update.rs


missed these

Mohiiit · 2025-01-15T05:38:50Z

crates/madara/client/sync/src/utils.rs

we have this in primitives/utils/src/hash.rs

removed maybe left during the merge

crates/madara/client/settlement_client/src/starknet/test_contracts/appchain_test.cairo

apoorvsadana · 2025-01-17T11:13:42Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+        while let Some(events) = ctx
+            .run_until_cancelled(self.get_events(
+                BlockId::Number(self.get_latest_block_number().await?),
+                BlockId::Number(self.get_latest_block_number().await?),
+                self.l2_core_contract,
+                vec![get_selector_from_name("LogStateUpdate")?],
+            ))


can move this to a closure for better readability as discussed

crates/madara/client/settlement_client/src/state_update.rs

crates/madara/client/settlement_client/src/messaging/tests.rs

crates/client/settlement_client/src/client.rs

apoorvsadana · 2025-01-17T11:43:38Z

crates/madara/client/settlement_client/src/messaging/mod.rs

ideally we should create an issue for a test case/inform MSL where if a block has 2 events and madara processes one and shuts down, then on restart the next message is picked up of that same block

crates/madara/client/settlement_client/src/eth/mod.rs

apoorvsadana · 2025-01-17T11:58:03Z

crates/madara/client/settlement_client/src/starknet/event.rs

+                }
+                Poll::Pending => Poll::Pending,
+            },
+            None => Poll::Ready(Some(None)), // Handle the case where future is None


i think we should throw an error here to inform users something went wrong

crates/madara/client/settlement_client/src/eth/mod.rs

crates/client/settlement_client/src/messaging/sync.rs

Trantorian1

I glossed over the last few files, so there could be some stuff I didn't catch. Overall this seems interesting but needs as few changes here and there, nothing major.

Trantorian1 · 2025-01-31T14:52:56Z

.github/workflows/coverage.yml

+      # TODO : For now madara binary is stored in aws s3 bucket :
+      #        After the proper release binaries are implemented
+      #        We can directly use that and we can remove this
+      #        temporary AWS implementation
+      - name: Download madara binary for l2 client testing
+        run: |
+          curl -L https://madara-test-binary.s3.us-west-1.amazonaws.com/madara-linux -o ./test-artifacts/madara
+          chmod +x ./test-artifacts/madara
+


Why are we downloading a madara binary for testing instead of running against a build of the code in the pr?

Trantorian1 · 2025-01-31T14:53:19Z

.github/workflows/rust-test.yml

+      - name: Download madara binary for l2 client testing
+        run: |
+          curl -L https://madara-test-binary.s3.us-west-1.amazonaws.com/madara-linux -o ./test-artifacts/madara
+          chmod +x ./test-artifacts/madara


Trantorian1 · 2025-01-31T14:54:07Z

.github/workflows/rust-test.yml

+      - name: Launch Anvil
+        run: anvil --fork-url $ANVIL_FORK_URL --fork-block-number $ANVIL_BLOCK_NUMBER &
+        env:
+          ANVIL_FORK_URL: "https://eth.merkle.io"
+          ANVIL_BLOCK_NUMBER: 20395662
+      - name: Wait for Anvil to be ready
+        run: |
+          while ! nc -z localhost 8545; do
+            sleep 1
+          done


Are we sure this is necessary? If I remember the rust unit tests should start anvil anyway no?

Trantorian1 · 2025-01-31T14:56:33Z

Cargo.lock

+[[package]]
+name = "serial_test"
+version = "3.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1b258109f244e1d6891bf1053a55d63a5cd4f8f4c30cf9a1280989f80e7a1fa9"
+dependencies = [
+ "futures",
+ "log",
+ "once_cell",
+ "parking_lot 0.12.3",
+ "scc",
+ "serial_test_derive",
+]
+
+[[package]]
+name = "serial_test_derive"
+version = "3.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5d69265a08751de7844521fd15003ae0a888e035773ba05695c5c759a6f89eef"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn 2.0.89",
+]


This should not be here. I believe we removed serial-test from madara so the CI can run in parallel. Consider implementing a locking mechanism by hand otherwise, I am not convinced we need an external crate for this kind of simple functionality.

Trantorian1 · 2025-01-31T14:56:45Z

Cargo.toml

@@ -222,6 +220,7 @@ dotenv = "0.15.0"
 httpmock = "0.7.0"
 tempfile = "3.10.1"
 mockall = "0.13.0"
+serial_test = "3.1.1"


Trantorian1 · 2025-01-31T16:30:49Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+    }
+
+    #[rstest]
+    #[traced_test]


Trantorian1 · 2025-01-31T16:31:08Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+    #[tokio::test]
+    // This test is redundant now as the event poller will not return the same
+    // event twice with same nonce that's why added ignore here.
+    #[ignore]


If this is redundant then consider removing it

Trantorian1 · 2025-01-31T16:31:17Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+    }
+
+    #[rstest]
+    #[traced_test]


Trantorian1 · 2025-01-31T16:31:39Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+    use tempfile::TempDir;
+    use url::Url;
+
+    #[rstest]


Why are we using rstest if there is no fixture?

Trantorian1 · 2025-01-31T16:33:07Z

crates/madara/client/settlement_client/src/sync.rs

+    backend: Arc<MadaraBackend>,
+    settlement_client: Arc<Box<dyn ClientTrait<Config = C, StreamType = S>>>,
+    chain_id: ChainId,
+    l1_gas_provider: GasPriceProvider,
+    gas_price_sync_disabled: bool,
+    gas_price_poll_ms: Duration,
+    mempool: Arc<Mempool>,
+    ctx: ServiceContext,
+    l1_block_metrics: Arc<L1BlockMetrics>,


Consider refactoring some of these arguments behind a struct.

jbcaron · 2025-01-31T13:37:02Z

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## Next release

 - fix(primitives): limit legacy class sizes
+- feat : l3 support


you can remove it

jbcaron · 2025-01-31T14:04:20Z

crates/madara/client/settlement_client/Cargo.toml

Please update the dependencies to use workspace instead of fixed versions to ensure consistency, avoid version conflicts, and simplify dependency management

jbcaron · 2025-01-31T14:14:12Z

crates/madara/client/settlement_client/src/client.rs

+#[derive(Debug, Default, PartialEq)]
+pub struct DummyConfig;
+pub type DummyStream = BoxStream<'static, Option<anyhow::Result<CommonMessagingEventData>>>;
+
+#[automock(
+    type Config = DummyConfig;
+    type StreamType = DummyStream;
+)]


Consider moving this behind a "test" feature flag to avoid including them in production builds.

jbcaron · 2025-01-31T14:18:38Z

crates/madara/client/settlement_client/src/client.rs

+    ///     - 0 if the message has not been cancelled
+    ///     - timestamp of the cancellation if it has been cancelled
+    /// - An Error if the call fail
+    async fn get_l1_to_l2_message_cancellations(&self, msg_hash: Vec<u8>) -> anyhow::Result<Felt>;


Suggested change

async fn get_l1_to_l2_message_cancellations(&self, msg_hash: Vec<u8>) -> anyhow::Result<Felt>;

async fn get_l1_to_l2_message_cancellations(&self, msg_hash: &[u8]) -> anyhow::Result<Felt>;

jbcaron · 2025-01-31T14:23:26Z

crates/madara/client/settlement_client/src/client.rs

+    // Create a new instance of the client
+    async fn new(config: Self::Config) -> anyhow::Result<Self>
+    where
+        Self: Sized;


Consider removing new from the trait and implementing it as an associated function in the struct to avoid issues with Self: Sized, async trait limitations, and the inability to use it with dyn ClientTrait

jbcaron · 2025-01-31T17:37:11Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+                let current_processed = self.processed_update_state_block.load(Ordering::Relaxed);
+                if current_processed < block_number {
+                    let formatted_event =
+                        StateUpdate { block_number, global_root: data.data[0], block_hash: data.data[2] };


ditto (direct access)

jbcaron · 2025-01-31T17:37:19Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+            let events_fetched = events?;
+            if let Some(event) = events_fetched.last() {
+                let data = event;
+                let block_number = data.data[1].to_u64().ok_or(anyhow!("Block number conversion failed"))?;


ditto (direct access)

jbcaron · 2025-01-31T17:40:43Z

crates/madara/client/settlement_client/src/starknet/mod.rs

+    fn event_to_felt_array(&self, event: &CommonMessagingEventData) -> Vec<Felt> {
+        let mut felt_vec = vec![event.from, event.to, event.selector, event.nonce];
+        felt_vec.push(Felt::from(event.payload.len()));
+        event.payload.clone().into_iter().for_each(|felt| {
+            felt_vec.push(felt);
+        });
+
+        felt_vec
+    }


Suggested change

fn event_to_felt_array(&self, event: &CommonMessagingEventData) -> Vec<Felt> {

let mut felt_vec = vec![event.from, event.to, event.selector, event.nonce];

felt_vec.push(Felt::from(event.payload.len()));

event.payload.clone().into_iter().for_each(|felt| {

felt_vec.push(felt);

});

felt_vec

}

fn event_to_felt_array(&self, event: &CommonMessagingEventData) -> Vec<Felt> {

std::iter::once(event.from)

.chain(std::iter::once(event.to))

.chain(std::iter::once(event.selector))

.chain(std::iter::once(event.nonce))

.chain(std::iter::once(Felt::from(event.payload.len())))

.chain(event.payload.iter().cloned())

.collect()

}

jbcaron · 2025-01-31T17:43:13Z

crates/madara/client/settlement_client/src/starknet/utils.rs

+    }
+}
+
+pub fn starknet_account() -> anyhow::Result<StarknetAccount> {


Is it only for testing?

jbcaron · 2025-01-31T17:49:53Z

crates/madara/node/src/service/l1.rs

    db_backend: Arc<MadaraBackend>,
-    eth_client: Option<Arc<EthereumClient>>,
+    settlement_client: Option<Arc<Box<dyn ClientTrait<Config = C, StreamType = S>>>>,


Is Box really necessary?

ocdbytes added 7 commits December 18, 2024 23:54

feat : l3 support init

8949443

code : refactor

abc808e

feat : added tests for starknet client

a49e209

changelog update

0d6168e

Merge branch 'main' into l3-support

76bc944

fix : lint

75229a4

feat: added update state worker test for starknet client

661cd33

apoorvsadana marked this pull request as draft December 23, 2024 03:40

apoorvsadana reviewed Dec 23, 2024

View reviewed changes

ocdbytes added 4 commits December 26, 2024 19:13

feat : added messaging implementation for l2 client & resolved commen…

62fb337

…ts 1

feat : updated l2 messaging implementation

d4aa988

feat : updated l2 messaging implementation

a4314be

feat : updated messaging logic

9dea223

apoorvsadana reviewed Dec 28, 2024

View reviewed changes

Trantorian1 assigned ocdbytes Dec 31, 2024

Trantorian1 added the research Research and exploration required before implementation label Dec 31, 2024

ocdbytes added 3 commits January 6, 2025 17:25

clients refactoring done

fafd2c3

refactor : tests

aa7834e

fix : tests

6c1ff95

apoorvsadana reviewed Jan 7, 2025

View reviewed changes

crates/client/settlement_client/src/messaging/mod.rs Outdated Show resolved Hide resolved

crates/client/settlement_client/src/starknet/mod.rs Outdated Show resolved Hide resolved

apoorvsadana reviewed Jan 7, 2025

View reviewed changes

Mohiiit reviewed Jan 9, 2025

View reviewed changes

ocdbytes added 5 commits January 14, 2025 15:38

feat : updated streams and added tests for streams

26c9319

fix : changed event params in messaging contract

2a19391

refactor : removed external devnet.yaml file

8abf999

feat : removed expects

749df1c

refactor crates structure

b63cf61

apoorvsadana reviewed Jan 14, 2025

View reviewed changes

crates/client/settlement_client/src/starknet/mod.rs Outdated Show resolved Hide resolved

crates/client/settlement_client/src/starknet/mod.rs Outdated Show resolved Hide resolved

merge main

c837570

Mohiiit reviewed Jan 15, 2025

View reviewed changes

ocdbytes added 2 commits January 16, 2025 01:27

resolved comments

1d4d3a4

refactor

fffd9dd

apoorvsadana reviewed Jan 17, 2025

View reviewed changes

ocdbytes added 4 commits January 21, 2025 17:09

feat : added automock tests and resolved comments

71c5524

merge : main

62d1df3

refactor

79ef5ab

refactor code and comments resolved

8657da3

Trantorian1 added sequencer Related to the sequencing logic and implementation node Related to the full node implementation labels Jan 23, 2025

Mohiiit marked this pull request as ready for review January 31, 2025 13:45

Trantorian1 requested changes Jan 31, 2025

View reviewed changes

jbcaron reviewed Jan 31, 2025

View reviewed changes

Merge branch 'main' into l3-support

7afd1a3

	// Get the block number of the last occurrence of a specific event
	// Get the block number of the last occurrence of the state update event

		@@ -1,37 +1,32 @@
		use std::sync::Arc;

		<<<<<<<< HEAD:crates/madara/client/settlement_client/src/state_update.rs

	async fn get_l1_to_l2_message_cancellations(&self, msg_hash: Vec<u8>) -> anyhow::Result<Felt>;
	async fn get_l1_to_l2_message_cancellations(&self, msg_hash: &[u8]) -> anyhow::Result<Felt>;

feat: L3 support #437

Are you sure you want to change the base?

feat: L3 support #437

Conversation

ocdbytes commented Dec 20, 2024 • edited Loading

Pull Request type

What is the current behavior?

What is the new behavior?

Key Changes

Simple Diagram for messaging compoenents

Does this introduce a breaking change?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mohiiit left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mohiiit Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Trantorian1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ocdbytes commented Dec 20, 2024 •

edited

Loading

Mohiiit Jan 10, 2025 •

edited

Loading