ci: multichain-testing with `go-relayer` #10182

0xpatrickdev · 2024-10-02T17:03:23Z

Description

Ensures compatibility with go-relayer by adding workflow dispatch jobs to multichain-e2e for Orchestration API (config.go-relayer.yaml) and FastUSDC (config.fusdc.go-relayer.yaml). Our existing tests using hermes. Scope is mainly for the Orchestration API, but x/vibc is also covered to an extent.

Towards this goal:

enable starship faucet for agoric in all configs (required for go-relayer)
update channel-close tests to use go-relayer when RELAYER_TYPE env var is present

Security Considerations

n/a, test code

Scaling Considerations

Not currently part of integration tests that run on merges to master.

Documentation Considerations

Added a section in the README.md with instructions for running the tests with go-relayer.

Testing Considerations

Runs existing test suite against go-relayer, extending our test coverage across to the two main cosmos-sdk relayer clients. This should help identify any relayer-specific issues or discrepancies. Please note: these tests will not run on every PR and must be dispatched manually.

Upgrade Considerations

n/a

cloudflare-workers-and-pages · 2024-10-02T19:11:01Z

Deploying agoric-sdk with Cloudflare Pages

Latest commit:	`3f0038f`
Status:	✅ Deploy successful!
Preview URL:	https://929eb5d1.agoric-sdk.pages.dev
Branch Preview URL:	https://10179-go-relayer.agoric-sdk.pages.dev

View logs

turadg

Good stuff!

.github/workflows/multichain-e2e-template.yml

.github/workflows/multichain-e2e.yml

multichain-testing/package.json

0xpatrickdev · 2024-10-04T17:18:34Z

Hesitant to merge until we see this go through CI multiple times successfully.

I was under the impression each run is fully containerized, but am seeing weird behavior which leads me to believe these are stepping on each other's toes. Both jobs read from and write to the agoric-sdk/multichain-testing filesystem and access the same ports over the network.

Any folks with more gh-action-fu have more insights?

Edit: Bad hunch - everything is isolated in GitHub action runs, even without service containers. I was mainly thrown off by an opaque error in a starships client dependency. It also seems like the "Setup Starship Infrastructure" step can complete successfully without establishing connections and channels. For now, we can check in go-relayer tests as dispatch (on-demand) jobs to avoid flakes.

turadg

I'll approve when the work-arounds for upstream issues are gone, or there's a conspicuous plan for addressing them.

.github/workflows/multichain-e2e-template.yml

multichain-testing/README.md

- to match ChainName conventions used in `@cosmos/chain-registry`

- adds dispatch jobs to test orchestration-api and fusdc test suites against go-relayer

0xpatrickdev

It looks like GH has an outage, but this is R4R. I amended this comment with some details. I think the current approach renders a majority of the current feedback no longer relevant - lmk if that's not the case or you need me to jog your memory on anything @turadg.

turadg · 2025-01-14T00:52:52Z

.github/workflows/multichain-e2e.yml

+    name: Multichain E2E (Orchestration API - Hermes)
+    if: |
+      github.event_name == 'workflow_call' || 
+      (github.event_name == 'workflow_dispatch' && inputs.test_type == 'orchestration-api-hermes')


please update the job names to match the test_type selector.

turadg · 2025-01-14T00:54:37Z

.github/workflows/multichain-e2e.yml

-      test_command: test:fast-usdc
+      test_command: yarn test:fast-usdc
+
+  # Below jobs run on dispatch only


this kind of comment might be missed when someone changes whether it's true.

please comment on each job's if with when it runs

turadg · 2025-01-14T00:55:36Z

multichain-testing/test/fast-usdc/fast-usdc.test.ts

@@ -23,6 +23,8 @@ import type { CurrentWalletRecord } from '@agoric/smart-wallet/src/smartWallet.j
 import type { QueryBalanceResponseSDKType } from '@agoric/cosmic-proto/cosmos/bank/v1beta1/query.js';
 import type { USDCProposalShapes } from '@agoric/fast-usdc/src/pool-share-math.js';

+const RELAYER_TYPE = process.env.RELAYER_TYPE;


to make clear the name is unchanged,

Suggested change

const RELAYER_TYPE = process.env.RELAYER_TYPE;

const { RELAYER_TYPE } = process.env;

turadg · 2025-01-14T00:57:13Z

multichain-testing/test/ica-channel-close.test.ts

+      t.log(e); // hermes relayer throws, but go-relayer does not
+    }
+    t.log('Sleeping 10 seconds for potential channel closure to propagate');
+    await sleep(10000);


how was 10s determined? please make it more legible:

Suggested change

await sleep(10000);

await sleep(10 * 1000);

turadg · 2025-01-14T00:57:51Z

multichain-testing/test/config.ts

@@ -41,3 +41,8 @@ export const MAKE_ACCOUNT_AND_QUERY_BALANCE_TIMEOUT: RetryOptions = {
  retryIntervalMs: 5000,
  maxRetries: 24,
 };
+
+export const TWO_MINUTES: RetryOptions = {


please give this a semantic name. It's baking in more assumptions than two minutes

turadg · 2025-01-14T00:58:30Z

multichain-testing/test/support.ts

@@ -70,7 +70,11 @@ const makeKeyring = async (

 export const commonSetup = async (
  t: ExecutionContext,
-  { config = '../config.yaml' } = {},
+  {
+    env = process.env,


why propagate the whole env? if someone wants to override relayerType they don't have to mess with env at all

turadg · 2025-01-14T01:00:14Z

multichain-testing/test/tools/wallet.test.ts

-  t.deepEqual(balance, { denom: bondDenom, amount: '10000000000' });
+const walletScenario = test.macro({
+  title: (_, chainName: string) =>
+    `create a wallet and get tokens on ${chainName}`,


0xpatrickdev added the force:integration Force integration tests to run on PR label Oct 2, 2024

0xpatrickdev force-pushed the 10179-go-relayer branch 9 times, most recently from cd1daba to 2d38529 Compare October 3, 2024 18:45

0xpatrickdev marked this pull request as ready for review October 3, 2024 19:38

0xpatrickdev requested a review from a team as a code owner October 3, 2024 19:38

0xpatrickdev requested a review from AgoricTriage October 3, 2024 19:38

0xpatrickdev added force:integration Force integration tests to run on PR and removed force:integration Force integration tests to run on PR labels Oct 3, 2024

0xpatrickdev requested review from turadg and LuqiPan October 3, 2024 22:34

turadg reviewed Oct 3, 2024

View reviewed changes

0xpatrickdev force-pushed the 10179-go-relayer branch from 2d38529 to a0ae81b Compare October 4, 2024 15:32

turadg reviewed Oct 15, 2024

View reviewed changes

.github/workflows/multichain-e2e-template.yml Outdated Show resolved Hide resolved

.github/workflows/multichain-e2e-template.yml Outdated Show resolved Hide resolved

multichain-testing/README.md Outdated Show resolved Hide resolved

0xpatrickdev force-pushed the 10179-go-relayer branch 3 times, most recently from f0827ce to 11015a0 Compare October 16, 2024 19:09

0xpatrickdev marked this pull request as draft October 16, 2024 19:16

0xpatrickdev force-pushed the 10179-go-relayer branch 4 times, most recently from fa4b501 to 4414f4c Compare October 17, 2024 00:26

0xpatrickdev force-pushed the 10179-go-relayer branch 12 times, most recently from 82709be to a0c73e0 Compare October 18, 2024 14:09

0xpatrickdev added 3 commits January 13, 2025 10:42

feat: enable faucet on agoric

9b6354f

refactor: generic relayer tools

db066e0

feat: support go-relayer rly commands

45310b0

0xpatrickdev force-pushed the 10179-go-relayer branch from a0c73e0 to d87eda1 Compare January 13, 2025 19:45

chore: rename gaia* to cosmoshub*

ee7eda8

- to match ChainName conventions used in `@cosmos/chain-registry`

0xpatrickdev force-pushed the 10179-go-relayer branch from d87eda1 to e7559db Compare January 13, 2025 22:43

ci: test against go-relayer

3f0038f

- adds dispatch jobs to test orchestration-api and fusdc test suites against go-relayer

0xpatrickdev force-pushed the 10179-go-relayer branch from e7559db to 3f0038f Compare January 13, 2025 23:17

0xpatrickdev marked this pull request as ready for review January 14, 2025 00:20

0xpatrickdev commented Jan 14, 2025

View reviewed changes

0xpatrickdev requested a review from turadg January 14, 2025 00:21

turadg reviewed Jan 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: multichain-testing with `go-relayer` #10182

ci: multichain-testing with `go-relayer` #10182

0xpatrickdev commented Oct 2, 2024 •

edited

Loading

cloudflare-workers-and-pages bot commented Oct 2, 2024 •

edited

Loading

turadg left a comment

0xpatrickdev commented Oct 4, 2024 •

edited

Loading

turadg left a comment

0xpatrickdev left a comment

turadg Jan 14, 2025

turadg Jan 14, 2025

turadg Jan 14, 2025

turadg Jan 14, 2025

turadg Jan 14, 2025

turadg Jan 14, 2025

turadg Jan 14, 2025

	const RELAYER_TYPE = process.env.RELAYER_TYPE;
	const { RELAYER_TYPE } = process.env;

ci: multichain-testing with go-relayer #10182

Are you sure you want to change the base?

ci: multichain-testing with go-relayer #10182

Conversation

0xpatrickdev commented Oct 2, 2024 • edited Loading

Description

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Upgrade Considerations

cloudflare-workers-and-pages bot commented Oct 2, 2024 • edited Loading

Deploying agoric-sdk with Cloudflare Pages

turadg left a comment

Choose a reason for hiding this comment

0xpatrickdev commented Oct 4, 2024 • edited Loading

turadg left a comment

Choose a reason for hiding this comment

0xpatrickdev left a comment

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

turadg Jan 14, 2025

Choose a reason for hiding this comment

ci: multichain-testing with `go-relayer` #10182

ci: multichain-testing with `go-relayer` #10182

0xpatrickdev commented Oct 2, 2024 •

edited

Loading

cloudflare-workers-and-pages bot commented Oct 2, 2024 •

edited

Loading

0xpatrickdev commented Oct 4, 2024 •

edited

Loading