Create Indexer for Sputnik DAO

[ori-near]

Description

We need to build an indexer similar to the one used for DevHub proposals, designed to index proposals from Sputnik DAOs. This will enable enhanced functionality for filtering and searching within the Treasury Dashboard.

Scope

Decide Indexing Strategy

• Should we index all .sputnik-dao.near accounts, or only the accounts associated with our treasury?
• If we choose to index treasury-related accounts only, we need to implement an API to dynamically start indexing new accounts created through the self-create flow.

Data to be Indexed

• Proposal data as retrieved from the DAO contract.
• Execution transaction details, including the link and timestamp.

Required Endpoints

Sorting

• By creation time.
• By expiry time.

Filtering

• By creator.
• By proposal type.
• By number of votes.
• By proposal status.

Searching

• By proposal description.

[ori-near]

Notes from our standup call: Ideally create one indexer for all DAOs, leverage sharding ... instead of deploying an indexer per DAO instance.

[Tguntenaar]

Game Plan for Sputnik DAO Indexer

---

Indexing Strategy

We will use a centralized indexer to handle all DAOs rather than deploying a separate instance for each DAO. The endpoints will support filtering and querying by DAO account.

Since our Fly.io machines remain offline unless needed, we’ll implement a "just-in-time" indexing strategy. This means we’ll only index data when the frontend requests it for a specific DAO account. This approach will index all relevant data, but only when an updated table for a DAO account is requested.

To fetch uncached DAO proposals, we’ll index each DAO subaccount separately using the Nearblocks API. This ensures we retrieve all transactions not already stored in our cache.

Additionally, we could index the DAO factory to detect and dynamically start indexing new accounts created through the self-create flow.

---

Data to Be Indexed

Most of the proposal data is already indexed, including searching by description, which is functional. However, we need to index the following new data:

• Execution transaction details, including:
  • Transaction link (e.g., to Explorer).
  • Execution timestamp.

---

New API Endpoint

The new endpoint will allow querying proposals with sorting and filtering options:

• /dao/proposals/{DAO_ID}/?sort={sorting}&filters={filters}
  • Sorting Options:
    • By creation time.
    • By expiry time.
  • Filters:
    • By creator.
    • By proposal type.
    • By number of votes.
    • By proposal status.

This endpoint will integrate seamlessly with the frontend, providing the necessary data for the Treasury Dashboard.

[Tguntenaar]

Note that sorting by creation time and expiry time will be the same since the expiry time is defined by the creation time + policy duration time.

[Tguntenaar]

Update

I'm currently working on indexing add_proposal and act_proposal functions from the DAO contract.

At first I wanted to use the transaction arguments to indext the state. However these functions change the proposal's inside the contract with a ton of logic.

So instead of recreating that we just get the proposal from contract on the transaction blockheight. I'm now (Friday Dec 20) working on deserializing the string U128. So we can use the RPC response to cache the proposal.

@frol do you have any idea why it won't deserialize U128?

JsonRpcError(ServerError(HandlerError(ContractExecutionError { vm_error: "wasm execution failed with error: HostError(GuestPanic { panic_msg: \"panicked at 'ERR_NO_PROPOSAL', sputnikdao2/src/views.rs:102:48\" })", block_height: 114574681, block_hash: CEx1UdXoBRxsBFoHys9m7zaMwQsRP9e7qiVvHZWW1erf })))
DeserializeError(Error("u128 is not supported", line: 1, column: 2807))

[Tguntenaar]

Update

The deserialisation error from above has been fixed. I've deployed the first version to fly.io here: https://testing-indexer.fly.dev/

Theses fields are indexed properly:

• id
• description
• kind
• proposer
• status
• hash

I'm currently working on indexing the following fields:

• tx_timestamp
• vote_counts
• votes
• submission_time
• proposal_action