Finality is much faster in reality than visible on Subscan UI

[eskimor]

Confirmation

• My issue isn't already found on the issue tracker.
• I have read the Subscan API documentation (if it seemed to be an API bug).

Affected Network(s)

Polkadot, maybe others as well.

Steps to reproduce

1. Visit: https://polkadot.subscan.io/block and see that it takes almost a minute for blocks to get the "green" checkmark.
2. Check e.g. in Polkadot JS that actual finality lag is never more than 5 blocks (30s), usually much less (2-3 blocks).

Expected output

I would expect the UI to reflect the actual finality status much closer.

Actual output

Time to finality is about a minute in the UI.

Additional factoids or references

The issue with the slow UI update is that it lets Polkadot look worse than it actually is.

[freehere107]

Hi. @eskimor The issue you’re experiencing is related to how transaction indexing works in the current Substrate-based blockchain systems.
The reason for this delay is that Substrate chains still experience a significant number of block reorganizations (reorgs). To mitigate the impact of these reorgs and ensure data consistency, Subscan currently indexes transfer data only after a block has been finalized for 2-3 blocks.
This approach helps to avoid situations where transactions might appear and then disappear due to chain reorganizations. While this does cause a delay in transaction visibility, it ensures that the data you see is more stable and reliable.

[freehere107]

In our latest plan, we also investigate how to minimize the delay. If there is more progress, we will update to subscan as soon as possible.

[eskimor]

Hi @freehere107 ! Thanks for the very quick response. I am not quite following how reorgs should affect blocks that are already finalized. Once a block is final, there can not be any reorg invalidating that block, otherwise we would be having a serious issue.

What instabilities have there been?

[freehere107]

@eskimor I reviewed our current solution and found that the finalized block we considered previously had some problems. We are currently making adjustments and the indexing speed of the finalized block should be faster after the upgrade.