hotfix(operator): manage subscription to events without panic

[JuArce]

Manage Operator's event subscription

Description

Previously, the Operator panicked if one connection (either main or fallback) failed.

The previous solution to this (#1692) was to iterate infinitely over both connections, this had the "silent failure" problem, as an operator could fail and retry forever without notice to its owner.

Now if one connection fails, the Operator will use the other one, while continiously trying to reconnect to the failed one; and if both connections fail, and the retryables are consumed, the Operator will exit, logging the errors of each connection RPC, so the owner/manager of the Operator server can fix it accordingly.

This PR also added the usage of fallback when operator calls DisableVerifiers

To test

Follow guide in #1692 to set up a proxy to your anvil, but I recommend setting up 2 proxies so you can individually control each Operator RPC connection, with the following docker-compose.yml:

version: '3'
services:
  nginx:
    image: nginx:alpine
    container_name: nginx-anvil-proxy
    volumes:
      - ./nginx:/etc/nginx
    ports:
      - "8082:8082"

  nginx2:
    image: nginx:alpine
    container_name: nginx-anvil-proxy-2
    volumes:
      - ./nginx:/etc/nginx
    ports:
      - "8083:8082"

• Verify the Operator behaves as described, and while submitting proofs take down 1 connection, take down the other connection, take down both connections.

Type of change

Please delete options that are not relevant.

• New feature
• Bug fix
• Optimization
• Refactor

Checklist

• “Hotfix” to testnet, everything else to staging
• Linked to Github Issue
• This change depends on code or research by an external entity
  • Acknowledgements were updated to give credit
• Unit tests added
• This change requires new documentation.
  • Documentation has been added/updated.
• This change is an Optimization
  • Benchmarks added/run
• Has a known issue
  • Link to the open issue addressing it
• If your PR changes the Operator compatibility (Ex: Upgrade prover versions)
  • This PR adds compatibility for operator for both versions and do not change batcher/docs/examples
  • This PR updates batcher and docs/examples to the newer version. This requires the operator are already updated to be compatible

[uri-99]

biased self-approve

[JuArce]

The same comments apply for V3