Run queries in dedicated threads

[abizjak]

Purpose

Use a dedicated threadpool for running queries.

Since the queries call into Haskell it typically takes much more than 100us between await points. As a result there are problems with availability where queries pause for some time before resuming. The main reason for this seems to be that all the tokio workers are stuck in the "haskell code" so the scheduler cannot schedule other tasks, such as accepting connections, responding to requests, etc.

The guideline from tokio is that there should not be more than 10-100us in between await points.

To give a sense of scale of how much time is too much, a good rule of thumb is no more than 10 to 100 microseconds between each .await. That said, this depends on the kind of application you are writing.

(from one of the tokio developers https://ryhl.io/blog/async-what-is-blocking/ )

This is much less than the queries take typically (and queries can also block for a time due to locking).

Changes

• Run node queries in a dedicated threadpool
• Introduce a configuration option to limit the number of threads used for executing queries in Haskell code.
  This also allows limiting the resources the node is using for non-consensus related operations.
• Default max-concurrent-streams to 200. This is also the default in new releases of the hyper server. Without limiting this there seem to occassionally be some blocking issues as well where the queries end up doing nothing. The default value makes the node more robust in the default configuration.
• Update dependencies, mainly tonic & prost to get fixes for some other blocking behaviour.

Checklist

• My code follows the style of this project.
• The code compiles without warnings.
• I have performed a self-review of the changes.
• I have documented my code, in particular the intent of the
  hard-to-understand areas.
• (If necessary) I have updated the CHANGELOG.

[td202]

Looks good.