How to: share underlying HTTP clients

[skeet70]

If I make M (potentially a million+) clients of smithy sdks, each with potentially different credentials or other config values, how can I optionally get them to share the same underlying HTTP client to avoid M clients being created each with their own duplicate cert chains and other cruft? Is this the HttpConnector and the DynConnector vs MakeConnectorFn stuff, or is that something else entirely? If it is that, could it be made to apply across multiple different clients (maybe a shared connector fn that clones an Arc), or is it only controlling per-call behavior of that client?

[jdisanti]

There's no built-in support for sharing connectors currently, but it is possible to do it manually.

To share the connector between clients, you would need to use the connector() function on the Builder, and give it some kind of shared connector implementation. The built-in DynConnector calls the Clone implementation of whatever is inside of it.

A connector needs to implement the SmithyConnector trait, which is automatically implemented for anything that respects these bounds.

I think you'll have to do something like this:

#[derive(Clone)]
struct SharedConnector<I> {
    inner: /* some shareable type here, such as Arc<Mutex<I>> */
}

impl<I> SharedConnector<I> {
   fn new(inner: I) -> Self { Self { inner } }
}

impl<I> Service<Request<SdkBody>> for SharedConnector<I>
where
    I: Service<Request<SdkBody>, Response = Response<SdkBody>>,
    I::Error: Into<ConnectorError> + Send + Sync + 'static,
    I::Future: Send + 'static,
{
    type Response = Response<SdkBody>;
    type Error = I::Error;
    type Future = I::Future;

    fn poll_ready(
        &mut self,
        cx: &mut std::task::Context<'_>,
    ) -> std::task::Poll<Result<(), Self::Error>> {
        // Defer to `inner` (needs some tweaking depending on synchronization chosen):
        self.inner.poll_ready(cx)
    }

    fn call(&mut self, req: http::Request<SdkBody>) -> Self::Future {
        // Defer to `inner` (needs some tweaking depending on synchronization chosen):
        self.inner.call(req)
    }
}

You'll have to decide how to share the underlying connector stored in inner, whether that be via Arc<Mutex<...>> or something else.

Ultimately, you should be able to do something like this:

let shared_connector = SharedConnector::new(aws_smithy_client::conn::https());
let client1 = Builder::new().connector(shared_connector.clone()).middleware(...).build();
let client2 = Builder::new().connector(shared_connector.clone()).middleware(...).build();

[skeet70]

That seems like it's a bit miserable to implement with my currently accessible reqwest::Client, but I'll give it a try and find out.

[Velfi]

A possible alternative is creating requests yourself and then attempting to sign them with our aws-sigv4 runtime crate.

[skeet70]

vs using the SDK at all? Or could that be plugged in to the existing SDK? I do have a custom sharable client written for GCP and Thales CTM since they don't have good Rust options. I have a custom sharable Azure client that is a wrapper around reqwest and uses their rust signing crate. I was previously using Rusoto (now deprecated/unmaintained because of this project) for AWS and was hoping with an official client in Rust to avoid building the error stack, auth refresh, and retries myself that I've done with the other cloud clients.

[Velfi]

You wouldn't use the SDK at all, except for the request signer. You'd have to handle your own credentials somehow although it's possible you could re-use some of our credential fetching code for that.

Ideally, the SDKs would allow for this, and it's something we're looking into once hyper v1 is released. Still, I thought you might like to know that there are other options.

[skeet70]

I started down this path and ran into all kinds of issues implementing something that would be performant and could meet those bounds. The Service requirement on I (and as far as I could tell using the Adapter) was particularly restrictive. I'm going to try testing with the client as-is to see if it tips over with large numbers of clients or not. If so I'll revert to our Rusoto code until this project is more mature and take the hit on it not being maintained for now but working. This has taken too much time already.

If performant clients sharing an http connector is something you care about, I'd suggest trying to implement a working version of this yourself to see if there are improvements that could be made to the interface. My use case is a particular one, but it seems like wanting to use a single connector across multiple SDKs would be a common one and would have similar issues.

Thanks for your help!

[rcoh]

I just wanted to follow up on this—HTTPS_NATIVE_ROOTS used by the SDK is actually a lazy static that is only created once: https://github.com/awslabs/smithy-rs/blob/3cdf49e6cb08bf4c1a18f229de5da47db12a3d43/rust-runtime/aws-smithy-client/src/lib.rs#L60-L71

So there will be only one certificate chain created as it is with Rustls—if you're seeing different behavior please let us know!