[FLINK-34369][connectors/elasticsearch] Elasticsearch connector supports SSL context

[liuml07]

https://issues.apache.org/jira/browse/FLINK-34369

02/05/2024

This is a preliminary PR that was tested locally with secure ES clusters. Plan is to get early feedback, add some tests, and test with deployed jobs.

[boring-cyborg]

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

[reta]

@liuml07 could you please use the same configuration/API model as [1] does for SSL support in OpenSearch? Besides just having familiar configuration, the API is friendly to SQL connector (where providing hostname verifier could be challenging), thank you.

The idea basically is that NetworkClientConfig has a setting:

• allowInsecure: boolean (uses trustall model in case of self-signed certs)
• you could certainly also keep more elaborate configuration with SSLContext / SSLEngine / ... in case it is needed

Thank you.

[1] https://github.com/apache/flink-connector-opensearch/tree/main/flink-connector-opensearch

[liuml07]

@reta Thanks for the advice. I have updated the patch accordingly. I think the allowInsecure and the sslContextSupplier are mutually exclusive, so I added some validation and unit test for that. I can work on a separate on for table if this looks right.

[snuyanzin]

@liuml07 thanks for the contribution
it seems the ci failed because of spotless
could you please apply spotless to the PR?

[snuyanzin]

@reta may I ask you to have another review iteration please?

[reta]

Since there is overlap between this setting and suppliers, you may just rely on SSL context supplier here:

this.allowInsecure = allowInsecure;
if (this.allowInsecure) {
    return setSslContextSupplier(SSLContexts.custom().loadTrustMaterial(TrustAllStrategy.INSTANCE)::build);
}

[reta]

I think wiring allowInsecure through sslContextSupplier (here https://github.com/apache/flink-connector-elasticsearch/pull/91/files#r1574279683) would make the logic simpler

[liuml07]

Thanks for reviewing and your approval. Could you help merge this @reta?

[snuyanzin]

Thanks for the contribution @liuml07
Thanks for the review @reta

[boring-cyborg]

Awesome work, congrats on your first merged pull request!