Enable testing for ExtensiblePlugins using classpath plugins

[cwperks]

Description

This PR enables testing for ExtensiblePlugins using the classpath plugin pattern present in internalClusterTest.

To do this, this PR tries to load all extensions for classpath plugins by defining an extendedPlugin relationship between a classpath plugin and all other classpath plugins. In normal scenarios, a plugin will declare that it extends another plugin through the extendedPlugins keyword that it places in build.gradle (example). This PR also puts logic in place to ensure that any extensions are only loaded once.

Additional Context

I ran into this issue while using the integration test framework in the security plugin to test out a change I have been experimenting with. The integration test framework utilizes classpath plugins and it was impossible to load an extensible plugin and its extension simultaneously and have the extensions loaded.

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 4176c46: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 4176c46: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for ef041b0: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Attention: Patch coverage is 83.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 72.28%. Comparing base (d7641ca) to head (7ddd9a3).
Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...in/java/org/opensearch/plugins/PluginsService.java	81.81%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16908      +/-   ##
============================================
+ Coverage     72.11%   72.28%   +0.16%     
- Complexity    65151    65275     +124     
============================================
  Files          5299     5299              
  Lines        303534   303535       +1     
  Branches      43941    43941              
============================================
+ Hits         218900   219404     +504     
+ Misses        66648    66118     -530     
- Partials      17986    18013      +27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

✅ Gradle check result for f840850: SUCCESS

[github-actions]

❕ Gradle check result for 69ceaab: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❕ Gradle check result for 7a07307: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[cwperks]

@reta Could you take a look at this small PR? Its a PR related to testing. Instead of trying to create relationships for extending -> extended plugins for classpath plugins it haphazardly tries to loadExtensions for classpathPlugins which guarantees that at some point loadExtensions will be called for the correct extended plugin.

[reta]

@reta Could you take a look at this small PR? Its a PR related to testing. Instead of trying to create relationships for extending -> extended plugins for classpath plugins it haphazardly tries to loadExtensions for classpathPlugins which guarantees that at some point loadExtensions will be called for the correct extended plugin.

@cwperks I am not sure I understand this change:

• this is not the way we load plugins (in production) but we change the logic of PluginService
• we do have mechanisms to run internal cluster tests with plugins, we should be using it for extensible plugins as well I think

[cwperks]

@reta Could you take a look at this small PR? Its a PR related to testing. Instead of trying to create relationships for extending -> extended plugins for classpath plugins it haphazardly tries to loadExtensions for classpathPlugins which guarantees that at some point loadExtensions will be called for the correct extended plugin.

@cwperks I am not sure I understand this change:

• this is not the way we load plugins (in production) but we change the logic of PluginService
• we do have mechanisms to run internal cluster tests with plugins, we should be using it for extensible plugins as well I think

That's the current problem with classpathPlugins, there is no way to define a relationship from classpathPluginA -> classpathPluginB. The reason for that is because its always putting an empty list here and does not call loadExtensions since it always assumes empty list.

This PR creates a haphazard way to solve this problem where each classpathPlugin extends all other classpathPlugins so that it tries to load the extensions. If extensions don't exist it will proceed as normal.

Since its haphazard, the same extensions may be loaded multiple times so this PR also adds logic to ensure they are loaded once. I chose to do it this way to minimize the changes, but if its more desirable to have a mechanism to define relationships between classpathPluginA -> classpathPluginB then I would need to revisit what changes are required.

[reta]

That's the current problem with classpathPlugins, there is no way to define a relationship from classpathPluginA -> classpathPluginB. The reason for that is because its always putting an empty list here and does not call loadExtensions since it always assumes empty list.

To my understanding, the classpathPlugins were quick patch to support limited plugin testing scenarios. May be the cleaner way would be to introduce the alternative constructor to PluginsService which accepts Collection<PluginInfo> classpathPlugins instead of raw Plugins?

    public PluginsService(
        Settings settings,
        Path configPath,
        Path modulesDirectory,
        Path pluginsDirectory,
        Collection<PluginInfo> classpathPlugins
    ) {

The best option would certainly to not build on top of that and use dedicated test plugin, packaged and installed as a plugin instead, I believe.

[github-actions]

❌ Gradle check result for 7ddd9a3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 7ddd9a3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 7ddd9a3: SUCCESS