WIP - Parallelization of DistToPoint metrics

[drewoldag]

Problem & Solution Description (including issue #)

Initial parallelization of PIT metric. Not fully tested yet.
Still need to parallelize CDELoss and Brier.

Code Quality

• My code follows the code style of this project
• I have written unit tests or justified all instances of #pragma: no cover; in the case of a bugfix, a new test that breaks as a result of the bug has been added
• My code contains relevant comments and necessary documentation for future maintainers; the change is reflected in applicable demos/tutorials (with output cleared!) and added/updated docstrings use the NumPy docstring format
• Any breaking changes, described above, are accompanied by backwards compatibility and deprecation warnings

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (96b3e13) to head (515f40e).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #217   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           37        37           
  Lines         2646      2707   +61     
=========================================
+ Hits          2646      2707   +61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[eacharles]

Thanks for doing this. One issue that that there is now a potential circular dependency:

qp.Ensemble.py -> from qp.metrics import quick_moment

qp.metrics.concrete_metric_classes.py -> from qp.ensemble import Ensemble

which means that we can't have
from .concrete_metric_classes import *
in metrics/init.py

We should think a bit about this

[drewoldag]

It seems like we can clean up the imports in a couple ways. Either would be good. Both would be better in my opinion.

1. from qp.ensemble import Ensemle isn't required in concrete_metrics_classes.py. It's only used for type hints on the return type.
2. I might argue that the moment_partial method in the Ensemble class isn't really required, since there are several ways to calculate the moments of an ensemble, (i.e. concrete_metric_classes' MomemMetric(moment_order=X).evaluate(ensemble)) - so we could remove Ensemble's dependency from qp.metrics import quick_moment.

[drewoldag]

I sketched out the UML, and I don't think we have a circular dependency, but I might be wrong about that. I think it's something like this:

quick_moment
^ ^
| calculate_moment
| ^
Ensemble <- concrete_metrics

Given the previous comment, I think we can cut the dependency that concrete_metrics takes on Ensemble and potentially the dependency that Ensemble takes on quick_moment. Then the UML would be:

quick_moment
^
calculate_moment
^
concrete_metrics

And Ensemble wouldn't have any metric-related dependencies.

[eacharles]

Yes, to cleaning up the imports as you described.

[drewoldag]

I can easily pull out the concrete_metrics from qp import Ensemble. I'm less confident about removing the dependency and code in qp.Ensemble.moment_partial. I don't know where that is used, but I can create an issue to do it.

Created issue #219 to track the removal of moment_partial.

[eacharles]

Thanks!!