support length 1 arrays and relative metrics

[elray1]

Two distinct sets of updates here:

• update the schema to support length 1 arrays. I'm not sure if this is a core thing about YAML or just some behavior in the R packages we're using to read and validate the data structures -- but it seems like to get validations to pass in the case of arrays of strings with length 1, we need to say that valid entries may be either strings or arrays of strings. Added this to the schema and some test cases.
• support computation of relative metrics:
  • update schema to accept relative_metrics and baseline as properties of target entries
  • add appropriate validations of those things and tests for them
  • if relative metrics are specified, compute them, and add tests for that

[elray1]

the bulk of this code was previously in test-generate_eval_data.R. It's been moved to this separate helper-*.R file, with some minor updates around the include_rel parameter.

[zkamvar]

Per my suggestion in hubverse-org/hubEvals#69 (review), I would recommend saving these outputs as test fixtures instead of generating them when the tests are run. From the looks of it, you only need to generate the tables for the relative metrics and subset them to the columns without relative metrics.

[elray1]

unfortunately the git diff for this file is useless, diff got confused. For this test file, there are three changes:

1. deleted check_exp_scores_for_window function (this moved to separate R script)
2. Added ", no relative metrics" to the description of the first test case, no changes to the test itself.
3. Added a second test case with relative metrics. This is a near-duplicate of the first test case, but it refers to a config file that includes relative metrics and asks check_exp_scores_for_window to include relative metric results.

[zkamvar]

This looks okay to me with the exception of the test helper, as noted in my comment below.

Regarding this comment:

update the schema to support length 1 arrays. I'm not sure if this is a core thing about YAML or just some behavior in the R packages we're using to read and validate the data structures -- but it seems like to get validations to pass in the case of arrays of strings with length 1, we need to say that valid entries may be either strings or arrays of strings. Added this to the schema and some test cases.

This is partially a symptom of jsonlite and partially a symptom of the yaml package. Both have their weird idiosyncrasies and something always gets lost in translation. I think your solution is a good workaround.

[zkamvar]

Per my suggestion in hubverse-org/hubEvals#69 (review), I would recommend saving these outputs as test fixtures instead of generating them when the tests are run. From the looks of it, you only need to generate the tables for the relative metrics and subset them to the columns without relative metrics.

[zkamvar]

Thank you! I have one minor suggestion to remove calls to library(), but otherwise this looks good!

[elray1]

I tried removing the library() calls, but just using rlang::.data didn't work immediately. My take is that this is good enough at this point and it's likely not worth our time to sort out removing that last library(rlang) call.

[zkamvar]

I tried removing the library() calls, but just using rlang::.data didn't work immediately. My take is that this is good enough at this point and it's likely not worth our time to sort out removing that last library(rlang) call.

Yeah, that's a leaky abstraction and definitely not worth it. I'm a little surprised that .data is coming from rlang, since it's a dplyr idiom.