Find by jobid and output aggregate filter data #1322
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
milroy
force-pushed
the
find-jobid
branch
2 times, most recently
from
9952079 to
ed2e53b
Compare
4 tasks
milroy
force-pushed
the
find-jobid
branch
4 times, most recently
from
0ca9f9b to
3ddb74b
Compare
I'd lie to have @trws feedback before merging this PR. |
Problem: Partial cancellation is failing under rabbit-related circumstances, possibly orphaning job-related data on allocated vertices. Having a way to find all vertices with jobid-related data would be very useful in diagnosing the problem. Furthermore, it is likely the partial cancellation failures are related to aggregate filter (pruning filter) counts. Add capability to the evaulator to validate and evaluate by jobid, span, tag, reservation, allocation, and agfilter.
Problem: a jobid and aggregate filter boolean are now supported in the evaluator and validator. The evaluator needs support for returning the parsed jobid and agfilter bool to the traverser, so it can change behavior based on whether a jobid or agfilter boolean is present in the evaluation string. Add the capability to extract the jobid as a uint64_t and agfilter as a bool and return them to the client.
Problem: the evaluator unit tests do not support extraction and the new options. Add support in the tests and test cases for the new options.
Problem: the issues with partial cancel have exposed the need for printing the state of resource graph vertices based on the jobid and aggregate filter. Add traverser support for gathering agfilter utilization and total counts for all resources via updated function calls.
Problem: a planner_multi C API function that returns the number of resources utilized by a jobid by resource type is needed to write out aggregate filter counts. Add a new C API function to planner_multi that returns the number of resources utilized by a jobid by calling the planner C API function for the specified resource type index.
Problem: the writer needs support for outputting the aggregate filter counts by type. Add the capability to write out the aggregate filter state by passing the data to map2json.
Problem: after extracting the jobid and/or agfilter boolean with the evaluator, the traverser needs to fetch the aggregate filter counts by jobid (if the ID is present in the find string) or total utilization. Add conditional behavior to fetch appropriate aggregate filter data depending on the presence of jobid and the value of the agfilter boolean.
Problem: there is a typo in the planner C API function to reduce a span. Fix the typo.
Problem: the testsuite doesn't have tests for the new find behavior Add tests.
milroy
changed the title
|
I updated the PR to include an alternative method of extracting This PR is no longer WIP and is ready for review. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1322 +/- ##
========================================
+ Coverage 75.3% 75.5% +0.2%
========================================
Files 111 111
Lines 16042 16253 +211
========================================
+ Hits 12081 12276 +195
- Misses 3961 3977 +16
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Errors in partial cancel are causing inability to schedule resources in production. While evidence points to orphaned (i.e., not cleared) vertex data such as aggregate filters, it is difficult to investigate and debug without being able to inspect the state of the vertices.
The PR adds capability to print vertex-specific data by
jobidand anagfilterboolean. The state can be printed based on whether the jobid is in theallocations,reservations,job2span, andtagsmaps.This PR is WIP because I need to add support for all writers.