Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compact into a CSR #52

Merged
merged 26 commits into from
Apr 21, 2024
Merged

compact into a CSR #52

merged 26 commits into from
Apr 21, 2024

Conversation

meyerzinn
Copy link
Collaborator

LS_CSR compaction is already slow. We might as well re-build the CSR into an actual, bona-fide CSR.

@meyerzinn
Copy link
Collaborator Author

A little bit more on my thinking here: Our ingestion/compaction right now is slower than rebuilding the CSR from an adjacency vector. That's wild and means the contention on the atomic variable is really hurting our ability to do copies in parallel. This PR is a bet that it pays off to just CSR-ify the graph now and get the optimization benefits for workloads like triangle counting.

If we had more time, I would also change the ingestion logic to take an entire batch at once, and regularize it into a CSR to just directly append on the log. Unfortunately I don't think we have time to retool that.

@meyerzinn meyerzinn force-pushed the meyer/csrify-lscsr branch from 621c447 to 4f7dd0e Compare April 17, 2024 20:54
@meyerzinn meyerzinn force-pushed the meyer/csrify-lscsr branch from 2fb3af4 to 562ba70 Compare April 21, 2024 22:14
@meyerzinn
Copy link
Collaborator Author

This PR:

  • Changes LargeVector to have the same construction semantics as LargeArray (i.e. constructors and destructors are not called by the vector -- it is up to the user to do so if the constructor is nontrivial).
  • Removes some extraneous methods from LargeVector
  • Adds a new method, addBatchTopologyOnly, which ingests a batched update to an LS_CSR. We basically turn the batched update into a mini-CSR to make copies onto the log more parallel. addEdgesTopologyOnly still exists and continues to work, but we are going to start benchmarking addBatch instead.

@meyerzinn meyerzinn merged commit 3cff9f5 into master Apr 21, 2024
4 checks passed
@meyerzinn meyerzinn deleted the meyer/csrify-lscsr branch April 21, 2024 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants