Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Row group selection support in Parquet chunked reader #13913

Closed
karthikeyann opened this issue Aug 18, 2023 · 2 comments
Closed

Row group selection support in Parquet chunked reader #13913

karthikeyann opened this issue Aug 18, 2023 · 2 comments
Assignees
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.

Comments

@karthikeyann
Copy link
Contributor

          The chunked reader and row-group specification should work together fine (though I don't think we have a test specifically for it).  The only input to the chunked reader is "number of bytes in a chunk" so it doesn't care at all if the row groups being inspected in the first place are the entire file or part of it.

Originally posted by @nvdbaranec in #13348 (comment)

Chunked parquet reader ignores row group selection silently. Chunked parquet reader could support row group selection, and also predicate pushdown using row group filtering.

Enable user row group selection for chunked parquet reader.
As a follow up, add predicate pushdown support using row group filtering to chunked reader too.

@GregoryKimball GregoryKimball added feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue labels Aug 18, 2023
@mhaseeb123
Copy link
Member

@karthikeyann, do you think #15113 and #15764 together close this issue or is there any other work needed?

@karthikeyann
Copy link
Contributor Author

karthikeyann commented Jun 24, 2024

Yes. These are enough.
Great work enabling chunking reader.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

No branches or pull requests

4 participants