You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Clark0 (via GitHub)" <gi...@apache.org> on 2023/11/14 07:21:12 UTC

[I] Parquet RecordReader Skip performance is bad [arrow]

Clark0 opened a new issue, #38707:
URL: https://github.com/apache/arrow/issues/38707

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   ![sK1imZDrYQ](https://github.com/apache/arrow/assets/26629109/2817e9d7-0f19-4b25-9939-5feaa025dd89)
   I did a perf and found most cpu time spent on alloc and dealloc memory. The issue looks like relates to here https://github.com/apache/arrow/blob/9e5c5b5274928c92a642657e73e8cf84f17bd7b7/cpp/src/parquet/column_reader.cc#L1494C18-L1494C32
   
   ### Component(s)
   
   C++


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Parquet RecordReader Skip performance is bad [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #38707:
URL: https://github.com/apache/arrow/issues/38707#issuecomment-1845515341

   Or this could be a fixed-size buffer and skipping the levels could be done in chunks...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Parquet RecordReader Skip performance is bad [arrow]

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #38707:
URL: https://github.com/apache/arrow/issues/38707#issuecomment-1811638762

   Would you mind try to set these record "cached" like `valid_bits_` and `Resize` without thrink to fit?
   
   I think this might help in some cases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org