You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2025/04/01 03:45:38 UTC

[I] Improve performance of native scan [datafusion-comet]

wForget opened a new issue, #1586:
URL: https://github.com/apache/datafusion-comet/issues/1586

   ### What is the problem the feature request solves?
   
   I found several possible optimizations for native scan by profiling CometReadBenchmark.
   
   1. Snappy decompression performance is poor, `output_buf.resize` seems to account for a high proportion, is it necessary? https://github.com/apache/arrow-rs/blob/5b44d6702984fbac8a0faa93b85ec4c1120decd4/parquet/src/compression.rs#L234
   2. Skip utf8 validation, see: https://github.com/apache/arrow-rs/issues/6701
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Improve performance of native scan [datafusion-comet]

Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on issue #1586:
URL: https://github.com/apache/datafusion-comet/issues/1586#issuecomment-4403705914

   > is this issue still valid [@wForget](https://github.com/wForget) ?
   
   I’m no longer working on this, I’ll close this issue for now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Improve performance of native scan [datafusion-comet]

Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget closed issue #1586: Improve performance of native scan
URL: https://github.com/apache/datafusion-comet/issues/1586


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Improve performance of native scan [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1586:
URL: https://github.com/apache/datafusion-comet/issues/1586#issuecomment-4392140004

   is this issue still valid @wForget ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org