You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "thexiay (via GitHub)" <gi...@apache.org> on 2024/03/04 09:18:34 UTC

[I] Support native parquet read and hdfs read? [arrow-datafusion-comet]

thexiay opened a new issue, #162:
URL: https://github.com/apache/arrow-datafusion-comet/issues/162

   ### What is the problem the feature request solves?
   
   Now comet read parqeut firstly using javaļ¼Œthen pass it into Rust ScanExec, Can it read from scratch using rust?
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Support native parquet read and hdfs read? [arrow-datafusion-comet]

Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on issue #162:
URL: https://github.com/apache/arrow-datafusion-comet/issues/162#issuecomment-1977130962

   Hi @thexiay , we do have plan to move to full native Parquet implementation, but it probably won't happen anytime soon. The current implementation should support HDFS and most other storage connectors. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Support native parquet read and hdfs read? [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #162: Support native parquet read and hdfs read?
URL: https://github.com/apache/datafusion-comet/issues/162


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support native parquet read and hdfs read? [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #162:
URL: https://github.com/apache/datafusion-comet/issues/162#issuecomment-2775820890

   We now have three Parquet scan implementations - `native_comet` and `native_iceberg_compat` which are hybrid JVM and Rust, and we also have `native_datafusion` which is pure Rust (uses DataFusion's `DataSourceExec`).
   
   `native_iceberg_compat` and `native_datafusion` are experimental still but will be enabled soon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support native parquet read and hdfs read? [arrow-datafusion-comet]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on issue #162:
URL: https://github.com/apache/arrow-datafusion-comet/issues/162#issuecomment-1993775332

   I hope we could reserve current jvm based hdfs impl, which is useful for those hadoop compatible filesystem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org