You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Ying Zhou <yz...@gmail.com> on 2020/12/12 09:50:42 UTC
[C++] Are stream adapters necessary for the Arrow2ORC adapter?
Hi,
As the developer who is testing the APIs in the Arrow2ORC adapter I have a question on whether I should necessarily take some Arrow I/O interfaces as parameters. Are we not supposed to directly use the path of the file we write to and directly use an ORC function to open it? If we do need to exclusively use classes in arrow/io to open files given how the Arrow integration with Parquet and ORC2Arrow adapter work it seems that I should wrap arrrow::io::OutputStream in an implementation of orc::OutputStream . Is it one of the right ways to do it? Thanks!
Ying Zhou
Re: [C++] Are stream adapters necessary for the Arrow2ORC adapter?
Posted by Wes McKinney <we...@gmail.com>.
It would be more flexible to use the Arrow IO interfaces. That would
enable you to read and write to remote filesystems as well. I would
recommend that over e.g. passing in a file path.
On Sat, Dec 12, 2020 at 3:51 AM Ying Zhou <yz...@gmail.com> wrote:
>
> Hi,
>
> As the developer who is testing the APIs in the Arrow2ORC adapter I have a question on whether I should necessarily take some Arrow I/O interfaces as parameters. Are we not supposed to directly use the path of the file we write to and directly use an ORC function to open it? If we do need to exclusively use classes in arrow/io to open files given how the Arrow integration with Parquet and ORC2Arrow adapter work it seems that I should wrap arrrow::io::OutputStream in an implementation of orc::OutputStream . Is it one of the right ways to do it? Thanks!
>
> Ying Zhou
Re: [C++] Are stream adapters necessary for the Arrow2ORC adapter?
Posted by Wes McKinney <we...@gmail.com>.
It would be more flexible to use the Arrow IO interfaces. That would
enable you to read and write to remote filesystems as well. I would
recommend that over e.g. passing in a file path.
On Sat, Dec 12, 2020 at 3:51 AM Ying Zhou <yz...@gmail.com> wrote:
>
> Hi,
>
> As the developer who is testing the APIs in the Arrow2ORC adapter I have a question on whether I should necessarily take some Arrow I/O interfaces as parameters. Are we not supposed to directly use the path of the file we write to and directly use an ORC function to open it? If we do need to exclusively use classes in arrow/io to open files given how the Arrow integration with Parquet and ORC2Arrow adapter work it seems that I should wrap arrrow::io::OutputStream in an implementation of orc::OutputStream . Is it one of the right ways to do it? Thanks!
>
> Ying Zhou