You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Shawn Yang <sh...@gmail.com> on 2020/10/12 07:37:40 UTC
Is there a `write_record_batch` method corresonding to `pa.ipc.read_record_batch`?
I want to write a record batch as ipc message separately without writing a
schema. In my case, the schema is known to peers ahead of time. I noticed
arrow java already has this method
`org.apache.arrow.vector.ipc.message.MessageSerializer#serialize(org.apache.arrow.vector.ipc.WriteChannel,
org.apache.arrow.vector.ipc.message.ArrowRecordBatch)`
Re: Is there a `write_record_batch` method corresonding to `pa.ipc.read_record_batch`?
Posted by Wes McKinney <we...@gmail.com>.
Use RecordBatch.serialize to do this.
On Wed, Oct 21, 2020 at 11:18 PM Micah Kornfield <em...@gmail.com>
wrote:
> Hi Shawn,
> This method exists in the C++ implementation [1], so it is likely
> reasonable to expose some form in python (I couldn't find it either in
> pyarrow). This should be reasonable straight-forward (you could follow the
> path taken for read implementation) if it is something you wanted to
> contribute.
>
> Thanks,
> -Micah
>
> [1]
> https://github.com/apache/arrow/blob/3694794bdfd0677b95b8c95681e392512f1c9237/cpp/src/arrow/ipc/writer.h#L169
>
> On Mon, Oct 12, 2020 at 12:38 AM Shawn Yang <sh...@gmail.com>
> wrote:
>
>> I want to write a record batch as ipc message separately without writing
>> a schema. In my case, the schema is known to peers ahead of time. I noticed
>> arrow java already has this method
>> `org.apache.arrow.vector.ipc.message.MessageSerializer#serialize(org.apache.arrow.vector.ipc.WriteChannel,
>> org.apache.arrow.vector.ipc.message.ArrowRecordBatch)`
>>
>
Re: Is there a `write_record_batch` method corresonding to `pa.ipc.read_record_batch`?
Posted by Micah Kornfield <em...@gmail.com>.
Hi Shawn,
This method exists in the C++ implementation [1], so it is likely
reasonable to expose some form in python (I couldn't find it either in
pyarrow). This should be reasonable straight-forward (you could follow the
path taken for read implementation) if it is something you wanted to
contribute.
Thanks,
-Micah
[1]
https://github.com/apache/arrow/blob/3694794bdfd0677b95b8c95681e392512f1c9237/cpp/src/arrow/ipc/writer.h#L169
On Mon, Oct 12, 2020 at 12:38 AM Shawn Yang <sh...@gmail.com> wrote:
> I want to write a record batch as ipc message separately without writing a
> schema. In my case, the schema is known to peers ahead of time. I noticed
> arrow java already has this method
> `org.apache.arrow.vector.ipc.message.MessageSerializer#serialize(org.apache.arrow.vector.ipc.WriteChannel,
> org.apache.arrow.vector.ipc.message.ArrowRecordBatch)`
>