You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Jeff Wickstrom <jw...@esri.com> on 2021/06/10 14:55:10 UTC
Is there a standard Arrow file/table extension?
Hello,
Is there a standard extension for Arrow files with a table (one or more RecordBatch) in them? I have seen ".arrow" and ".dat" used in samples. Seems like I have seen ".at" as well.
I know the Arrow library is agnostic about this. Some usages probably hide the Arrow aspect and have their own file extension.
In my case I want to provide a sample for using our api that outputs and Arrow table in RecordBatch(es) in a file. Then my user will use that in some other library/package for analysis (R, Matlab, etc.). I want the sample(s) to suggest the appropriate extension if one exists.
Thank you in advance for any suggestions.
Jeff
Re: Is there a standard Arrow file/table extension?
Posted by Antoine Pitrou <an...@python.org>.
On Thu, 10 Jun 2021 14:55:10 +0000
Jeff Wickstrom <jw...@esri.com> wrote:
> Hello,
>
> Is there a standard extension for Arrow files with a table (one or more RecordBatch) in them? I have seen ".arrow" and ".dat" used in samples. Seems like I have seen ".at" as well.
>
> I know the Arrow library is agnostic about this. Some usages probably hide the Arrow aspect and have their own file extension.
>
> In my case I want to provide a sample for using our api that outputs and Arrow table in RecordBatch(es) in a file. Then my user will use that in some other library/package for analysis (R, Matlab, etc.). I want the sample(s) to suggest the appropriate extension if one exists.
We would recommend ".arrow" for Arrow IPC files, and potentially
".arrows" for the Arrow IPC streams.
Regards
Antoine.
Re: Is there a standard Arrow file/table extension?
Posted by Nate Bauernfeind <na...@deephaven.io>.
The standard file format for arrow is the IPC file format which writes down
only the flatbuffer + data payloads. There are some specific requirements
about dictionaries that otherwise don't apply to Flight's implementation of
the IPC.
You can find details about it here:
http://arrow.apache.org/docs/format/Columnar.html#ipc-file-format
On Thu, Jun 10, 2021 at 8:56 AM Jeff Wickstrom <jw...@esri.com> wrote:
> Hello,
>
>
>
> Is there a standard extension for Arrow files with a table (one or more
> RecordBatch) in them? I have seen “.arrow” and “.dat” used in samples.
> Seems like I have seen “.at” as well.
>
>
>
> I know the Arrow library is agnostic about this. Some usages probably hide
> the Arrow aspect and have their own file extension.
>
>
>
> In my case I want to provide a sample for using our api that outputs and
> Arrow table in RecordBatch(es) in a file. Then my user will use that in
> some other library/package for analysis (R, Matlab, etc.). I want the
> sample(s) to suggest the appropriate extension if one exists.
>
>
>
> Thank you in advance for any suggestions.
>
>
>
> Jeff
>
--