You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "David Li (Jira)" <ji...@apache.org> on 2021/01/08 02:43:00 UTC
[jira] [Commented] (ARROW-11177) [Java] ArrowMessage failed to
parse compressed grpc stream
[ https://issues.apache.org/jira/browse/ARROW-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260950#comment-17260950 ]
David Li commented on ARROW-11177:
----------------------------------
Thanks for the report!
We haven't tried to use gRPC's compression yet. Note Arrow itself supports compression of data (e.g. for the FeatherV2 format) which may also be an option (but that is also not yet something we've tried.) It would be interesting to see which performs better. IIRC, gRPC only offers a few simple codecs (gz), but Arrow supports more expensive ones too (lz4, zstd, etc).
> [Java] ArrowMessage failed to parse compressed grpc stream
> ----------------------------------------------------------
>
> Key: ARROW-11177
> URL: https://issues.apache.org/jira/browse/ARROW-11177
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Affects Versions: 2.0.1, 3.0.0
> Reporter: Jackey Lee
> Priority: Major
>
> InvalidProtocolBufferException will be thrown in ArrowMessage.frame if we use gzip compress in Grpc.
> The reason is stream.available will still be 1 after we read compressed data. There should be another attempt read to change InflaterInputStream.reachEOF to be true, and make stream.available to be 0.
> Thus we need check stream.available after read tag from stream.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)