You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/03/16 09:41:07 UTC
[GitHub] [arrow] hunght3101 opened a new issue #9719: Serialization ipc by golang
hunght3101 opened a new issue #9719:
URL: https://github.com/apache/arrow/issues/9719
When i serialization ipc by golang then i got error, please help me!
`arrow/ipc: could not read schema from stream: arrow/ipc: could not read message schema: arrow/ipc: could not read message metadata: unexpected EOF`
Cause, i think it is
```
var (
cid = binary.LittleEndian.Uint32(buf)
msgLen int32
)
...
msgLen = int32(cid)
...
buf = make([]byte, msgLen)
_, err = io.ReadFull(r.r, buf)
if err != nil {
return nil, xerrors.Errorf("arrow/ipc: could not read message metadata: %w", err)
}
```
With msgLen very very big it export error
My source
```package main // import "github.com/apache/arrow/go/arrow/ipc/cmd/arrow-cat"
import (
"fmt"
"github.com/apache/arrow/go/arrow"
"github.com/apache/arrow/go/arrow/array"
"github.com/apache/arrow/go/arrow/ipc"
"github.com/apache/arrow/go/arrow/memory"
"io"
"strings"
)
func main() {
meta := arrow.NewMetadata([]string{"k1", "k2"}, []string{"v1", "v2"})
mem := memory.NewGoAllocator()
schema := arrow.NewSchema(
[]arrow.Field{
arrow.Field{Name: "f1-i32", Type: arrow.BinaryTypes.String},
arrow.Field{Name: "f2-f64", Type: arrow.BinaryTypes.String},
},
&meta,
)
b := array.NewRecordBuilder(mem, schema)
defer b.Release()
b.Field(0).(*array.StringBuilder).AppendValues(
[]string{"1", "2"},
nil,
)
b.Field(0).(*array.StringBuilder).AppendValues(
[]string{"1", "2","1", "2"},
[]bool{true, true, false, true},
)
b.Field(1).(*array.StringBuilder).AppendValues(
[]string{"1", "2","1", "2","1", "2","1", "2","1", "2"},
nil,
)
var r io.Reader
r = strings.NewReader("Object Apache Arrow")
rr, err := ipc.NewReader(r, ipc.WithSchema(schema), ipc.WithAllocator(mem))
if err != nil {
fmt.Println(err)
}
fmt.Println(rr)
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] emkornfield commented on issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
emkornfield commented on issue #9719:
URL: https://github.com/apache/arrow/issues/9719#issuecomment-800792035
Please see details at: https://arrow.apache.org/community/ for subscribing to a mailing list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] hunght3101 commented on issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
hunght3101 commented on issue #9719:
URL: https://github.com/apache/arrow/issues/9719#issuecomment-800787463
`If you have further questions we prefer to answer them on the user@ mailing list.`
Sorry, i dont understand. May i put my email when i have a question?
My email : hunght.jp@gmail.com
Could you show an example IPC format using in golang language ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] emkornfield closed issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
emkornfield closed issue #9719:
URL: https://github.com/apache/arrow/issues/9719
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] emkornfield commented on issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
emkornfield commented on issue #9719:
URL: https://github.com/apache/arrow/issues/9719#issuecomment-800784715
@hunght3101 It looks like the data you are trying to read is the string "Object Apache Arrow" which is most definitely no the IPC format. If you have further questions we prefer to answer them on the user@ mailing list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] hunght3101 commented on issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
hunght3101 commented on issue #9719:
URL: https://github.com/apache/arrow/issues/9719#issuecomment-800126751
@eerhardt
Please explain for go case
When i want serialization from apache arrow to flatbuffers
https://github.com/apache/arrow/pull/5280
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] hunght3101 edited a comment on issue #9719: Serialization ipc by golang
Posted by GitBox <gi...@apache.org>.
hunght3101 edited a comment on issue #9719:
URL: https://github.com/apache/arrow/issues/9719#issuecomment-800787463
@emkornfield
Thanks for your reply!
`If you have further questions we prefer to answer them on the user@ mailing list.`
Sorry, i dont understand. May i put my email when i have a question?
My email : hunght.jp@gmail.com
Could you show an example IPC format using in golang language ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org