pyarrow using socket as sink for StreamWriter

[Bryan Cutler <cu...@gmail.com>]

Hi Devs,

What is the recommended way to use the pyarrow StreamWriter to write to a
socket?  I've the following:

- Use socket directly and get "TypeError: Unable to read from object of
type: <class 'socket._socketobject'>"
  looks like pyarrow expects the object to have a "write" attribute

- Using as a file object with: "sock.makefile("wb", 65536)" get the error
"pyarrow.error.ArrowIOError: IOError: '_fileobject' object has no attribute
'tell'"
  Not sure why exactly "tell" is required.

- using "os.fdopen(sock.fileno(), "wb", 65536)" get a rather cryptic error
that I've been unable to track down:
  terminate called after throwing an instance of 'std::logic_error'
   what():  basic_string::_M_construct null not valid

  Seems to indicate some kind of std::string construction with a NULL, but
I can't find that anywhere

Any suggestions for this?

Thanks,
Bryan

Re: pyarrow using socket as sink for StreamWriter

[Bryan Cutler <cu...@gmail.com>]

Made a Jira for this issue https://issues.apache.org/jira/browse/ARROW-822

On Apr 13, 2017 11:31 AM, "Bryan Cutler" <cu...@gmail.com> wrote:

> Hi Devs,
>
> What is the recommended way to use the pyarrow StreamWriter to write to a
> socket?  I've the following:
>
> - Use socket directly and get "TypeError: Unable to read from object of
> type: <class 'socket._socketobject'>"
>   looks like pyarrow expects the object to have a "write" attribute
>
> - Using as a file object with: "sock.makefile("wb", 65536)" get the error
> "pyarrow.error.ArrowIOError: IOError: '_fileobject' object has no attribute
> 'tell'"
>   Not sure why exactly "tell" is required.
>
> - using "os.fdopen(sock.fileno(), "wb", 65536)" get a rather cryptic error
> that I've been unable to track down:
>   terminate called after throwing an instance of 'std::logic_error'
>    what():  basic_string::_M_construct null not valid
>
>   Seems to indicate some kind of std::string construction with a NULL, but
> I can't find that anywhere
>
> Any suggestions for this?
>
> Thanks,
> Bryan
>