You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Robert Nishihara (JIRA)" <ji...@apache.org> on 2018/01/06 21:39:05 UTC
[jira] [Updated] (ARROW-1972) Deserialization of buffer objects
(and pandas dataframes) segfaults on different processes.
[ https://issues.apache.org/jira/browse/ARROW-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Nishihara updated ARROW-1972:
------------------------------------
Description:
To see the issue, first serialize a pyarrow buffer.
{code}
import pyarrow as pa
serialized = pa.serialize(pa.frombuffer(b'hello')).to_buffer().to_pybytes()
print(serialized) # b'\x00\x00\x00\x00\x01...'
{code}
Deserializing it within the same process succeeds, however deserializing it in a **separate process** causes a segfault. E.g.,
{code}
import pyarrow as pa
pa.deserialize(b'\x00\x00\x00\x00\x01...') # This segfaults
{code}
The backtrace is
{code}
(lldb) bt
* thread #1, queue = ‘com.apple.main-thread’, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000000000000
frame #1: 0x0000000105605534 libarrow_python.0.dylib`arrow::py::wrap_buffer(buffer=std::__1::shared_ptr<arrow::Buffer>::element_type @ 0x000000010060c348 strong=1 weak=1) at pyarrow.cc:48
frame #2: 0x000000010554fdee libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x0000000100645438, arr=0x0000000100622938, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfd218) at arrow_to_python.cc:173
frame #3: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x0000000100645438, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfd470) at arrow_to_python.cc:208
frame #4: 0x000000010554d302 libarrow_python.0.dylib`arrow::py::DeserializeDict(context=0x0000000108f17818, array=0x0000000100645338, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfddd8) at arrow_to_python.cc:74
frame #5: 0x000000010554f249 libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x00000001006377a8, arr=0x0000000100645298, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfddd8) at arrow_to_python.cc:158
frame #6: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x00000001006377a8, start_idx=0, stop_idx=1, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:208
frame #7: 0x0000000105551fbf libarrow_python.0.dylib`arrow::py::DeserializeObject(context=0x0000000108f17818, obj=0x0000000108f09588, base=0x0000000108f0e528, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:287
frame #8: 0x0000000104abecae lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_18SerializedPyObject_2deserialize(__pyx_v_self=0x0000000108f09570, __pyx_v_context=0x0000000108f17818) at lib.cxx:88592
frame #9: 0x0000000104abdec4 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_18SerializedPyObject_3deserialize(__pyx_v_self=0x0000000108f09570, __pyx_args=0x000000010231f358, __pyx_kwds=0x0000000000000000) at lib.cxx:88514
frame #10: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #11: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108f302d0, arg=0x000000010231f358, kw=0x0000000000000000) at lib.cxx:116108
frame #12: 0x0000000104b0e3fa lib.cpython-36m-darwin.so`__Pyx__PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116147
frame #13: 0x0000000104944bc6 lib.cpython-36m-darwin.so`__Pyx_PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116166
frame #14: 0x0000000104b09873 lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_124deserialize_from(__pyx_self=0x0000000000000000, __pyx_v_source=0x0000000108ddeee8, __pyx_v_base=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90327
frame #15: 0x0000000104b09310 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_125deserialize_from(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108f10d38, __pyx_kwds=0x0000000000000000) at lib.cxx:90260
frame #16: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #17: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf1b0, arg=0x0000000108f10d38, kw=0x0000000000000000) at lib.cxx:116108
frame #18: 0x0000000104b0bf9d lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_128deserialize(__pyx_self=0x0000000000000000, __pyx_v_obj=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90770
frame #19: 0x0000000104b0b7ec lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_129deserialize(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108def1c8, __pyx_kwds=0x0000000000000000) at lib.cxx:90680
frame #20: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #21: 0x0000000108d5c468 plasma.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf240, arg=0x0000000108def1c8, kw=0x0000000000000000) at plasma.cxx:11200
frame #22: 0x0000000108d744a7 plasma.cpython-36m-darwin.so`__pyx_pf_7pyarrow_6plasma_12PlasmaClient_10get(__pyx_v_self=0x0000000108f0e210, __pyx_v_object_ids=0x0000000108deb248, __pyx_v_timeout_ms=0, __pyx_v_serialization_context=0x0000000108f17818) at plasma.cxx:6480
frame #23: 0x0000000108d6c250 plasma.cpython-36m-darwin.so`__pyx_pw_7pyarrow_6plasma_12PlasmaClient_11get(__pyx_v_self=0x0000000108f0e210, __pyx_args=0x0000000102363630, __pyx_kwds=0x0000000000000000) at plasma.cxx:6274
frame #24: 0x000000010008bc5b python`_PyCFunction_FastCallDict + 363
frame #25: 0x00000001001637f2 python`call_function + 146
frame #26: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #27: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #28: 0x0000000100163c4c python`fast_function + 348
frame #29: 0x000000010016383e python`call_function + 222
frame #30: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #31: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #32: 0x0000000100163c4c python`fast_function + 348
frame #33: 0x000000010016383e python`call_function + 222
frame #34: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #35: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #36: 0x0000000100163c4c python`fast_function + 348
frame #37: 0x000000010016383e python`call_function + 222
frame #38: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #39: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #40: 0x00000001001b01dc python`PyRun_InteractiveOneObject + 1132
frame #41: 0x00000001001ad15e python`PyRun_InteractiveLoopFlags + 334
frame #42: 0x00000001001acfeb python`PyRun_AnyFileExFlags + 139
frame #43: 0x00000001001d3378 python`Py_Main + 4632
frame #44: 0x00000001000016bd python`main + 509
frame #45: 0x00007fffb6073235 libdyld.dylib`start + 1
{code}
Note however that if we first serialize something, then it works. E.g., the following succeeds.
{code}
import pyarrow as pa
pa.serialize(1)
pa.deserialize(b'\x00\x00\x00\x00\x01...') # This succeeds!
{code}
I have a potential fix/workaround, which I will post momentarily.
was:
To see the issue, first serialize a pyarrow buffer.
{code}
import pyarrow as pa
serialized = pa.serialize(pa.frombuffer(b'hello')).to_buffer().to_pybytes()
print(serialized) # b'\x00\x00\x00\x00\x01\x00\x00\x00\xcc\x00\x00\x00\x10\x00\x00\x00\x0c\x00\x0e\x00\x06\x00\x05\x00\x08\x00\x00\x00\x0c\x00\x00\x00\x00\x01\x03\x00\x10\x00\x00\x00\x00\x00\n\x00\x08\x00\x00\x00\x04\x00\x00\x00\n\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\xc6\xff\xff\xff\x00\x00\x01\x0e|\x00\x00\x00\x18\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x004\x00\x00\x00\x08\x00\x0c\x00\x06\x00\x08\x00\x08\x00\x00\x00\x00\x00\x01\x00\x04\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x14\x00\x08\x00\x06\x00\x07\x00\x0c\x00\x00\x00\x10\x00\x00\x00\x12\x00\x00\x00\x00\x00\x01\x02$\x00\x00\x00\x14\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x00\x0c\x00\x08\x00\x07\x00\x08\x00\x00\x00\x00\x00\x00\x01 \x00\x00\x00\x06\x00\x00\x00buffer\x00\x00\x04\x00\x00\x00list\x00\x00\x00\x00\x00\x00\x00\x00\xcc\x00\x00\x00\x14\x00\x00\x00\x00\x00\x00\x00\x0c\x00\x16\x00\x06\x00\x05\x00\x08\x00\x0c\x00\x0c\x00\x00\x00\x00\x03\x03\x00\x18\x00\x00\x00h\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00\x18\x00\x0c\x00\x04\x00\x08\x00\n\x00\x00\x00l\x00\x00\x00\x10\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00`\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00`\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00hello'
{code}
Deserializing it within the same process succeeds, however deserializing it in a **separate process** causes a segfault. E.g.,
{code}
import pyarrow as pa
pa.deserialize(b'\x00\x00\x00\x00\x01...') # This segfaults
{code}
The backtrace is
{code}
(lldb) bt
* thread #1, queue = ‘com.apple.main-thread’, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000000000000
frame #1: 0x0000000105605534 libarrow_python.0.dylib`arrow::py::wrap_buffer(buffer=std::__1::shared_ptr<arrow::Buffer>::element_type @ 0x000000010060c348 strong=1 weak=1) at pyarrow.cc:48
frame #2: 0x000000010554fdee libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x0000000100645438, arr=0x0000000100622938, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfd218) at arrow_to_python.cc:173
frame #3: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x0000000100645438, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfd470) at arrow_to_python.cc:208
frame #4: 0x000000010554d302 libarrow_python.0.dylib`arrow::py::DeserializeDict(context=0x0000000108f17818, array=0x0000000100645338, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfddd8) at arrow_to_python.cc:74
frame #5: 0x000000010554f249 libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x00000001006377a8, arr=0x0000000100645298, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfddd8) at arrow_to_python.cc:158
frame #6: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x00000001006377a8, start_idx=0, stop_idx=1, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:208
frame #7: 0x0000000105551fbf libarrow_python.0.dylib`arrow::py::DeserializeObject(context=0x0000000108f17818, obj=0x0000000108f09588, base=0x0000000108f0e528, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:287
frame #8: 0x0000000104abecae lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_18SerializedPyObject_2deserialize(__pyx_v_self=0x0000000108f09570, __pyx_v_context=0x0000000108f17818) at lib.cxx:88592
frame #9: 0x0000000104abdec4 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_18SerializedPyObject_3deserialize(__pyx_v_self=0x0000000108f09570, __pyx_args=0x000000010231f358, __pyx_kwds=0x0000000000000000) at lib.cxx:88514
frame #10: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #11: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108f302d0, arg=0x000000010231f358, kw=0x0000000000000000) at lib.cxx:116108
frame #12: 0x0000000104b0e3fa lib.cpython-36m-darwin.so`__Pyx__PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116147
frame #13: 0x0000000104944bc6 lib.cpython-36m-darwin.so`__Pyx_PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116166
frame #14: 0x0000000104b09873 lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_124deserialize_from(__pyx_self=0x0000000000000000, __pyx_v_source=0x0000000108ddeee8, __pyx_v_base=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90327
frame #15: 0x0000000104b09310 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_125deserialize_from(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108f10d38, __pyx_kwds=0x0000000000000000) at lib.cxx:90260
frame #16: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #17: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf1b0, arg=0x0000000108f10d38, kw=0x0000000000000000) at lib.cxx:116108
frame #18: 0x0000000104b0bf9d lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_128deserialize(__pyx_self=0x0000000000000000, __pyx_v_obj=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90770
frame #19: 0x0000000104b0b7ec lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_129deserialize(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108def1c8, __pyx_kwds=0x0000000000000000) at lib.cxx:90680
frame #20: 0x000000010008b5f1 python`PyCFunction_Call + 145
frame #21: 0x0000000108d5c468 plasma.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf240, arg=0x0000000108def1c8, kw=0x0000000000000000) at plasma.cxx:11200
frame #22: 0x0000000108d744a7 plasma.cpython-36m-darwin.so`__pyx_pf_7pyarrow_6plasma_12PlasmaClient_10get(__pyx_v_self=0x0000000108f0e210, __pyx_v_object_ids=0x0000000108deb248, __pyx_v_timeout_ms=0, __pyx_v_serialization_context=0x0000000108f17818) at plasma.cxx:6480
frame #23: 0x0000000108d6c250 plasma.cpython-36m-darwin.so`__pyx_pw_7pyarrow_6plasma_12PlasmaClient_11get(__pyx_v_self=0x0000000108f0e210, __pyx_args=0x0000000102363630, __pyx_kwds=0x0000000000000000) at plasma.cxx:6274
frame #24: 0x000000010008bc5b python`_PyCFunction_FastCallDict + 363
frame #25: 0x00000001001637f2 python`call_function + 146
frame #26: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #27: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #28: 0x0000000100163c4c python`fast_function + 348
frame #29: 0x000000010016383e python`call_function + 222
frame #30: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #31: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #32: 0x0000000100163c4c python`fast_function + 348
frame #33: 0x000000010016383e python`call_function + 222
frame #34: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #35: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #36: 0x0000000100163c4c python`fast_function + 348
frame #37: 0x000000010016383e python`call_function + 222
frame #38: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
frame #39: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
frame #40: 0x00000001001b01dc python`PyRun_InteractiveOneObject + 1132
frame #41: 0x00000001001ad15e python`PyRun_InteractiveLoopFlags + 334
frame #42: 0x00000001001acfeb python`PyRun_AnyFileExFlags + 139
frame #43: 0x00000001001d3378 python`Py_Main + 4632
frame #44: 0x00000001000016bd python`main + 509
frame #45: 0x00007fffb6073235 libdyld.dylib`start + 1
{code}
Note however that if we first serialize something, then it works. E.g., the following succeeds.
{code}
import pyarrow as pa
pa.serialize(1)
pa.deserialize(b'\x00\x00\x00\x00\x01...') # This succeeds!
{code}
I have a potential fix/workaround, which I will post momentarily.
> Deserialization of buffer objects (and pandas dataframes) segfaults on different processes.
> -------------------------------------------------------------------------------------------
>
> Key: ARROW-1972
> URL: https://issues.apache.org/jira/browse/ARROW-1972
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Robert Nishihara
>
> To see the issue, first serialize a pyarrow buffer.
> {code}
> import pyarrow as pa
> serialized = pa.serialize(pa.frombuffer(b'hello')).to_buffer().to_pybytes()
> print(serialized) # b'\x00\x00\x00\x00\x01...'
> {code}
> Deserializing it within the same process succeeds, however deserializing it in a **separate process** causes a segfault. E.g.,
> {code}
> import pyarrow as pa
> pa.deserialize(b'\x00\x00\x00\x00\x01...') # This segfaults
> {code}
> The backtrace is
> {code}
> (lldb) bt
> * thread #1, queue = ‘com.apple.main-thread’, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
> * frame #0: 0x0000000000000000
> frame #1: 0x0000000105605534 libarrow_python.0.dylib`arrow::py::wrap_buffer(buffer=std::__1::shared_ptr<arrow::Buffer>::element_type @ 0x000000010060c348 strong=1 weak=1) at pyarrow.cc:48
> frame #2: 0x000000010554fdee libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x0000000100645438, arr=0x0000000100622938, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfd218) at arrow_to_python.cc:173
> frame #3: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x0000000100645438, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfd470) at arrow_to_python.cc:208
> frame #4: 0x000000010554d302 libarrow_python.0.dylib`arrow::py::DeserializeDict(context=0x0000000108f17818, array=0x0000000100645338, start_idx=0, stop_idx=2, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfddd8) at arrow_to_python.cc:74
> frame #5: 0x000000010554f249 libarrow_python.0.dylib`arrow::py::GetValue(context=0x0000000108f17818, parent=0x00000001006377a8, arr=0x0000000100645298, index=0, type=0, base=0x0000000108f0e528, blobs=0x0000000108f09588, result=0x00007fff5fbfddd8) at arrow_to_python.cc:158
> frame #6: 0x000000010554d93a libarrow_python.0.dylib`arrow::py::DeserializeList(context=0x0000000108f17818, array=0x00000001006377a8, start_idx=0, stop_idx=1, base=0x0000000108f0e528, blobs=0x0000000108f09588, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:208
> frame #7: 0x0000000105551fbf libarrow_python.0.dylib`arrow::py::DeserializeObject(context=0x0000000108f17818, obj=0x0000000108f09588, base=0x0000000108f0e528, out=0x00007fff5fbfdfe8) at arrow_to_python.cc:287
> frame #8: 0x0000000104abecae lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_18SerializedPyObject_2deserialize(__pyx_v_self=0x0000000108f09570, __pyx_v_context=0x0000000108f17818) at lib.cxx:88592
> frame #9: 0x0000000104abdec4 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_18SerializedPyObject_3deserialize(__pyx_v_self=0x0000000108f09570, __pyx_args=0x000000010231f358, __pyx_kwds=0x0000000000000000) at lib.cxx:88514
> frame #10: 0x000000010008b5f1 python`PyCFunction_Call + 145
> frame #11: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108f302d0, arg=0x000000010231f358, kw=0x0000000000000000) at lib.cxx:116108
> frame #12: 0x0000000104b0e3fa lib.cpython-36m-darwin.so`__Pyx__PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116147
> frame #13: 0x0000000104944bc6 lib.cpython-36m-darwin.so`__Pyx_PyObject_CallOneArg(func=0x0000000108f302d0, arg=0x0000000108f17818) at lib.cxx:116166
> frame #14: 0x0000000104b09873 lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_124deserialize_from(__pyx_self=0x0000000000000000, __pyx_v_source=0x0000000108ddeee8, __pyx_v_base=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90327
> frame #15: 0x0000000104b09310 lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_125deserialize_from(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108f10d38, __pyx_kwds=0x0000000000000000) at lib.cxx:90260
> frame #16: 0x000000010008b5f1 python`PyCFunction_Call + 145
> frame #17: 0x0000000104941208 lib.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf1b0, arg=0x0000000108f10d38, kw=0x0000000000000000) at lib.cxx:116108
> frame #18: 0x0000000104b0bf9d lib.cpython-36m-darwin.so`__pyx_pf_7pyarrow_3lib_128deserialize(__pyx_self=0x0000000000000000, __pyx_v_obj=0x0000000108f0e528, __pyx_v_context=0x0000000108f17818) at lib.cxx:90770
> frame #19: 0x0000000104b0b7ec lib.cpython-36m-darwin.so`__pyx_pw_7pyarrow_3lib_129deserialize(__pyx_self=0x0000000000000000, __pyx_args=0x0000000108def1c8, __pyx_kwds=0x0000000000000000) at lib.cxx:90680
> frame #20: 0x000000010008b5f1 python`PyCFunction_Call + 145
> frame #21: 0x0000000108d5c468 plasma.cpython-36m-darwin.so`__Pyx_PyObject_Call(func=0x0000000108baf240, arg=0x0000000108def1c8, kw=0x0000000000000000) at plasma.cxx:11200
> frame #22: 0x0000000108d744a7 plasma.cpython-36m-darwin.so`__pyx_pf_7pyarrow_6plasma_12PlasmaClient_10get(__pyx_v_self=0x0000000108f0e210, __pyx_v_object_ids=0x0000000108deb248, __pyx_v_timeout_ms=0, __pyx_v_serialization_context=0x0000000108f17818) at plasma.cxx:6480
> frame #23: 0x0000000108d6c250 plasma.cpython-36m-darwin.so`__pyx_pw_7pyarrow_6plasma_12PlasmaClient_11get(__pyx_v_self=0x0000000108f0e210, __pyx_args=0x0000000102363630, __pyx_kwds=0x0000000000000000) at plasma.cxx:6274
> frame #24: 0x000000010008bc5b python`_PyCFunction_FastCallDict + 363
> frame #25: 0x00000001001637f2 python`call_function + 146
> frame #26: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
> frame #27: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
> frame #28: 0x0000000100163c4c python`fast_function + 348
> frame #29: 0x000000010016383e python`call_function + 222
> frame #30: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
> frame #31: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
> frame #32: 0x0000000100163c4c python`fast_function + 348
> frame #33: 0x000000010016383e python`call_function + 222
> frame #34: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
> frame #35: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
> frame #36: 0x0000000100163c4c python`fast_function + 348
> frame #37: 0x000000010016383e python`call_function + 222
> frame #38: 0x00000001001614d5 python`_PyEval_EvalFrameDefault + 47093
> frame #39: 0x0000000100154aab python`_PyEval_EvalCodeWithName + 427
> frame #40: 0x00000001001b01dc python`PyRun_InteractiveOneObject + 1132
> frame #41: 0x00000001001ad15e python`PyRun_InteractiveLoopFlags + 334
> frame #42: 0x00000001001acfeb python`PyRun_AnyFileExFlags + 139
> frame #43: 0x00000001001d3378 python`Py_Main + 4632
> frame #44: 0x00000001000016bd python`main + 509
> frame #45: 0x00007fffb6073235 libdyld.dylib`start + 1
> {code}
> Note however that if we first serialize something, then it works. E.g., the following succeeds.
> {code}
> import pyarrow as pa
> pa.serialize(1)
> pa.deserialize(b'\x00\x00\x00\x00\x01...') # This succeeds!
> {code}
> I have a potential fix/workaround, which I will post momentarily.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)