You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Antoine Pitrou <an...@python.org> on 2020/04/16 09:47:39 UTC
ORC JNI wrapper bugs [Re: 0.17 release procedure]
The ORC JNI wrapper is currently crashing on these lines:
https://github.com/apache/arrow/blob/master/cpp/src/jni/orc/jni_wrapper.cpp#L279-L281
because in C++, certain buffers can be omitted by passing a null
popinter (specifically the null bitmap, if there are no nulls).
Therefore `buffer` in the lines above is a null pointer.
(I tried replacing the null buffer with a 0-byte buffer: it crashes
further down the road...)
Since this code has been there since ARROW-4714 was committed, my
intuition is that the JNI ORC wrapper was only exercised in very
specific use cases where C++ buffers are never null.
My opinion is therefore that the ORC JNI tests should be ignored for
this release, and fixed later by some motivated developer.
Regards
Antoine.
Le 16/04/2020 à 02:17, Krisztián Szűcs a écrit :
> Hi,
>
> We've merged the last required pull requests later today[/yesterday],
> so I started to cut RC0.
> The release process doesn't go smoothly, among other smaller problems
> I discovered a crash with the ORC Java JNI bindings (local error [1]),
> turned out that we don't run the orc-jni tests on the CI. I put up a PR
> to enable them [2], it has not reproduced the exact issue yet.
>
> Any help from the JNI developers would be appreciated. I can also cut
> RC0 with JNI disabled.
>
> [1] https://gist.github.com/kszucs/67205eda6cd19e3cd08c86894f5b4c2d
> [2] https://github.com/apache/arrow/pull/6953
>
> Regards, Krisztian
>
Re: ORC JNI wrapper bugs [Re: 0.17 release procedure]
Posted by Fan Liya <li...@gmail.com>.
One way to skip a test class is to place a "@Ignore" annotation in front of
the class declaration.
Best,
Liya Fan
On Thu, Apr 16, 2020 at 7:29 PM Krisztián Szűcs <sz...@gmail.com>
wrote:
> On Thu, Apr 16, 2020 at 11:47 AM Antoine Pitrou <an...@python.org>
> wrote:
> >
> >
> > The ORC JNI wrapper is currently crashing on these lines:
> >
> https://github.com/apache/arrow/blob/master/cpp/src/jni/orc/jni_wrapper.cpp#L279-L281
> >
> > because in C++, certain buffers can be omitted by passing a null
> > popinter (specifically the null bitmap, if there are no nulls).
> > Therefore `buffer` in the lines above is a null pointer.
> >
> > (I tried replacing the null buffer with a 0-byte buffer: it crashes
> > further down the road...)
> >
> > Since this code has been there since ARROW-4714 was committed, my
> > intuition is that the JNI ORC wrapper was only exercised in very
> > specific use cases where C++ buffers are never null.
> >
> > My opinion is therefore that the ORC JNI tests should be ignored for
> > this release, and fixed later by some motivated developer.
> Sounds good to me. Anyone knows a way to skip certain tests with
> maven during maven release? Commenting?
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 16/04/2020 à 02:17, Krisztián Szűcs a écrit :
> > > Hi,
> > >
> > > We've merged the last required pull requests later today[/yesterday],
> > > so I started to cut RC0.
> > > The release process doesn't go smoothly, among other smaller problems
> > > I discovered a crash with the ORC Java JNI bindings (local error [1]),
> > > turned out that we don't run the orc-jni tests on the CI. I put up a PR
> > > to enable them [2], it has not reproduced the exact issue yet.
> > >
> > > Any help from the JNI developers would be appreciated. I can also cut
> > > RC0 with JNI disabled.
> > >
> > > [1] https://gist.github.com/kszucs/67205eda6cd19e3cd08c86894f5b4c2d
> > > [2] https://github.com/apache/arrow/pull/6953
> > >
> > > Regards, Krisztian
> > >
>
Re: ORC JNI wrapper bugs [Re: 0.17 release procedure]
Posted by Krisztián Szűcs <sz...@gmail.com>.
On Thu, Apr 16, 2020 at 11:47 AM Antoine Pitrou <an...@python.org> wrote:
>
>
> The ORC JNI wrapper is currently crashing on these lines:
> https://github.com/apache/arrow/blob/master/cpp/src/jni/orc/jni_wrapper.cpp#L279-L281
>
> because in C++, certain buffers can be omitted by passing a null
> popinter (specifically the null bitmap, if there are no nulls).
> Therefore `buffer` in the lines above is a null pointer.
>
> (I tried replacing the null buffer with a 0-byte buffer: it crashes
> further down the road...)
>
> Since this code has been there since ARROW-4714 was committed, my
> intuition is that the JNI ORC wrapper was only exercised in very
> specific use cases where C++ buffers are never null.
>
> My opinion is therefore that the ORC JNI tests should be ignored for
> this release, and fixed later by some motivated developer.
Sounds good to me. Anyone knows a way to skip certain tests with
maven during maven release? Commenting?
>
> Regards
>
> Antoine.
>
>
> Le 16/04/2020 à 02:17, Krisztián Szűcs a écrit :
> > Hi,
> >
> > We've merged the last required pull requests later today[/yesterday],
> > so I started to cut RC0.
> > The release process doesn't go smoothly, among other smaller problems
> > I discovered a crash with the ORC Java JNI bindings (local error [1]),
> > turned out that we don't run the orc-jni tests on the CI. I put up a PR
> > to enable them [2], it has not reproduced the exact issue yet.
> >
> > Any help from the JNI developers would be appreciated. I can also cut
> > RC0 with JNI disabled.
> >
> > [1] https://gist.github.com/kszucs/67205eda6cd19e3cd08c86894f5b4c2d
> > [2] https://github.com/apache/arrow/pull/6953
> >
> > Regards, Krisztian
> >