You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@orc.apache.org by "Andrew S." <lu...@yahoo.com> on 2019/11/08 19:59:32 UTC
Possible bug in ORC C++ seekToRow?
Hi,I'm starting to work with the ORC C++ API for a project I'm on.In a very basic test, I am seeing an assertion which I think is a bug. As I'm not sure where to report bugs (and also, as I'm just started looking at this API and maybe I'm misusing the API) I thought I'd post here.
It is easy to reproduce: Take the Filecontents.cpp program which is the source for orc-contents example application and add the following code right before the while loop that calls rowReader->next(*batch):
rowReader->seekToRow(1000);
Then, using the example file TestOrcFile.testSeek.orc as input to orc-contents, this will crash with an assertion because of dereferencing an iterator that's pointing to end(). Using other example data files works fine in this case.
Looking at PositionProvider::PositionProvider in InputStream.cc, it's assigning posns.begin() to the position variable. In this specific case for some of these assignments, the posns list is empty, and therefore the position variable ends up with an iterator that's pointing to end(). After that when it calls seekToRowGroup, it eventually gets to SeekableFileInputStream::seek() where it tries to call next() on that PositionProvider, tries to dereference the iterator and gives the assertion.
I'm on Win10 using VC 2017.
Any idea if this is a bug, or am I missing something?
Re: Possible bug in ORC C++ seekToRow?
Posted by Owen O'Malley <ow...@gmail.com>.
The bug that I know is in the Java writer. I haven’t verified if the C++ writer has the same bug. There will be a patch for the java and c++ readers to work around the problem.
.. Owen
> On Nov 11, 2019, at 05:45, Andrew Shakinovsky <An...@sas.com> wrote:
>
>
> Thanks, Owen,
> I looked at the ticket, and in it you describe the issue in the context of Java. When these types of issues are resolved, are they typically resolved in parallel in the Java and C++ API’s?
>
>
> From: Owen O'Malley <ow...@gmail.com>
> Sent: Friday, November 8, 2019 6:24 PM
> To: user@orc.apache.org; Andrew S. <lu...@yahoo.com>
> Subject: Re: Possible bug in ORC C++ seekToRow?
>
> Ok, I've been able to reproduce this. I've filed https://issues.apache.org/jira/browse/ORC-569 .
>
> .. Owen
>
> On Fri, Nov 8, 2019 at 1:30 PM Owen O'Malley <ow...@gmail.com> wrote:
> Andrew,
> For bugs or feature requests, please file a jira: http://orc.apache.org/jira
>
> If the positions list is empty, that is a bug. Can you describe how you generated the ORC file? Which version of the software was used?
>
> Thanks,
> Owen
>
> On Fri, Nov 8, 2019 at 11:59 AM Andrew S. <lu...@yahoo.com> wrote:
> Hi,
> I'm starting to work with the ORC C++ API for a project I'm on.
> In a very basic test, I am seeing an assertion which I think is a bug. As I'm not sure where to report bugs (and also, as I'm just started looking at this API and maybe I'm misusing the API) I thought I'd post here.
>
> It is easy to reproduce: Take the Filecontents.cpp program which is the source for orc-contents example application and add the following code right before the while loop that calls rowReader->next(*batch):
>
> rowReader->seekToRow(1000);
>
> Then, using the example file TestOrcFile.testSeek.orc as input to orc-contents, this will crash with an assertion because of dereferencing an iterator that's pointing to end(). Using other example data files works fine in this case.
>
> Looking at PositionProvider::PositionProvider in InputStream.cc, it's assigning posns.begin() to the position variable. In this specific case for some of these assignments, the posns list is empty, and therefore the position variable ends up with an iterator that's pointing to end(). After that when it calls seekToRowGroup, it eventually gets to SeekableFileInputStream::seek() where it tries to call next() on that PositionProvider, tries to dereference the iterator and gives the assertion.
>
> I'm on Win10 using VC 2017.
>
> Any idea if this is a bug, or am I missing something?
>
RE: Possible bug in ORC C++ seekToRow?
Posted by Andrew Shakinovsky <An...@sas.com>.
Thanks, Owen,
I looked at the ticket, and in it you describe the issue in the context of Java. When these types of issues are resolved, are they typically resolved in parallel in the Java and C++ API’s?
From: Owen O'Malley <ow...@gmail.com>
Sent: Friday, November 8, 2019 6:24 PM
To: user@orc.apache.org; Andrew S. <lu...@yahoo.com>
Subject: Re: Possible bug in ORC C++ seekToRow?
Ok, I've been able to reproduce this. I've filed https://issues.apache.org/jira/browse/ORC-569<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FORC-569&data=02%7C01%7CAndrew.Shakinovsky%40sas.com%7C8f1a1361ca4c497d4fa508d764a2c089%7Cb1c14d5c362545b3a4309552373a0c2f%7C0%7C0%7C637088522497371176&sdata=3oobeGIZSSXjHQ1acECdex7%2FNAasEueyKBr6X797X5Y%3D&reserved=0> .
.. Owen
On Fri, Nov 8, 2019 at 1:30 PM Owen O'Malley <ow...@gmail.com>> wrote:
Andrew,
For bugs or feature requests, please file a jira: http://orc.apache.org/jira<https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Forc.apache.org%2Fjira&data=02%7C01%7CAndrew.Shakinovsky%40sas.com%7C8f1a1361ca4c497d4fa508d764a2c089%7Cb1c14d5c362545b3a4309552373a0c2f%7C0%7C0%7C637088522497371176&sdata=ufV8kzYR4fEjenA1OFohA0uKCB2ZClaUFJLeVs%2FC1eE%3D&reserved=0>
If the positions list is empty, that is a bug. Can you describe how you generated the ORC file? Which version of the software was used?
Thanks,
Owen
On Fri, Nov 8, 2019 at 11:59 AM Andrew S. <lu...@yahoo.com>> wrote:
Hi,
I'm starting to work with the ORC C++ API for a project I'm on.
In a very basic test, I am seeing an assertion which I think is a bug. As I'm not sure where to report bugs (and also, as I'm just started looking at this API and maybe I'm misusing the API) I thought I'd post here.
It is easy to reproduce: Take the Filecontents.cpp program which is the source for orc-contents example application and add the following code right before the while loop that calls rowReader->next(*batch):
rowReader->seekToRow(1000);
Then, using the example file TestOrcFile.testSeek.orc as input to orc-contents, this will crash with an assertion because of dereferencing an iterator that's pointing to end(). Using other example data files works fine in this case.
Looking at PositionProvider::PositionProvider in InputStream.cc, it's assigning posns.begin() to the position variable. In this specific case for some of these assignments, the posns list is empty, and therefore the position variable ends up with an iterator that's pointing to end(). After that when it calls seekToRowGroup, it eventually gets to SeekableFileInputStream::seek() where it tries to call next() on that PositionProvider, tries to dereference the iterator and gives the assertion.
I'm on Win10 using VC 2017.
Any idea if this is a bug, or am I missing something?
Re: Possible bug in ORC C++ seekToRow?
Posted by Owen O'Malley <ow...@gmail.com>.
Ok, I've been able to reproduce this. I've filed
https://issues.apache.org/jira/browse/ORC-569 .
.. Owen
On Fri, Nov 8, 2019 at 1:30 PM Owen O'Malley <ow...@gmail.com> wrote:
> Andrew,
> For bugs or feature requests, please file a jira:
> http://orc.apache.org/jira
>
> If the positions list is empty, that is a bug. Can you describe how you
> generated the ORC file? Which version of the software was used?
>
> Thanks,
> Owen
>
> On Fri, Nov 8, 2019 at 11:59 AM Andrew S. <lu...@yahoo.com> wrote:
>
>> Hi,
>> I'm starting to work with the ORC C++ API for a project I'm on.
>> In a very basic test, I am seeing an assertion which I think is a bug. As
>> I'm not sure where to report bugs (and also, as I'm just started looking at
>> this API and maybe I'm misusing the API) I thought I'd post here.
>>
>> It is easy to reproduce: Take the Filecontents.cpp program which is the
>> source for orc-contents example application and add the following code
>> right before the while loop that calls rowReader->next(*batch):
>>
>> rowReader->seekToRow(1000);
>>
>> Then, using the example file TestOrcFile.testSeek.orc as input to
>> orc-contents, this will crash with an assertion because of dereferencing an
>> iterator that's pointing to end(). Using other example data files works
>> fine in this case.
>>
>> Looking at PositionProvider::PositionProvider in InputStream.cc, it's
>> assigning posns.begin() to the position variable. In this specific case for
>> some of these assignments, the posns list is empty, and therefore the
>> position variable ends up with an iterator that's pointing to end(). After
>> that when it calls seekToRowGroup, it eventually gets to
>> SeekableFileInputStream::seek() where it tries to call next() on that
>> PositionProvider, tries to dereference the iterator and gives the assertion.
>>
>> I'm on Win10 using VC 2017.
>>
>> Any idea if this is a bug, or am I missing something?
>>
>>
Re: Possible bug in ORC C++ seekToRow?
Posted by Owen O'Malley <ow...@gmail.com>.
Andrew,
For bugs or feature requests, please file a jira:
http://orc.apache.org/jira
If the positions list is empty, that is a bug. Can you describe how you
generated the ORC file? Which version of the software was used?
Thanks,
Owen
On Fri, Nov 8, 2019 at 11:59 AM Andrew S. <lu...@yahoo.com> wrote:
> Hi,
> I'm starting to work with the ORC C++ API for a project I'm on.
> In a very basic test, I am seeing an assertion which I think is a bug. As
> I'm not sure where to report bugs (and also, as I'm just started looking at
> this API and maybe I'm misusing the API) I thought I'd post here.
>
> It is easy to reproduce: Take the Filecontents.cpp program which is the
> source for orc-contents example application and add the following code
> right before the while loop that calls rowReader->next(*batch):
>
> rowReader->seekToRow(1000);
>
> Then, using the example file TestOrcFile.testSeek.orc as input to
> orc-contents, this will crash with an assertion because of dereferencing an
> iterator that's pointing to end(). Using other example data files works
> fine in this case.
>
> Looking at PositionProvider::PositionProvider in InputStream.cc, it's
> assigning posns.begin() to the position variable. In this specific case for
> some of these assignments, the posns list is empty, and therefore the
> position variable ends up with an iterator that's pointing to end(). After
> that when it calls seekToRowGroup, it eventually gets to
> SeekableFileInputStream::seek() where it tries to call next() on that
> PositionProvider, tries to dereference the iterator and gives the assertion.
>
> I'm on Win10 using VC 2017.
>
> Any idea if this is a bug, or am I missing something?
>
>