You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jean-Marc Spaggiari <je...@spaggiari.org> on 2013/02/27 13:32:30 UTC
HFilePerformanceEvaluation: Nothing follows logic
Hi,
In HFilePerformanceEvaluation, there is 2 places where we are doing
scanner.next() but still trying to read the key and/or the value. This
is failing. I'm wondering if there is any logic behind that or if I
can fix it.
JM
for (int ii = 0; ii < 30; ii++) {
if (!scanner.next()) {
System.out.println("NOTHING FOLLOWS");
}
scanner.getKey();
scanner.getValue();
}
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Ok. I have done some tests.
And never came above a 0.0001% difference between the number of calls
expected to be done vs the number of calls really done.
So I don't think we need to worry about that and simply return when
scanner.next returns false.
JM
2013/2/27 Jean-Marc Spaggiari <je...@spaggiari.org>:
> Thanks for your feedback.
>
> I will modify the patch to go with the 3rd option instead of the 2nd one.
>
> JM
>
> 2013/2/27 Ted <yu...@gmail.com>:
>> Thanks for your quick action.
>>
>> I think the third option is better - we get performance comparison without repeating the test.
>>
>> Cheers
>>
>> On Feb 27, 2013, at 5:18 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>>
>>> Just opened HBASE-7953. Can you please take a look at it?
>>>
>>> There was 3 ways to fix that.
>>>
>>> 1) Simply exit the loop when scanner.next() return false
>>> 2) Exit the loop and restart the test
>>> 3) Count the number of scanner.next() effectively done and measure
>>> performances based on that and not on the ROW_COUNT value.
>>>
>>> I chose the 2nd one, but maybe there is some preferences for the 3rd one?
>>>
>>> JM
>>>
>>> 2013/2/27 Jean-Marc Spaggiari <je...@spaggiari.org>:
>>>> Yep, I will open one and submit (and not commit ;) ) a patch.
>>>>
>>>> Thanks.
>>>>
>>>> JM
>>>>
>>>> 2013/2/27 Ted <yu...@gmail.com>:
>>>>> Looks like a bug.
>>>>>
>>>>> Mind opening Jira ?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In HFilePerformanceEvaluation, there is 2 places where we are doing
>>>>>> scanner.next() but still trying to read the key and/or the value. This
>>>>>> is failing. I'm wondering if there is any logic behind that or if I
>>>>>> can fix it.
>>>>>>
>>>>>> JM
>>>>>>
>>>>>>
>>>>>> for (int ii = 0; ii < 30; ii++) {
>>>>>> if (!scanner.next()) {
>>>>>> System.out.println("NOTHING FOLLOWS");
>>>>>> }
>>>>>> scanner.getKey();
>>>>>> scanner.getValue();
>>>>>> }
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Thanks for your feedback.
I will modify the patch to go with the 3rd option instead of the 2nd one.
JM
2013/2/27 Ted <yu...@gmail.com>:
> Thanks for your quick action.
>
> I think the third option is better - we get performance comparison without repeating the test.
>
> Cheers
>
> On Feb 27, 2013, at 5:18 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>
>> Just opened HBASE-7953. Can you please take a look at it?
>>
>> There was 3 ways to fix that.
>>
>> 1) Simply exit the loop when scanner.next() return false
>> 2) Exit the loop and restart the test
>> 3) Count the number of scanner.next() effectively done and measure
>> performances based on that and not on the ROW_COUNT value.
>>
>> I chose the 2nd one, but maybe there is some preferences for the 3rd one?
>>
>> JM
>>
>> 2013/2/27 Jean-Marc Spaggiari <je...@spaggiari.org>:
>>> Yep, I will open one and submit (and not commit ;) ) a patch.
>>>
>>> Thanks.
>>>
>>> JM
>>>
>>> 2013/2/27 Ted <yu...@gmail.com>:
>>>> Looks like a bug.
>>>>
>>>> Mind opening Jira ?
>>>>
>>>> Thanks
>>>>
>>>> On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> In HFilePerformanceEvaluation, there is 2 places where we are doing
>>>>> scanner.next() but still trying to read the key and/or the value. This
>>>>> is failing. I'm wondering if there is any logic behind that or if I
>>>>> can fix it.
>>>>>
>>>>> JM
>>>>>
>>>>>
>>>>> for (int ii = 0; ii < 30; ii++) {
>>>>> if (!scanner.next()) {
>>>>> System.out.println("NOTHING FOLLOWS");
>>>>> }
>>>>> scanner.getKey();
>>>>> scanner.getValue();
>>>>> }
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Ted <yu...@gmail.com>.
Thanks for your quick action.
I think the third option is better - we get performance comparison without repeating the test.
Cheers
On Feb 27, 2013, at 5:18 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
> Just opened HBASE-7953. Can you please take a look at it?
>
> There was 3 ways to fix that.
>
> 1) Simply exit the loop when scanner.next() return false
> 2) Exit the loop and restart the test
> 3) Count the number of scanner.next() effectively done and measure
> performances based on that and not on the ROW_COUNT value.
>
> I chose the 2nd one, but maybe there is some preferences for the 3rd one?
>
> JM
>
> 2013/2/27 Jean-Marc Spaggiari <je...@spaggiari.org>:
>> Yep, I will open one and submit (and not commit ;) ) a patch.
>>
>> Thanks.
>>
>> JM
>>
>> 2013/2/27 Ted <yu...@gmail.com>:
>>> Looks like a bug.
>>>
>>> Mind opening Jira ?
>>>
>>> Thanks
>>>
>>> On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> In HFilePerformanceEvaluation, there is 2 places where we are doing
>>>> scanner.next() but still trying to read the key and/or the value. This
>>>> is failing. I'm wondering if there is any logic behind that or if I
>>>> can fix it.
>>>>
>>>> JM
>>>>
>>>>
>>>> for (int ii = 0; ii < 30; ii++) {
>>>> if (!scanner.next()) {
>>>> System.out.println("NOTHING FOLLOWS");
>>>> }
>>>> scanner.getKey();
>>>> scanner.getValue();
>>>> }
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Just opened HBASE-7953. Can you please take a look at it?
There was 3 ways to fix that.
1) Simply exit the loop when scanner.next() return false
2) Exit the loop and restart the test
3) Count the number of scanner.next() effectively done and measure
performances based on that and not on the ROW_COUNT value.
I chose the 2nd one, but maybe there is some preferences for the 3rd one?
JM
2013/2/27 Jean-Marc Spaggiari <je...@spaggiari.org>:
> Yep, I will open one and submit (and not commit ;) ) a patch.
>
> Thanks.
>
> JM
>
> 2013/2/27 Ted <yu...@gmail.com>:
>> Looks like a bug.
>>
>> Mind opening Jira ?
>>
>> Thanks
>>
>> On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>>
>>> Hi,
>>>
>>> In HFilePerformanceEvaluation, there is 2 places where we are doing
>>> scanner.next() but still trying to read the key and/or the value. This
>>> is failing. I'm wondering if there is any logic behind that or if I
>>> can fix it.
>>>
>>> JM
>>>
>>>
>>> for (int ii = 0; ii < 30; ii++) {
>>> if (!scanner.next()) {
>>> System.out.println("NOTHING FOLLOWS");
>>> }
>>> scanner.getKey();
>>> scanner.getValue();
>>> }
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Yep, I will open one and submit (and not commit ;) ) a patch.
Thanks.
JM
2013/2/27 Ted <yu...@gmail.com>:
> Looks like a bug.
>
> Mind opening Jira ?
>
> Thanks
>
> On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
>
>> Hi,
>>
>> In HFilePerformanceEvaluation, there is 2 places where we are doing
>> scanner.next() but still trying to read the key and/or the value. This
>> is failing. I'm wondering if there is any logic behind that or if I
>> can fix it.
>>
>> JM
>>
>>
>> for (int ii = 0; ii < 30; ii++) {
>> if (!scanner.next()) {
>> System.out.println("NOTHING FOLLOWS");
>> }
>> scanner.getKey();
>> scanner.getValue();
>> }
Re: HFilePerformanceEvaluation: Nothing follows logic
Posted by Ted <yu...@gmail.com>.
Looks like a bug.
Mind opening Jira ?
Thanks
On Feb 27, 2013, at 4:32 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
> Hi,
>
> In HFilePerformanceEvaluation, there is 2 places where we are doing
> scanner.next() but still trying to read the key and/or the value. This
> is failing. I'm wondering if there is any logic behind that or if I
> can fix it.
>
> JM
>
>
> for (int ii = 0; ii < 30; ii++) {
> if (!scanner.next()) {
> System.out.println("NOTHING FOLLOWS");
> }
> scanner.getKey();
> scanner.getValue();
> }