You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Dan Washusen <da...@reactive.org> on 2010/02/24 08:17:17 UTC
Question regarding PerformanceEvaluation.RandomSeekScanTest
performance.
Hey All,
While testing the recent MemStoreScanner slowness I noticed that each scan
in the randomSeekScan test takes about 19 seconds to complete. The scan in
question provides a startRow and a WhileMatchFilter containing a PageFilter
that asks for 120 rows. I would have expected this scan to return in
roughly the same amount of time as a scan that specifies a startRow and
stopRow that spans a similar number of rows.
Before I start investigating further I was wondering if anyone had any
thoughts?
Cheers,
Dan
p.s. Running the test on the latest 0.20 code...
Re: Question regarding PerformanceEvaluation.RandomSeekScanTest
performance.
Posted by Ryan Rawson <ry...@gmail.com>.
Wow that is brutal.
Are you able to make a patch? How could we add a test to protect
ourselves from these kinds of things?
On Tue, Feb 23, 2010 at 11:56 PM, Dan Washusen <da...@reactive.org> wrote:
> It's an issue with WhileMatchFilter. I've created issue HBASE-2258...
>
> On 24 February 2010 18:25, Ryan Rawson <ry...@gmail.com> wrote:
>
>> That is very interesting, there are definitely some hidden performance
>> landmines in the filter code... you have clearly found one.
>>
>> strange in that while match filter is just a thin wrapper.
>>
>>
>>
>> On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <da...@reactive.org> wrote:
>> > OK, a bit more info; removing the WhileMatchFilter drops the scans down
>> to
>> > 25ms each.
>> >
>> > Updated scan looks like:
>> >
>> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>> >>
>> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>> >>
>> >> scan.setFilter(new PageFilter(120));
>> >>
>> >>
>> > The slow scan uses:
>> >
>> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>> >>
>> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>> >>
>> >> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
>> >>
>> >>
>> >
>> >
>> > On 24 February 2010 18:17, Dan Washusen <da...@reactive.org> wrote:
>> >
>> >> Hey All,
>> >> While testing the recent MemStoreScanner slowness I noticed that each
>> scan
>> >> in the randomSeekScan test takes about 19 seconds to complete. The scan
>> in
>> >> question provides a startRow and a WhileMatchFilter containing a
>> PageFilter
>> >> that asks for 120 rows. I would have expected this scan to return in
>> >> roughly the same amount of time as a scan that specifies a startRow and
>> >> stopRow that spans a similar number of rows.
>> >>
>> >> Before I start investigating further I was wondering if anyone had any
>> >> thoughts?
>> >>
>> >> Cheers,
>> >> Dan
>> >>
>> >> p.s. Running the test on the latest 0.20 code...
>> >>
>> >
>>
>
Re: Question regarding PerformanceEvaluation.RandomSeekScanTest
performance.
Posted by Dan Washusen <da...@reactive.org>.
It's an issue with WhileMatchFilter. I've created issue HBASE-2258...
On 24 February 2010 18:25, Ryan Rawson <ry...@gmail.com> wrote:
> That is very interesting, there are definitely some hidden performance
> landmines in the filter code... you have clearly found one.
>
> strange in that while match filter is just a thin wrapper.
>
>
>
> On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <da...@reactive.org> wrote:
> > OK, a bit more info; removing the WhileMatchFilter drops the scans down
> to
> > 25ms each.
> >
> > Updated scan looks like:
> >
> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
> >>
> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
> >>
> >> scan.setFilter(new PageFilter(120));
> >>
> >>
> > The slow scan uses:
> >
> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
> >>
> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
> >>
> >> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
> >>
> >>
> >
> >
> > On 24 February 2010 18:17, Dan Washusen <da...@reactive.org> wrote:
> >
> >> Hey All,
> >> While testing the recent MemStoreScanner slowness I noticed that each
> scan
> >> in the randomSeekScan test takes about 19 seconds to complete. The scan
> in
> >> question provides a startRow and a WhileMatchFilter containing a
> PageFilter
> >> that asks for 120 rows. I would have expected this scan to return in
> >> roughly the same amount of time as a scan that specifies a startRow and
> >> stopRow that spans a similar number of rows.
> >>
> >> Before I start investigating further I was wondering if anyone had any
> >> thoughts?
> >>
> >> Cheers,
> >> Dan
> >>
> >> p.s. Running the test on the latest 0.20 code...
> >>
> >
>
Re: Question regarding PerformanceEvaluation.RandomSeekScanTest
performance.
Posted by Ryan Rawson <ry...@gmail.com>.
That is very interesting, there are definitely some hidden performance
landmines in the filter code... you have clearly found one.
strange in that while match filter is just a thin wrapper.
On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <da...@reactive.org> wrote:
> OK, a bit more info; removing the WhileMatchFilter drops the scans down to
> 25ms each.
>
> Updated scan looks like:
>
>> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>>
>> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>>
>> scan.setFilter(new PageFilter(120));
>>
>>
> The slow scan uses:
>
>> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>>
>> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>>
>> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
>>
>>
>
>
> On 24 February 2010 18:17, Dan Washusen <da...@reactive.org> wrote:
>
>> Hey All,
>> While testing the recent MemStoreScanner slowness I noticed that each scan
>> in the randomSeekScan test takes about 19 seconds to complete. The scan in
>> question provides a startRow and a WhileMatchFilter containing a PageFilter
>> that asks for 120 rows. I would have expected this scan to return in
>> roughly the same amount of time as a scan that specifies a startRow and
>> stopRow that spans a similar number of rows.
>>
>> Before I start investigating further I was wondering if anyone had any
>> thoughts?
>>
>> Cheers,
>> Dan
>>
>> p.s. Running the test on the latest 0.20 code...
>>
>
Re: Question regarding PerformanceEvaluation.RandomSeekScanTest
performance.
Posted by Dan Washusen <da...@reactive.org>.
OK, a bit more info; removing the WhileMatchFilter drops the scans down to
25ms each.
Updated scan looks like:
> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>
> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>
> scan.setFilter(new PageFilter(120));
>
>
The slow scan uses:
> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>
> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>
> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
>
>
On 24 February 2010 18:17, Dan Washusen <da...@reactive.org> wrote:
> Hey All,
> While testing the recent MemStoreScanner slowness I noticed that each scan
> in the randomSeekScan test takes about 19 seconds to complete. The scan in
> question provides a startRow and a WhileMatchFilter containing a PageFilter
> that asks for 120 rows. I would have expected this scan to return in
> roughly the same amount of time as a scan that specifies a startRow and
> stopRow that spans a similar number of rows.
>
> Before I start investigating further I was wondering if anyone had any
> thoughts?
>
> Cheers,
> Dan
>
> p.s. Running the test on the latest 0.20 code...
>