You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2012/08/14 02:11:38 UTC
[jira] [Created] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Lars Hofhansl created HBASE-6577:
------------------------------------
Summary: RegionScannerImpl.nextRow() should seek to next row
Key: HBASE-6577
URL: https://issues.apache.org/jira/browse/HBASE-6577
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.96.0, 0.94.2
RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Attachment: 6577-v3.txt
Something like this.
The "100" should probably be configurable.
This should take care of the case of a few version and the case of very many versions.
... let me know what you think.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435265#comment-13435265 ]
ramkrishna.s.vasudevan commented on HBASE-6577:
-----------------------------------------------
In one of our case we have added a hook there. Patch wise it is fine Lars.
also the comment over HRegion.nextInternal() says this.
bq.// 2. provide a hook to fast forward the row (used by subclasses)
You can commit this Lars. No problem. +1.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479302#comment-13479302 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
I think you had just deployed it and because of the way the thrift uses filters you ran into this issue. You said this on the mailing list:
{code}
We use thrift's scanOpenWithPrefix which does this:
Filter f = new WhileMatchFilter(
new PrefixFilter(getBytes(startAndPrefix)));
So that might just be it.
{code}
No hurry, I am too scared of this to want to commit this any time soon :)
(I did a lot of testing beforehand and I did not see the issue you reported).
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435426#comment-13435426 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-0.94 #395 (See [https://builds.apache.org/job/HBase-0.94/395/])
HBASE-6577 RegionScannerImpl.nextRow() should seek to next row (Revision 1373533)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Fix Version/s: 0.96.0
0.94.3
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440847#comment-13440847 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
Reverted from 0.94 and 0.96
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Attachment: 6577-0.94.txt
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Status: Patch Available (was: Open)
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Status: Patch Available (was: Reopened)
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Status: Open (was: Patch Available)
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477458#comment-13477458 ]
Hadoop QA commented on HBASE-6577:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549397/6577-v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3059//console
This message is automatically generated.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440843#comment-13440843 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
>From Elliot:
"IPC Server handler 43 on 10304" daemon prio=10 tid=0x00007f16b8b1f000 nid=0x6414 runnable [0x00007f16b47c6000]
java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.hbase.KeyValue.createFirstOnRowColTS(KeyValue.java:1893)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:310)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:297)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:256)
at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:522)
- locked <0x00000006cec5bd88> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54)
at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.requestSeek(NonLazyKeyValueScanner.java:38)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:297)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:256)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRow(HRegion.java:3508)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3456)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3406)
- locked <0x00000006cb842910> (a org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3423)
- locked <0x00000006cb842910> (a org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2393)
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1389)
Looks like this can cause busy waiting somehow. Until this interaction is understood, I should revert the patch.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433898#comment-13433898 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
@Ram: I see.
I think filters and coprocessors should be kept separately. There are many places where filters fast forward and if we decorate all those points with a CP hooks we'd slow things down again.
If one needs to hook into fast forward seeking with a CP hook I think one should use the reseek that you added RegionScanner from within a coprocessor instead of a filter.
(I hope I unserstood the comment correctly).
So from that viewpoint I would find it weird to place a CP hook inside of filter fast forward optimization code.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479372#comment-13479372 ]
Jean-Daniel Cryans commented on HBASE-6577:
-------------------------------------------
Yeah I do remember that it's a thrift issue but we have >200 tables and 3 clusters that get reads through Thrift :(
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440864#comment-13440864 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
The jstack indicates that the former call causes the problem. This is quite strange.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478515#comment-13478515 ]
Jean-Daniel Cryans commented on HBASE-6577:
-------------------------------------------
bq. Jean-Daniel Cryans Would you be able the recreate the issue you saw with the initial version of this patch in production?
Is there anything in particular I should look for?
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435256#comment-13435256 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
@Ram?
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435317#comment-13435317 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
TestMasterNoCluster passes locally.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478539#comment-13478539 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
@J-D: with the initial version of this patch you encountered this problem on your production cluster:
https://issues.apache.org/jira/browse/HBASE-6577?focusedCommentId=13440843&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13440843
And
https://issues.apache.org/jira/browse/HBASE-6577?focusedCommentId=13440924&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13440924
Just wanna make sure this does not happen again with this patch.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440855#comment-13440855 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
There are two calls to nextRow in nextInternal. One is the result of filter the other is just to reset the filters. In the 2nd case we shouldn't seek.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433910#comment-13433910 ]
Hadoop QA commented on HBASE-6577:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12540810/6577.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 9 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterNoCluster
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2561//console
This message is automatically generated.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Attachment: 6577.txt
As simple as this. Just seek to the end of the row first, then iterate to the beginning of the next row. In wide rows KVs with many versions when used with filters this should be a big win.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6577) RegionScannerImpl.nextRow()
should seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433768#comment-13433768 ]
Lars Hofhansl edited comment on HBASE-6577 at 8/14/12 11:28 AM:
----------------------------------------------------------------
As simple as this. Just seek to the end of the row first, then iterate to the beginning of the next row. In wide rows or KVs with many versions when used with filters this should be a big win.
was (Author: lhofhansl):
As simple as this. Just seek to the end of the row first, then iterate to the beginning of the next row. In wide rows KVs with many versions when used with filters this should be a big win.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Fix Version/s: (was: 0.94.3)
I tried to reproduce the issue on the mailing (using a PrefixFilter), but I couldn't. Probably too risky at this point for 0.94.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl resolved HBASE-6577.
----------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to 0.94 and 0.96. Thanks for checking Ram!
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479282#comment-13479282 ]
Jean-Daniel Cryans commented on HBASE-6577:
-------------------------------------------
Ah sorry I totally missed that you posted a new patch.
So I tried to remember exactly how we got the issue (the exact cluster, which table was doing that, etc) and it's fuzzy for both [~eclark] and I. This week wouldn't definitely not be a good week to break things tho, so I currently don't feel like deploying it in the wild.
I can try to see if it's possible put it on a RS that has a copy of the data and run those filters on it but I don't have time for that right now.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6577:
---------------------------------
Attachment: 6577-v2.txt
Final 0.96 patch
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434686#comment-13434686 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
@Ram: Do you think this patch is incorrect? Or would add something else?
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435407#comment-13435407 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-TRUNK #3219 (See [https://builds.apache.org/job/HBase-TRUNK/3219/])
HBASE-6577 RegionScannerImpl.nextRow() should seek to next row (Revision 1373531)
Result = SUCCESS
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440896#comment-13440896 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-0.94 #425 (See [https://builds.apache.org/job/HBase-0.94/425/])
HBASE-6577 REVERT (Revision 1376804)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433842#comment-13433842 ]
Zhihong Ted Yu commented on HBASE-6577:
---------------------------------------
Is the removal of filterRow() intentional ?
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435316#comment-13435316 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
Thanks Ram... I'll change that comment as well, as now we are fast forwarding to the row.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477600#comment-13477600 ]
Anoop Sam John commented on HBASE-6577:
---------------------------------------
bq.In HRegionScannerImpl.nextRow(...) we try the current "naive" iteration for N KVs (let's say 100). If by then we have not reached the next row, we'll issue a direct seek.
That way if there are few version we avoid unnecessary seeks
Lars with HBASE-6032 in trunk, will it be a problem with calls to seek? I hope with this change now seek within same block will not have overhead. So may be we do not need configurable KV number(like 100).. Pls correct me if my understanding is wrong
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt, 6577-v3.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448336#comment-13448336 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #7 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/7/])
HBASE-6577 REVERT (Revision 1376804)
HBASE-6577 RegionScannerImpl.nextRow() should seek to next row (Revision 1373533)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433843#comment-13433843 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
Yeah, it wasn't used anywhere, should have said something in the jira comment.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439513#comment-13439513 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-0.94-security #48 (See [https://builds.apache.org/job/HBase-0.94-security/48/])
HBASE-6577 RegionScannerImpl.nextRow() should seek to next row (Revision 1373533)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl reopened HBASE-6577:
----------------------------------
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435643#comment-13435643 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #131 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/131/])
HBASE-6577 RegionScannerImpl.nextRow() should seek to next row (Revision 1373531)
Result = FAILURE
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440895#comment-13440895 ]
Hudson commented on HBASE-6577:
-------------------------------
Integrated in HBase-TRUNK #3262 (See [https://builds.apache.org/job/HBase-TRUNK/3262/])
HBASE-6577 REVERT (Revision 1376803)
Result = FAILURE
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440924#comment-13440924 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
J-D and Elliot probably saw this because they're using thrift's scanOpenWithPrefix which (according to J-D) does this:
{code}
Filter f = new WhileMatchFilter(
new PrefixFilter(getBytes(startAndPrefix)));
{code}
Interestingly TestFilter.testWhileMatchFilterWithFilterRowKey does exactly that, and it does not fail with this change.
So until I have more data my theory is this: If a filter filters very many rows in a narrow table, the seeking might actually be far more expensive then the dump forward iteration.
Also, I noticed that most Filters that filter in filterRowKey are smart enough to remember their decision and also filter all KV of the row they just filtered. So maybe this is not even problem. PrefixFilter is an exception, though.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477379#comment-13477379 ]
Lars Hofhansl commented on HBASE-6577:
--------------------------------------
This just came up on the mailing list again:
{code}
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1027)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:461)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:493)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:242)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:167)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:521)
- locked <0x000000059584fab8> (a
org.apache.hadoop.hbase.regionserver.StoreScanner)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:402)
- locked <0x000000059584fab8> (a
org.apache.hadoop.hbase.regionserver.StoreScanner)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRow(HRegion.java:3507)
at
...
{code}
zahoor mentioned there that his KVs have very many version 1500+.
Presumably each new column (likely) starts on a new (HBase) block, because of the many versions, which is why we see a lot of seeking.
I wonder whether a solution like the following would work:
In HRegionScannerImpl.nextRow(...) we try the current "naive" iteration for N KVs (let's say 100). If by then we have not reached the next row, we'll issue a direct seek.
That way if there are few version we avoid unnecessary seeks, but with many version we can seek past a lot of KVs (and thus also avoid unnecessary seeks).
I can make a patch for that.
[~jdcryans] Would you be able the recreate the issue you saw with the initial version of this patch in production?
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433891#comment-13433891 ]
ramkrishna.s.vasudevan commented on HBASE-6577:
-----------------------------------------------
@Lars
Pls take a look at HBASE-5664. Anoop wanted to give an hook over there whereever nextRow is getting called. Ideally in the hook what ever the patch does that is what one will be interested to do.
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-6577) RegionScannerImpl.nextRow() should
seek to next row
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl reopened HBASE-6577:
----------------------------------
> RegionScannerImpl.nextRow() should seek to next row
> ---------------------------------------------------
>
> Key: HBASE-6577
> URL: https://issues.apache.org/jira/browse/HBASE-6577
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6577-0.94.txt, 6577.txt, 6577-v2.txt
>
>
> RegionScannerImpl.nextRow() is called when a filter filters the entire row. In that case we should seek to the next row rather then iterating over all versions of all columns to get there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira