You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Alex Newman (JIRA)" <ji...@apache.org> on 2012/10/02 01:41:12 UTC
[jira] [Created] (HBASE-6912) Filters are not properly applied in
certain cases
Alex Newman created HBASE-6912:
----------------------------------
Summary: Filters are not properly applied in certain cases
Key: HBASE-6912
URL: https://issues.apache.org/jira/browse/HBASE-6912
Project: HBase
Issue Type: Bug
Reporter: Alex Newman
Steps to reproduce:
Create a table, load data into it. Flush the table.
Do a scan with
1. Some filter which should not match the first entry in the scan
2. Where one specifies a family and column.
You will notice that the first entry is returned even though it doesn't match the filter.
It looks like the when the first KeyValue of a scan in the column from the point of view of the code
HRegion.java
} else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
Is generated by
public static KeyValue createLastOnRow(final byte [] row,
final int roffset, final int rlength, final byte [] family,
final int foffset, final int flength, final byte [] qualifier,
final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
So it is always internal from that point of the code.
Only later from within
StoreScanner.java
public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
....
LOOP: while((kv = this.heap.peek()) != null) {
( The second time through)
Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: (was: 6912-0.96.txt)
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6912-0.94.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469897#comment-13469897 ]
Hudson commented on HBASE-6912:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #8 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/8/])
HBASE-6912 Filters are not properly applied in certain cases (revert HBASE-6562) (Revision 1393854)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468729#comment-13468729 ]
Lars Hofhansl commented on HBASE-6912:
--------------------------------------
Yeah, I modified the test and integrated it into TestFromClientSide.
OK... So. I'll reopen HBASE-6562, revert that change, and move to 0.94.3 or even 0.96.
As part of this jira I'll just commit Alex' test.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468439#comment-13468439 ]
ramkrishna.s.vasudevan commented on HBASE-6912:
-----------------------------------------------
Lazy seeking scenarios will be broken right Lars?
bq.I have a patch, which fixes RowFilter.
Can you upload this patch?
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467332#comment-13467332 ]
Alex Newman commented on HBASE-6912:
------------------------------------
[~lhofhansl] I created a new jira based on your feedback.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> Is generated by
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469091#comment-13469091 ]
Hudson commented on HBASE-6912:
-------------------------------
Integrated in HBase-0.94 #504 (See [https://builds.apache.org/job/HBase-0.94/504/])
HBASE-6912 Filters are not properly applied in certain cases (revert HBASE-6562) (Revision 1393854)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468595#comment-13468595 ]
Ted Yu commented on HBASE-6912:
-------------------------------
+1 to Lars' plan.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl reassigned HBASE-6912:
------------------------------------
Assignee: Lars Hofhansl
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469082#comment-13469082 ]
Hudson commented on HBASE-6912:
-------------------------------
Integrated in HBase-TRUNK #3419 (See [https://builds.apache.org/job/HBase-TRUNK/3419/])
HBASE-6912 Filters are not properly applied in certain cases (revert HBASE-6562) (Revision 1393853)
Result = SUCCESS
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469303#comment-13469303 ]
Hudson commented on HBASE-6912:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #207 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/207/])
HBASE-6912 Filters are not properly applied in certain cases (revert HBASE-6562) (Revision 1393853)
Result = SUCCESS
larsh :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468663#comment-13468663 ]
ramkrishna.s.vasudevan commented on HBASE-6912:
-----------------------------------------------
Ok Lars.. Sounds good. Alex's test needs little changes like start mini cluster and stop minicluster.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to 0.94 and 0.96.
Thanks for finding the issue and providing a test case, Alex!
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-6912:
-------------------------------
Attachment: minimalTest.java
This triggers the problem. If you get rid of the flush the bug goes away. Only tested on a local filesystem with a running hbase.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> Is generated by
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Fix Version/s: 0.96.0
0.94.3
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-6912:
--------------------------
Description:
Steps to reproduce:
Create a table, load data into it. Flush the table.
Do a scan with
1. Some filter which should not match the first entry in the scan
2. Where one specifies a family and column.
You will notice that the first entry is returned even though it doesn't match the filter.
It looks like the when the first KeyValue of a scan in the column from the point of view of the code
HRegion.java
{code}
} else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
{code}
Is generated by
{code}
public static KeyValue createLastOnRow(final byte [] row,
final int roffset, final int rlength, final byte [] family,
final int foffset, final int flength, final byte [] qualifier,
final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
{code}
So it is always internal from that point of the code.
Only later from within
StoreScanner.java
{code}
public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
....
LOOP: while((kv = this.heap.peek()) != null) {
{code}
( The second time through)
Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
was:
Steps to reproduce:
Create a table, load data into it. Flush the table.
Do a scan with
1. Some filter which should not match the first entry in the scan
2. Where one specifies a family and column.
You will notice that the first entry is returned even though it doesn't match the filter.
It looks like the when the first KeyValue of a scan in the column from the point of view of the code
HRegion.java
} else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
Is generated by
(THE FIRST ENTRY IS STILL INTERNAL AT THIS POINT)
public static KeyValue createLastOnRow(final byte [] row,
final int roffset, final int rlength, final byte [] family,
final int foffset, final int flength, final byte [] qualifier,
final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
So it is always internal from that point of the code.
Only later from within
StoreScanner.java
public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
....
LOOP: while((kv = this.heap.peek()) != null) {
( The second time through)
Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
Affects Version/s: 0.94.1
Summary: Filters are not properly applied in certain cases (was: Filters are not properly applied to scans, to the first entry in the scan. )
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468789#comment-13468789 ]
Hadoop QA commented on HBASE-6912:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12547577/6912-0.96.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 83 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2994//console
This message is automatically generated.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467946#comment-13467946 ]
Lars Hofhansl commented on HBASE-6912:
--------------------------------------
I have a patch, which fixes RowFilter. But I do not like it.
Each "real" KV should through filterRowKey before it goes through filterKeyValue. That seems to be broken in some cases (which has to do with the lazy seeking we're doing now).
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467341#comment-13467341 ]
Lars Hofhansl commented on HBASE-6912:
--------------------------------------
Thanks Alex.
RowFilter seems to be wrong here. I checked all other filters and they all do the right thing (either only have filterRowKey, or filterKeyValue, or perform the same check in both methods)... Except RandomRowFilter, which does the same thing, but there's it's probably OK.
Before HBASE-6562 user code (in Filters) could actually see the internal KVs, which is definitely wrong.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468593#comment-13468593 ]
Lars Hofhansl commented on HBASE-6912:
--------------------------------------
I am inclined to revert HBASE-6562 for now, and add Alex' test at the same time to guard against this in the future.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: 6912-0.96.txt
Same for 0.96.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6912-0.94.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: 6912-0.94.txt
Right patch.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6912-0.94.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Fix Version/s: (was: 0.94.3)
0.94.2
Status: Patch Available (was: Open)
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467947#comment-13467947 ]
Alex Newman commented on HBASE-6912:
------------------------------------
Agreed, it would be unfortunate if we attempted, for instance to apply timestamp filters at that part of the code.
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied to
scans, to the first entry in the scan.
Posted by "Alex Newman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-6912:
-------------------------------
Description:
Steps to reproduce:
Create a table, load data into it. Flush the table.
Do a scan with
1. Some filter which should not match the first entry in the scan
2. Where one specifies a family and column.
You will notice that the first entry is returned even though it doesn't match the filter.
It looks like the when the first KeyValue of a scan in the column from the point of view of the code
HRegion.java
} else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
Is generated by
(THE FIRST ENTRY IS STILL INTERNAL AT THIS POINT)
public static KeyValue createLastOnRow(final byte [] row,
final int roffset, final int rlength, final byte [] family,
final int foffset, final int flength, final byte [] qualifier,
final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
So it is always internal from that point of the code.
Only later from within
StoreScanner.java
public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
....
LOOP: while((kv = this.heap.peek()) != null) {
( The second time through)
Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
was:
Steps to reproduce:
Create a table, load data into it. Flush the table.
Do a scan with
1. Some filter which should not match the first entry in the scan
2. Where one specifies a family and column.
You will notice that the first entry is returned even though it doesn't match the filter.
It looks like the when the first KeyValue of a scan in the column from the point of view of the code
HRegion.java
} else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
Is generated by
public static KeyValue createLastOnRow(final byte [] row,
final int roffset, final int rlength, final byte [] family,
final int foffset, final int flength, final byte [] qualifier,
final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
So it is always internal from that point of the code.
Only later from within
StoreScanner.java
public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
....
LOOP: while((kv = this.heap.peek()) != null) {
( The second time through)
Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
Summary: Filters are not properly applied to scans, to the first entry in the scan. (was: Filters are not properly applied in certain cases)
> Filters are not properly applied to scans, to the first entry in the scan.
> ---------------------------------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Attachments: minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> Is generated by
> (THE FIRST ENTRY IS STILL INTERNAL AT THIS POINT)
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: (was: 6912-0.94.txt)
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6912-0.94.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469940#comment-13469940 ]
Hudson commented on HBASE-6912:
-------------------------------
Integrated in HBase-0.94-security #59 (See [https://builds.apache.org/job/HBase-0.94-security/59/])
HBASE-6912 Filters are not properly applied in certain cases (revert HBASE-6562) (Revision 1393854)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Assignee: Lars Hofhansl
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: 6912-0.94.txt
0.94 revert of HBASE-6562, including Alex' test.
(Leaving isInternal() on KeyValue, though, because that is useful to have)
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 6912-0.94.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in
certain cases
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6912:
---------------------------------
Attachment: 6912-0.96.txt
real 0.96 version
> Filters are not properly applied in certain cases
> -------------------------------------------------
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.1
> Reporter: Alex Newman
> Fix For: 0.94.2, 0.96.0
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match the filter.
> It looks like the when the first KeyValue of a scan in the column from the point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset, rlength, family, foffset, flength, qualifier, qoffset, qlength, HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List<KeyValue> outResult, int limit, String metric) throws IOException {
> ....
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira