You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2016/01/22 04:19:39 UTC
[jira] [Commented] (PHOENIX-2613) Infinite loop in SkipScan when
seeking past null value in non leading primary key column
[ https://issues.apache.org/jira/browse/PHOENIX-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111846#comment-15111846 ]
Hudson commented on PHOENIX-2613:
---------------------------------
SUCCESS: Integrated in Phoenix-master #1098 (See [https://builds.apache.org/job/Phoenix-master/1098/])
PHOENIX-2613 Infinite loop in SkipScan when seeking past null value in (jtaylor: rev 462509625853bf85d0fdeb33fe3cf36bf5c94bec)
* phoenix-core/src/it/java/org/apache/phoenix/end2end/SkipScanQueryIT.java
* phoenix-core/src/test/java/org/apache/phoenix/filter/SkipScanFilterTest.java
* phoenix-core/src/main/java/org/apache/phoenix/filter/SkipScanFilter.java
> Infinite loop in SkipScan when seeking past null value in non leading primary key column
> ----------------------------------------------------------------------------------------
>
> Key: PHOENIX-2613
> URL: https://issues.apache.org/jira/browse/PHOENIX-2613
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.6.0
> Environment: HBase 0.98.6-cdh5.3.2, Phoenix 4.6.0-HBase-0.98
> Reporter: chenglei
> Assignee: James Taylor
> Fix For: 4.7.0
>
> Attachments: PHOENIX-2613.patch
>
>
> In pheonix 4.6,any column of multi-part primary key can be null.If a table has one row which has a column of multi-part primary key is null, and the java assertion is disable, when we do a query,the Skip Scan may cause RegionServer scan indefinite loop,just as the following unit test:
> {code:borderStyle=solid}
> @Test
> public void testNullInfiniteLoop() throws Exception
> {
> this.jdbcTemplate.update("drop table if exists NULL_TEST ");
>
> this.jdbcTemplate.update(
> "create table NULL_TEST"+
> "("+
> "CREATETIME VARCHAR,"+
> "ACCOUNTID VARCHAR,"+
> "SERVICENAME VARCHAR,"+
> "SPAN.APPID VARCHAR,"+
> "CONSTRAINT pk PRIMARY KEY(CREATETIME,ACCOUNTID,SERVICENAME)"+
> ")");
>
> this.jdbcTemplate.update("upsert into NULL_TEST(CREATETIME,SERVICENAME,SPAN.APPID) values('20160116141006','servlet','android')");
> this.jdbcTemplate.update("upsert into NULL_TEST(CREATETIME,ACCOUNTID,SERVICENAME,SPAN.APPID) values('20160116151006','2404787','jdbc','ios')");
> this.jdbcTemplate.queryForList("select * from NULL_TEST where CREATETIME>='20160116121006' and CREATETIME<='20160116181006' and ACCOUNTID='2404787'");
>
> }
> {code}
> As above unit test explained,we create a NULL_TEST table, and insert a row which ACCOUNTID column is null, When we do a query which condition is CREATETIME column and ACCOUNTID column, Phoenix will use SkipScanFilter to scan the table. Unfortunately,the query will run forever,can not return result.
> If we construct a SkipScanFilter using the above query condition,and we can see after the SkipScanFilter's filterKeyValue method is called on the KeyValue which rowKey is(the ACCOUNTID column is null) :
> {noformat}
> 20160116141006\\x00\\x00servlet
> {noformat}
> the SkipScanFilter's getNextCellHint method will return a erroneous nextHintCell value(2404787 is the ACCOUNTID):
> {noformat}
> 20160116141006\\x00\\x002404787
> {noformat}
> which should be:
> {noformat}
> 20160116141006\\x002404787
> {noformat}
> Just as the following unit test on the SkipScanFilter:
> {code:borderStyle=solid}
> @Test
> public void testNextCellHintError() throws Exception
> {
> List<List<KeyRange>> keyRanges=Arrays.asList( Collections.singletonList(KeyRange.getKeyRange(Bytes.toBytes("20160116121006"),true,Bytes.toBytes("20160116181006"),true)),
> Collections.singletonList(PChar.INSTANCE.getKeyRange(Bytes.toBytes("2404787"), true, Bytes.toBytes("2404787"), true)));
>
> RowKeySchemaBuilder rowKeySchemaBuilder = new RowKeySchemaBuilder(3);
>
> for(int i=1;i<=3;i++)
> {
> rowKeySchemaBuilder.addField(new PDatum() {
> @Override
> public boolean isNullable() {
> return true;
> }
> @Override
> public PDataType getDataType() {
> return PVarchar.INSTANCE;
> }
> @Override
> public Integer getMaxLength() {
> return null;
> }
> @Override
> public Integer getScale() {
> return null;
> }
> @Override
> public SortOrder getSortOrder() {
> return SortOrder.getDefault();
> }
> }, true, SortOrder.getDefault());
> }
>
> RowKeySchema rowKeySchema=rowKeySchemaBuilder.build();
>
> byte[] rowKey=Bytes.toBytesBinary("20160116141006\\x00\\x00servlet");
> KeyValue keyValue=new KeyValue(
> rowKey,
> Bytes.toBytes("SPAN"),
> Bytes.toBytes("APPID"),
> 1453117575829L,
> org.apache.hadoop.hbase.KeyValue.Type.Put);
> SkipScanFilter skipScanFilter=new SkipScanFilter(keyRanges, rowKeySchema);
> skipScanFilter.filterKeyValue(keyValue);
> Cell nextCellHint=skipScanFilter.getNextCellHint(keyValue);
> assertTrue(Bytes.toStringBinary(CellUtil.cloneRow(nextCellHint)).equals("20160116141006\\x00\\x002404787"));
> //try once again...
> skipScanFilter.filterKeyValue(keyValue);
> nextCellHint=skipScanFilter.getNextCellHint(keyValue);
> assertTrue(Bytes.toStringBinary(CellUtil.cloneRow(nextCellHint)).equals("20160116141006\\x00\\x002404787"));
> }
> {code}
> W also notice if we enable the java assertion, the above query will not run forever,but get a error reponse,caused by the following code in SkipScanFilter's setNextCellHint method:
> {code:borderStyle=solid}
> if (!isHintAfterPrevious) {
> String msg = "The next hint must come after previous hint (prev=" + previousCellHint + ", next=" + nextCellHint + ", kv=" + kv + ")";
> assert isHintAfterPrevious : msg;
> logger.warn(msg);
> }
> {code}
> but often java assertion is disable,so above code should throw a exception instead of assertion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)