You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2019/03/11 05:40:00 UTC
[jira] [Commented] (PHOENIX-5171) SkipScan incorrectly filters
composite primary key which the key range contains all values
[ https://issues.apache.org/jira/browse/PHOENIX-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789175#comment-16789175 ]
Hadoop QA commented on PHOENIX-5171:
------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12961907/PHOENIX-5171-master-v2.patch
against master branch at commit 26d3734d3c2a05c56e92a0ea09a99fd90059f568.
ATTACHMENT ID: 12961907
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100:
+ conn.createStatement().execute("upsert into aiolos values('2018-02-14','channel_agg',2,null,null,'A004',null,null,2,2)");
+ conn.createStatement().execute("upsert into aiolos values('2018-02-14','channel_agg',2,null,null,null,null,null,2,2)");
+ PVarchar.INSTANCE.getKeyRange(Bytes.toBytes("2018-02-10"), true, Bytes.toBytes("2019-02-19"), true),
+ PVarchar.INSTANCE.getKeyRange(Bytes.toBytes("channel"), true, Bytes.toBytes("channel"), true),
+ PChar.INSTANCE.getKeyRange(Bytes.toBytes("2"), true, Bytes.toBytes("2"), true),
+ PVarchar.INSTANCE.getKeyRange(Bytes.toBytes("A004"), true, Bytes.toBytes("A004"), true),
+ ByteUtil.concat(Bytes.toBytes("2018-02-14"), QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes("channel"),
+ ByteUtil.concat(Bytes.toBytes("2018-02-14"), QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes("channel"),
+ QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes("2"), QueryConstants.SEPARATOR_BYTE_ARRAY, QueryConstants.SEPARATOR_BYTE_ARRAY,
+ new Include(ByteUtil.concat(Bytes.toBytes("2018-02-15"), QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes("channel"),
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConcurrentMutationsIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2407//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2407//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2407//console
This message is automatically generated.
> SkipScan incorrectly filters composite primary key which the key range contains all values
> ------------------------------------------------------------------------------------------
>
> Key: PHOENIX-5171
> URL: https://issues.apache.org/jira/browse/PHOENIX-5171
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 5.0.0, 4.14.1
> Reporter: jaanai
> Assignee: jaanai
> Priority: Critical
> Fix For: 5.1.0
>
> Attachments: PHOENIX-5171-master-v2.patch, PHOENIX-5171-master.patch
>
>
> Running the below SQL:
> {code:sql}
> create table if not exists aiolos(
> vdate varchar,
> tab varchar,
> dev tinyint not null,
> app varchar,
> target varchar,
> channel varchar,
> one varchar,
> two varchar,
> count1 integer,
> count2 integer,
> CONSTRAINT PK PRIMARY KEY (vdate,tab,dev,app,target,channel,one,two));
> upsert into aiolos values('2018-02-14','channel_agg',2,null,null,'A004',null,null,2,2);
> upsert into aiolos values('2018-02-14','channel_agg',2,null,null,null,null,null,2,2);
> SELECT * FROM aiolos WHERE dev = 2 AND vdate BETWEEN '2018-02-10' AND '2019-02-19' AND tab = 'channel_agg' and channel='A004';
> {code}
> Throws exception:
> {code:java}
> Caused by: java.lang.IllegalStateException: The next hint must come after previous hint (prev=2018-02-14\x00channel_agg\x00\x82//LATEST_TIMESTAMP/Maximum/vlen=0/seqid=0, next=2018-02-14\x00channel_agg\x00\x82//LATEST_TIMESTAMP/Maximum/vlen=0/seqid=0, kv=2018-02-14\x00channel_agg\x00\x82/0:\x00\x00\x00\x00/1550642992223/Put/vlen=4/seqid=5445463)
> at org.apache.phoenix.filter.SkipScanFilter.setNextCellHint(SkipScanFilter.java:171)
> at org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:145)
> at org.apache.hadoop.hbase.filter.FilterList.filterKeyValue(FilterList.java:264)
> at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:418)
> at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:557)
> at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:6308)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6459)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6246)
> at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6232)
> at org.apache.phoenix.iterate.RegionScannerFactory$1.nextRaw(RegionScannerFactory.java:175)
> ... 8 more
> {code}
> The caused by incorrect next cell hint.
> adding skipped row into nextCellHintMap. Actually, due to we don't store NULL at the end of the key for the variable data type, these keys should be skipped when invokes filterKeyValue, because they are smaller than the rest of the positions of the slots.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)