You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2014/08/14 23:32:19 UTC
[jira] [Commented] (PHOENIX-965) Cannot aggregate on index with
NULL values
[ https://issues.apache.org/jira/browse/PHOENIX-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097702#comment-14097702 ]
Hudson commented on PHOENIX-965:
--------------------------------
SUCCESS: Integrated in Phoenix | 3.0 | Hadoop1 #184 (See [https://builds.apache.org/job/Phoenix-3.0-hadoop1/184/])
PHOENIX-965 Cannot aggregate on index with NULL values (jtaylor: rev 9173fa2e8f612c30ef44f63bcc5935dc739763f1)
* phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryIT.java
* phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ServerAggregators.java
> Cannot aggregate on index with NULL values
> ------------------------------------------
>
> Key: PHOENIX-965
> URL: https://issues.apache.org/jira/browse/PHOENIX-965
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 3.0.0, 4.0.0
> Reporter: Lukas Benes
> Fix For: 5.0.0, 3.1, 4.1
>
> Attachments: PHOENIX-965.patch, nullableIndexFix.patch
>
>
> I've found issue related to indexed NULL values (I've also applied PHOENIX-63 that writing NULL values to index )
> CREATE TABLE tbl (id INTEGER, col INTEGER CONSTRAINT pk PRIMARY KEY (id))
> CREATE index tbl_idx ON tbl (col, id) /* without index works ok */
> UPSERT INTO tbl (id, col) VALUES (1, NULL) /* problematic line with NULL value */
> UPSERT INTO tbl (id, col) VALUES (2, 4)
> UPSERT INTO tbl (id, col) VALUES (3, 5)
> SELECT SUM(col) FROM tbl
> SUM on index table with null values causes ServerAggregator exception, because empty byte array is send to aggregate function, due to evaluate
> expression [TO_INTEGER(PK[-1])] [\x00\x80\x00\x00\x01/0:_0/1398957414306/Put/vlen=0/ts=2] returns true and ptr is set to EMPTY_BYTE_ARRAY, because length is 0
> https://github.com/apache/incubator-phoenix/blob/3.0/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyColumnExpression.java#L116
> I'm not sure if it is fixable by returning false when evaluated to NULL, but here is hotfix
> https://github.com/tzolkincz/incubator-phoenix/commit/adf24c887d7c5248ee12c7970580c187e8f5e436#diff-6d1bc085da39a4ee2e51c36a47a33e21R117
> org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
> at org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:289)
> at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:44)
> at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:66)
> at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:86)
> at org.apache.phoenix.iterate.GroupedAggregatingResultIterator.next(GroupedAggregatingResultIterator.java:68)
> at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
> at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:732)
> at org.apache.phoenix.end2end.IndexSumIT.testIndexSumAggregate(IndexSumIT.java:45)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
> at java.util.concurrent.FutureTask.get(FutureTask.java:91)
> at org.apache.phoenix.iterate.ParallelIterators.getIterators(ParallelIterators.java:283)
> ... 33 more
> Caused by: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:97)
> at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:57)
> at org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:251)
> at org.apache.phoenix.iterate.ParallelIterators$2.call(ParallelIterators.java:245)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> at org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:256)
> at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166)
> at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:210)
> at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:125)
> at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:120)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:702)
> at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:54)
> ... 7 more
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.DoNotRetryIOException: TBL_IDX,,1399026966936.08f1d45504f9f6f2c9c3f01d5af612d0.: 0
> at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
> at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:68)
> at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2566)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2534)
> at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
> at org.apache.phoenix.schema.PDataType$IntCodec.decodeInt(PDataType.java:5709)
> at org.apache.phoenix.schema.PDataType$IntCodec.decodeLong(PDataType.java:5690)
> at org.apache.phoenix.schema.PDataType$BaseCodec.decodeLong(PDataType.java:5472)
> at org.apache.phoenix.expression.aggregator.NumberSumAggregator.aggregate(NumberSumAggregator.java:68)
> at org.apache.phoenix.expression.aggregator.ServerAggregators.aggregate(ServerAggregators.java:61)
> at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:277)
> at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
> ... 8 more
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
> at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
> at com.sun.proxy.$Proxy20.openScanner(Unknown Source)
> at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:224)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:126)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
> at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:164)
> ... 12 more
--
This message was sent by Atlassian JIRA
(v6.2#6252)