You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2018/11/08 00:37:00 UTC
[jira] [Updated] (PHOENIX-5007) DELETE FROM times out - even with
large HBase timeout setting
[ https://issues.apache.org/jira/browse/PHOENIX-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated PHOENIX-5007:
-----------------------------------
Description:
It seems there is currently no way to make a somewhat large DELETE FROM work without timing out. "Large" here includes size equivalent to a single HBase region, which means we cannot write a M/R (or Spark) task to delete a lot of data.
This is the exception:
{code:java}
0: jdbc:phoenix:localhost> delete from test where v1 < 0.999;
Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099 (state=08000,code=101)
org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1296)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1237)
at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
at org.apache.phoenix.compile.DeleteCompiler$ServerSelectDeleteMutationPlan.execute(DeleteCompiler.java:791)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1825)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:813)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)
Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1291)
... 19 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:171)
at org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
at org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
at org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:135)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:120)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:156)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:164)
... 10 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:329)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:242)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:277)
at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:438)
at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:312)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:147)
... 11 more
Caused by: java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:178)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
... 3 more
Caused by: java.io.IOException: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:389)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:405)
at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:188)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:625)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:700)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:428)
... 1 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
... 5 more
{code}
was:
It seems there is currently no way to make a somewhat large DELETE FROM work without a timeout. "Large" here includes size equivalent to a single HBase region, which means we cannot write a M/R (or Spark) task to delete a lot of data.
This is the exception:
{code:java}
0: jdbc:phoenix:localhost> delete from test where v1 < 0.999;
Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099 (state=08000,code=101)
org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1296)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1237)
at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
at org.apache.phoenix.compile.DeleteCompiler$ServerSelectDeleteMutationPlan.execute(DeleteCompiler.java:791)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1825)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:813)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)
Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1291)
... 19 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:171)
at org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
at org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
at org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:135)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:120)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:156)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:164)
... 10 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:329)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:242)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:277)
at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:438)
at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:312)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:147)
... 11 more
Caused by: java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:178)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
... 3 more
Caused by: java.io.IOException: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:389)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:405)
at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:188)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:625)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:700)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:428)
... 1 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
... 5 more
{code}
> DELETE FROM times out - even with large HBase timeout setting
> -------------------------------------------------------------
>
> Key: PHOENIX-5007
> URL: https://issues.apache.org/jira/browse/PHOENIX-5007
> Project: Phoenix
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Priority: Major
>
> It seems there is currently no way to make a somewhat large DELETE FROM work without timing out. "Large" here includes size equivalent to a single HBase region, which means we cannot write a M/R (or Spark) task to delete a lot of data.
> This is the exception:
> {code:java}
> 0: jdbc:phoenix:localhost> delete from test where v1 < 0.999;
> Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099 (state=08000,code=101)
> org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
> at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1296)
> at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1237)
> at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
> at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
> at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
> at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
> at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
> at org.apache.phoenix.compile.DeleteCompiler$ServerSelectDeleteMutationPlan.execute(DeleteCompiler.java:791)
> at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
> at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
> at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1825)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:813)
> at sqlline.SqlLine.begin(SqlLine.java:686)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1291)
> ... 19 more
> Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
> at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:171)
> at org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
> at org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
> at org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
> at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:135)
> at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:120)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
> at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:156)
> at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:164)
> ... 10 more
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
> Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:329)
> at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:242)
> at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
> at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:277)
> at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:438)
> at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:312)
> at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:147)
> ... 11 more
> Caused by: java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5., hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:178)
> at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> ... 3 more
> Caused by: java.io.IOException: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:389)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:405)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:188)
> at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:625)
> at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:700)
> at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:428)
> ... 1 more
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
> ... 5 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)