You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by ka...@barclays.com on 2016/08/02 19:59:27 UTC
Java Query timeout
Hello All,
We're using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com,60020,1469522152474, seqNum=245191
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
... 3 more
Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
... 13 more
The same query works fine in sqlline (taking about 30 seconds to finish).
From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
1) hbase.rpc.timeout
2) phoenix.query.keepAliveMs
3) phoenix.query.timeoutMs
Now:
1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn't be network latency or something
2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
Thanks in advance
Kannan.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
RE: Java Query timeout
Posted by ka...@barclays.com.
Increasing scanner timeout period with the below properties helped, thanks for your help.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Tuesday, August 09, 2016 3:29
To: user@phoenix.apache.org
Subject: Re: Java Query timeout
bq. 1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn’t be network latency or something
It timed out at 60306ms , it doesn't mean query will take this much time only, it may take longer too. If java client is running from remote machine and sqlline is local then network latency can affect RPC too.
This is actually , your scan RPC request is taking time to respond and is not returning within a timeout period. You need to increase scanner timeout period along with above properties you mentioned.
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>60000</value>
</property>
<property>
Regards,
Ankit Singhal
On Mon, Aug 8, 2016 at 6:55 PM, <ka...@barclays.com>> wrote:
Thanks Brian. I have added HBASE_CONF_DIR and it’s still timing out.
Any help?
From: Brian Jeltema [mailto:bdjeltema@gmail.com<ma...@gmail.com>]
Sent: Thursday, August 04, 2016 15:08
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Java Query timeout
Regarding your second question, try
export HBASE_CONF_DIR=<path/to/hbase/conf/directory>
before running the client.
On Aug 4, 2016, at 3:01 PM, kannan.ramanathan@barclays.com<ma...@barclays.com> wrote:
Any pointers?
From: Ramanathan, Kannan: R&A (NYK)
Sent: Wednesday, August 03, 2016 15:08
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: RE: Java Query timeout
Can someone please take a look?
From: Ramanathan, Kannan: IT (NYK)
Sent: Tuesday, August 02, 2016 15:59
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Java Query timeout
Hello All,
We’re using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
Caused by: java.net<http://java.net>.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com<http://abc.com/>,60020,1469522152474, seqNum=245191
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
... 3 more
Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020<http://abc.com/1.1.1.1:60020> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
... 13 more
The same query works fine in sqlline (taking about 30 seconds to finish).
From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
1) hbase.rpc.timeout
2) phoenix.query.keepAliveMs
3) phoenix.query.timeoutMs
Now:
1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn’t be network latency or something
2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
Thanks in advance
Kannan.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
Re: Java Query timeout
Posted by Ankit Singhal <an...@gmail.com>.
bq. 1) From the exceptions, it looks like timeout is set to 1min (60000ms)
and actual query takes little more than a minute (60306ms) to finish. Why
does Java client takes longer to finish than sqlline? The query returns
only one record so there shouldn’t be network latency or something
It timed out at 60306ms , it doesn't mean query will take this much time
only, it may take longer too. If java client is running from remote machine
and sqlline is local then network latency can affect RPC too.
This is actually , your scan RPC request is taking time to respond and is
not returning within a timeout period. You need to increase scanner timeout
period along with above properties you mentioned.
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>60000</value>
</property>
<property>
Regards,
Ankit Singhal
On Mon, Aug 8, 2016 at 6:55 PM, <ka...@barclays.com> wrote:
> Thanks Brian. I have added HBASE_CONF_DIR and it’s still timing out.
>
>
>
> Any help?
>
>
>
> *From:* Brian Jeltema [mailto:bdjeltema@gmail.com]
> *Sent:* Thursday, August 04, 2016 15:08
> *To:* user@phoenix.apache.org
> *Subject:* Re: Java Query timeout
>
>
>
> Regarding your second question, try
>
>
>
> export HBASE_CONF_DIR=<path/to/hbase/conf/directory>
>
>
>
> before running the client.
>
>
>
> On Aug 4, 2016, at 3:01 PM, kannan.ramanathan@barclays.com wrote:
>
>
>
> Any pointers?
>
>
>
> *From:* Ramanathan, Kannan: R&A (NYK)
> *Sent:* Wednesday, August 03, 2016 15:08
> *To:* user@phoenix.apache.org
> *Subject:* RE: Java Query timeout
>
>
>
> Can someone please take a look?
>
>
>
> *From:* Ramanathan, Kannan: IT (NYK)
> *Sent:* Tuesday, August 02, 2016 15:59
> *To:* user@phoenix.apache.org
> *Subject:* Java Query timeout
>
>
>
> Hello All,
>
>
>
> We’re using Phoenix 4.7 with CDH 5.7.1. The query from Java client is
> timing out with this error:
>
>
>
> Caused by: java.net.SocketTimeoutException: callTimeout=60000,
> callDuration=60306: row '' on table 'PHOENIX_TABLE' at region=
> PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=
> ABC.com <http://abc.com/>,60020,1469522152474, seqNum=245191
>
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.
> callWithRetries(RpcRetryingCaller.java:159)
>
> at org.apache.hadoop.hbase.client.
> ResultBoundedCompletionService$QueueingFuture.run(
> ResultBoundedCompletionService.java:64)
>
> ... 3 more
>
> Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020
> <http://abc.com/1.1.1.1:60020> failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60,
> waitTime=60005, operationTimeout=60000 expired.
>
> at org.apache.hadoop.hbase.ipc.
> RpcClientImpl.wrapException(RpcClientImpl.java:1255)
>
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(
> RpcClientImpl.java:1223)
>
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.
> callBlockingMethod(AbstractRpcClient.java:216)
>
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$
> BlockingRpcChannelImplementation.callBlockingMethod(
> AbstractRpcClient.java:300)
>
> at org.apache.hadoop.hbase.protobuf.generated.
> ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
>
> at org.apache.hadoop.hbase.client.ScannerCallable.call(
> ScannerCallable.java:213)
>
> at org.apache.hadoop.hbase.client.ScannerCallable.call(
> ScannerCallable.java:62)
>
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.
> callWithoutRetries(RpcRetryingCaller.java:200)
>
> at org.apache.hadoop.hbase.client.
> ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.
> java:371)
>
> at org.apache.hadoop.hbase.client.
> ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.
> java:345)
>
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.
> callWithRetries(RpcRetryingCaller.java:126)
>
> ... 4 more
>
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60,
> waitTime=60005, operationTimeout=60000 expired.
>
> at org.apache.hadoop.hbase.ipc.
> Call.checkAndSetTimeout(Call.java:70)
>
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(
> RpcClientImpl.java:1197)
>
> ... 13 more
>
>
>
> The same query works fine in sqlline (taking about 30 seconds to finish).
>
>
>
> From various posts/blogs/user forums, it appears that client side
> hbase-site.xml should have these properties set possibly with higher
> numbers:
>
>
>
> 1) hbase.rpc.timeout
>
> 2) phoenix.query.keepAliveMs
>
> 3) phoenix.query.timeoutMs
>
>
>
> Now:
>
>
>
> 1) From the exceptions, it looks like timeout is set to 1min (60000ms) and
> actual query takes little more than a minute (60306ms) to finish. Why does
> Java client takes longer to finish than sqlline? The query returns only one
> record so there shouldn’t be network latency or something
>
> 2) I have added the above three properties to client side hbase-site.xml
> but still getting the same error. I am pretty sure (checked the property
> values by querying HBaseConfiguration.create()) hbase-site.xml is set first
> in the class path. What am I missing here? How can I be sure these
> properties are actually fed correctly?
>
>
>
> Thanks in advance
>
>
>
> Kannan.
>
>
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation,
> advice, offer or solicitation to buy or sell a product or service nor an
> official confirmation of any transaction. It is directed at persons who are
> professionals and is not intended for retail customer use. Intended for
> recipient only. This message is subject to the terms at: www.barclays.com/
> emaildisclaimer.
>
> For important disclosures, please see: www.barclays.com/
> salesandtradingdisclaimer regarding market commentary from Barclays Sales
> and/or Trading, who are active market participants; and in respect of
> Barclays Research, including disclosures relating to specific issuers,
> please see http://publicresearch.barclays.com.
>
> _______________________________________________
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation,
> advice, offer or solicitation to buy or sell a product or service nor an
> official confirmation of any transaction. It is directed at persons who are
> professionals and is not intended for retail customer use. Intended for
> recipient only. This message is subject to the terms at: www.barclays.com/
> emaildisclaimer.
>
> For important disclosures, please see: www.barclays.com/
> salesandtradingdisclaimer regarding market commentary from Barclays Sales
> and/or Trading, who are active market participants; and in respect of
> Barclays Research, including disclosures relating to specific issuers,
> please see http://publicresearch.barclays.com.
>
> _______________________________________________
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation,
> advice, offer or solicitation to buy or sell a product or service nor an
> official confirmation of any transaction. It is directed at persons who are
> professionals and is not intended for retail customer use. Intended for
> recipient only. This message is subject to the terms at: www.barclays.com/
> emaildisclaimer.
>
> For important disclosures, please see: www.barclays.com/
> salesandtradingdisclaimer regarding market commentary from Barclays Sales
> and/or Trading, who are active market participants; and in respect of
> Barclays Research, including disclosures relating to specific issuers,
> please see http://publicresearch.barclays.com.
>
> _______________________________________________
>
>
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation,
> advice, offer or solicitation to buy or sell a product or service nor an
> official confirmation of any transaction. It is directed at persons who are
> professionals and is not intended for retail customer use. Intended for
> recipient only. This message is subject to the terms at: www.barclays.com/
> emaildisclaimer.
>
> For important disclosures, please see: www.barclays.com/
> salesandtradingdisclaimer regarding market commentary from Barclays Sales
> and/or Trading, who are active market participants; and in respect of
> Barclays Research, including disclosures relating to specific issuers,
> please see http://publicresearch.barclays.com.
>
> _______________________________________________
>
RE: Java Query timeout
Posted by ka...@barclays.com.
Thanks Brian. I have added HBASE_CONF_DIR and it’s still timing out.
Any help?
From: Brian Jeltema [mailto:bdjeltema@gmail.com]
Sent: Thursday, August 04, 2016 15:08
To: user@phoenix.apache.org
Subject: Re: Java Query timeout
Regarding your second question, try
export HBASE_CONF_DIR=<path/to/hbase/conf/directory>
before running the client.
On Aug 4, 2016, at 3:01 PM, kannan.ramanathan@barclays.com<ma...@barclays.com> wrote:
Any pointers?
From: Ramanathan, Kannan: R&A (NYK)
Sent: Wednesday, August 03, 2016 15:08
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: RE: Java Query timeout
Can someone please take a look?
From: Ramanathan, Kannan: IT (NYK)
Sent: Tuesday, August 02, 2016 15:59
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Java Query timeout
Hello All,
We’re using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com<http://abc.com/>,60020,1469522152474, seqNum=245191
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
... 3 more
Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020<http://abc.com/1.1.1.1:60020> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
... 13 more
The same query works fine in sqlline (taking about 30 seconds to finish).
From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
1) hbase.rpc.timeout
2) phoenix.query.keepAliveMs
3) phoenix.query.timeoutMs
Now:
1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn’t be network latency or something
2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
Thanks in advance
Kannan.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com<http://publicresearch.barclays.com/>.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
Re: Java Query timeout
Posted by Brian Jeltema <bd...@gmail.com>.
Regarding your second question, try
export HBASE_CONF_DIR=<path/to/hbase/conf/directory>
before running the client.
> On Aug 4, 2016, at 3:01 PM, kannan.ramanathan@barclays.com wrote:
>
> Any pointers?
>
> From: Ramanathan, Kannan: R&A (NYK)
> Sent: Wednesday, August 03, 2016 15:08
> To: user@phoenix.apache.org <ma...@phoenix.apache.org>
> Subject: RE: Java Query timeout
>
> Can someone please take a look?
>
> From: Ramanathan, Kannan: IT (NYK)
> Sent: Tuesday, August 02, 2016 15:59
> To: user@phoenix.apache.org <ma...@phoenix.apache.org>
> Subject: Java Query timeout
>
> Hello All,
>
> We’re using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
>
> Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com <http://abc.com/>,60020,1469522152474, seqNum=245191
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
> at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
> ... 3 more
> Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020 <http://abc.com/1.1.1.1:60020> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
> at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
> at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> ... 4 more
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
> ... 13 more
>
> The same query works fine in sqlline (taking about 30 seconds to finish).
>
> From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
>
> 1) hbase.rpc.timeout
> 2) phoenix.query.keepAliveMs
> 3) phoenix.query.timeoutMs
>
> Now:
>
> 1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn’t be network latency or something
> 2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
>
> Thanks in advance
>
> Kannan.
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer <http://www.barclays.com/emaildisclaimer>.
>
> For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer <http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com <http://publicresearch.barclays.com/>.
>
> _______________________________________________
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer <http://www.barclays.com/emaildisclaimer>.
>
> For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer <http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com <http://publicresearch.barclays.com/>.
>
> _______________________________________________
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer <http://www.barclays.com/emaildisclaimer>.
>
> For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer <http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com <http://publicresearch.barclays.com/>.
>
> _______________________________________________
>
RE: Java Query timeout
Posted by ka...@barclays.com.
Any pointers?
From: Ramanathan, Kannan: R&A (NYK)
Sent: Wednesday, August 03, 2016 15:08
To: user@phoenix.apache.org
Subject: RE: Java Query timeout
Can someone please take a look?
From: Ramanathan, Kannan: IT (NYK)
Sent: Tuesday, August 02, 2016 15:59
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Java Query timeout
Hello All,
We're using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com,60020,1469522152474, seqNum=245191
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
... 3 more
Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
... 13 more
The same query works fine in sqlline (taking about 30 seconds to finish).
From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
1) hbase.rpc.timeout
2) phoenix.query.keepAliveMs
3) phoenix.query.timeoutMs
Now:
1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn't be network latency or something
2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
Thanks in advance
Kannan.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
RE: Java Query timeout
Posted by ka...@barclays.com.
Can someone please take a look?
From: Ramanathan, Kannan: IT (NYK)
Sent: Tuesday, August 02, 2016 15:59
To: user@phoenix.apache.org
Subject: Java Query timeout
Hello All,
We're using Phoenix 4.7 with CDH 5.7.1. The query from Java client is timing out with this error:
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=60306: row '' on table 'PHOENIX_TABLE' at region= PHOENIX_TABLE,,1462867135721.5bc05e2b5485f3fd550e9f93a795a100., hostname=ABC.com,60020,1469522152474, seqNum=245191
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
... 3 more
Caused by: java.io.IOException: Call to ABC.com/1.1.1.1:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1255)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:213)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:345)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 4 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=60, waitTime=60005, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
... 13 more
The same query works fine in sqlline (taking about 30 seconds to finish).
From various posts/blogs/user forums, it appears that client side hbase-site.xml should have these properties set possibly with higher numbers:
1) hbase.rpc.timeout
2) phoenix.query.keepAliveMs
3) phoenix.query.timeoutMs
Now:
1) From the exceptions, it looks like timeout is set to 1min (60000ms) and actual query takes little more than a minute (60306ms) to finish. Why does Java client takes longer to finish than sqlline? The query returns only one record so there shouldn't be network latency or something
2) I have added the above three properties to client side hbase-site.xml but still getting the same error. I am pretty sure (checked the property values by querying HBaseConfiguration.create()) hbase-site.xml is set first in the class path. What am I missing here? How can I be sure these properties are actually fed correctly?
Thanks in advance
Kannan.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer<http://www.barclays.com/emaildisclaimer>.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer<http://www.barclays.com/salesandtradingdisclaimer> regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________