You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Conan Cook <co...@amee.com> on 2012/09/26 20:13:15 UTC
Truncate causing subsequent timeout on KeyIterator?
Hi,
I'm running a bunch of integration tests using an embedded cassandra
instance via the Cassandra Maven Plugin v1.0.0-1, using Hector v1.0-5.
I've got an issue where one of the tests is using a StringKeyIterator to
iterate over all the keys in a CF, but it gets TimedOutExceptions every
time when trying to communicate with Cassandra; all the other tests using
the same (Spring-wired) keyspace behave fine (stack trace below). A
previous test is calling a cluster.truncate() to ensure an empty CF before
each test, and it's this that seems to cause the problem - at least,
commenting it out causes the other test to run fine.
Any ideas on what could be causing this? Both tests are using the same
instance of Keyspace, autowired via Spring, and the same instance of
Cluster in the same way. No exceptions are being thrown by the truncate
operation - it completes successfully and does its job.
Thanks,
Conan
Stack trace:
[2012-09-26 18:59:53,002] [WARN ] [main] [m.p.c.c.HConnectionManager] Could
not fullfill request on this host CassandraClient<localhost:9170-195>
[2012-09-26 18:59:53,003] [WARN ] [main] [m.p.c.c.HConnectionManager]
Exception:
me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
at
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35)
~[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:163)
~[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:145)
~[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
~[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258)
~[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(KeyspaceServiceImpl.java:167)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:66)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:62)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(ThriftRangeSlicesQuery.java:61)
[hector-core-1.0-5.jar:na]
at
me.prettyprint.cassandra.service.KeyIterator.runQuery(KeyIterator.java:102)
[hector-core-1.0-5.jar:na]
...<snip>...
Caused by: org.apache.cassandra.thrift.TimedOutException: null
at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12270)
~[cassandra-thrift-1.1.0.jar:1.1.0]
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
~[libthrift-0.7.0.jar:0.7.0]
at
org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683)
~[cassandra-thrift-1.1.0.jar:1.1.0]
at
org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667)
~[cassandra-thrift-1.1.0.jar:1.1.0]
at
me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:151)
~[hector-core-1.0-5.jar:na]
Re: Truncate causing subsequent timeout on KeyIterator?
Posted by aaron morton <aa...@thelastpickle.com>.
Nothing jumps out.
Turn the cassandra logging up to DEBUG and take a look at the cassandra logs. Your read will be logged in a [ReadStage-*] thread.
Hope that helps.
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 27/09/2012, at 6:13 AM, Conan Cook <co...@amee.com> wrote:
> Hi,
>
> I'm running a bunch of integration tests using an embedded cassandra instance via the Cassandra Maven Plugin v1.0.0-1, using Hector v1.0-5. I've got an issue where one of the tests is using a StringKeyIterator to iterate over all the keys in a CF, but it gets TimedOutExceptions every time when trying to communicate with Cassandra; all the other tests using the same (Spring-wired) keyspace behave fine (stack trace below). A previous test is calling a cluster.truncate() to ensure an empty CF before each test, and it's this that seems to cause the problem - at least, commenting it out causes the other test to run fine.
>
> Any ideas on what could be causing this? Both tests are using the same instance of Keyspace, autowired via Spring, and the same instance of Cluster in the same way. No exceptions are being thrown by the truncate operation - it completes successfully and does its job.
>
> Thanks,
>
>
> Conan
>
> Stack trace:
>
> [2012-09-26 18:59:53,002] [WARN ] [main] [m.p.c.c.HConnectionManager] Could not fullfill request on this host CassandraClient<localhost:9170-195>
> [2012-09-26 18:59:53,003] [WARN ] [main] [m.p.c.c.HConnectionManager] Exception:
> me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
> at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35) ~[hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:163) ~[hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:145) ~[hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) ~[hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258) ~[hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(KeyspaceServiceImpl.java:167) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:66) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:62) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(ThriftRangeSlicesQuery.java:61) [hector-core-1.0-5.jar:na]
> at me.prettyprint.cassandra.service.KeyIterator.runQuery(KeyIterator.java:102) [hector-core-1.0-5.jar:na]
>
> ...<snip>...
>
> Caused by: org.apache.cassandra.thrift.TimedOutException: null
> at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12270) ~[cassandra-thrift-1.1.0.jar:1.1.0]
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) ~[libthrift-0.7.0.jar:0.7.0]
> at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683) ~[cassandra-thrift-1.1.0.jar:1.1.0]
> at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667) ~[cassandra-thrift-1.1.0.jar:1.1.0]
> at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:151) ~[hector-core-1.0-5.jar:na]