You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "bert Passek (JIRA)" <ji...@apache.org> on 2012/06/07 14:36:24 UTC
[jira] [Commented] (CASSANDRA-4228) Exception while reading from
cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290965#comment-13290965 ]
bert Passek commented on CASSANDRA-4228:
----------------------------------------
Well, i just checked the cluster configuration and i had a misconfiguration with one of our test clusters. We fixed this and it works like a charm :)
> Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
> -------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Environment: Debian Squeeze
> Reporter: bert Passek
> Attachments: CassandraTest.java
>
>
> We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze system. After that we can not use ColumnFamilyInputFormat anymore due to exceptions in cassandra. A simple unit test is provided via attachement.
> Here are some details about our simple setup:
> Ring:
> Address DC Rack Status State Load Owns Token
> 127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB 100,00% 55894951196891831822413178196787984716
> Schema Definition:
> create column family TestSuper
> with column_type = 'Super'
> and comparator = 'BytesType'
> and subcomparator = 'BytesType'
> and default_validation_class = 'BytesType'
> and key_validation_class = 'BytesType'
> and read_repair_chance = 0.1
> and dclocal_read_repair_chance = 0.0
> and gc_grace = 864000
> and min_compaction_threshold = 4
> and max_compaction_threshold = 32
> and replicate_on_write = true
> and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
> and caching = 'KEYS_ONLY'
> and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};
> While running the test we face following exception on client side:
> 12/05/09 10:18:22 INFO junit.TestRunner: testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest): org.apache.thrift.transport.TTransportException
> 12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
> at de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runners.Suite.runChild(Suite.java:115)
> at org.junit.runners.Suite.runChild(Suite.java:23)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:116)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:107)
> at org.junit.runner.JUnitCore.runClasses(JUnitCore.java:66)
> at de.unister.cpc.junit.TestRunner.run(TestRunner.java:55)
> at de.unister.cpc.MainRunner.runInternal(MainRunner.java:129)
> at de.unister.cpc.MainRunner.run(MainRunner.java:52)
> at de.unister.cpc.MainRunner.main(MainRunner.java:143)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> Caused by: org.apache.thrift.transport.TTransportException
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683)
> at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667)
> at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:356)
> and on server side:
> ==> /var/log/cassandra/system.log <==
> ERROR [Thrift:5] 2012-05-09 10:18:22,603 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.NumberFormatException: Zero length BigInteger
> at java.math.BigInteger.<init>(BigInteger.java:276)
> at java.math.BigInteger.<init>(BigInteger.java:451)
> at org.apache.cassandra.dht.RandomPartitioner$1.fromString(RandomPartitioner.java:136)
> at org.apache.cassandra.thrift.CassandraServer.get_range_slices(CassandraServer.java:685)
> at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Cassandra.java:2944)
> at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Cassandra.java:2932)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
> at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Maybe we are doing something wrong, but after the update we can not execute any hadoop jobs which reads from cassandra via ColumnFamilyInputFormat in combination with OrderPreservingPartitioner.
> Thanks in advance.
> Ciao Bert
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira