You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Alex Petrov (JIRA)" <ji...@apache.org> on 2016/12/23 11:58:58 UTC
[jira] [Comment Edited] (CASSANDRA-13025) dtest failure in
upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.static_columns_with_distinct_test
[ https://issues.apache.org/jira/browse/CASSANDRA-13025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767769#comment-15767769 ]
Alex Petrov edited comment on CASSANDRA-13025 at 12/23/16 11:58 AM:
--------------------------------------------------------------------
So far I've tracked it back to [CASSANDRA-12768] backport [95d0b671d1af154eaf1c1e81992c7f3f51469eee|https://github.com/apache/cassandra/commit/95d0b671d1af154eaf1c1e81992c7f3f51469eee].
We have a table with {{k PRIMARY KEY, s1 static, s2 static}}. When trying to query {{SELECT k,s1 FROM tbl}}
we're facing this issue. When all static columns are queried {{SELECT k,s1,s2 ...}}, everything works fine.
The problem is as follows: in {{3.X}}, the notion of {{hasAllColumns}} for statics has changed a bit.
Since they're taken from the {{ColumnFilter#fetchedColumns}}, {{hasAllColumns}} for statics will be
{{true}} all the time. Although {{3.9}} expects that if {{hasAllColumns}} is {{true}}, all columns
are present, and therefore it will try to deserialize them and will get into "corrupted" state (wrong
offset). So problem arises when {{3.9}} is trying to deserialize {{3.X}} row.
We can't just special-case it in the {{UnfilteredSerializer}} for statics, since 3.9 expects that columns
that aren't present, will be [skipped|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/rows/UnfilteredSerializer.java#L616].
In any case, because the format of "fetched" has changed, we're ending up with more columns in 3.9
than in 3.X and when we're trying to skip them, we end up in the wrong place.
I've tried to work around it by introducing a different way to pass columns to the header. Even though
technically it works, it was leading to double-reads: digest query was failing, because data is
different, and the "normal" read command was issued.
We might special-case {{3.X}} to return a "legacy" response, or split internal/external query fetch
mechanisms, but I do not like both ideas.
To be honest, so far I can not see the another solution rather than fetching all statics for
compatibility reasons.
cc [~slebresne] [~thobbs]
was (Author: ifesdjeen):
So far I've tracked it back to [CASSANDRA-12768] backport [95d0b671d1af154eaf1c1e81992c7f3f51469eee|https://github.com/apache/cassandra/commit/95d0b671d1af154eaf1c1e81992c7f3f51469eee].
Problem is that when static partition is fetched not fully (for example only s1 or s2 is fetched, but not both), statics are not fetched at all. Fetching all statics helps to resolve it, but I'll check if there's a better way to do it.
> dtest failure in upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x.static_columns_with_distinct_test
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-13025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13025
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
> Reporter: Sean McCarthy
> Assignee: Alex Petrov
> Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.X_dtest_upgrade/28/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3_Upgrade_current_3_x_To_indev_3_x/static_columns_with_distinct_test
> {code}
> Error Message
> <Error from server: code=0000 [Server error] message="java.io.IOError: java.io.IOException: Corrupt empty row found in unfiltered partition">
> {code}{code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/automaton/cassandra-dtest/tools/decorators.py", line 46, in wrapped
> f(obj)
> File "/home/automaton/cassandra-dtest/upgrade_tests/cql_tests.py", line 4010, in static_columns_with_distinct_test
> rows = list(cursor.execute("SELECT DISTINCT k, s1 FROM test2"))
> File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, timeout, execution_profile, paging_state).result()
> File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 3784, in result
> raise self._final_exception
> {code}{code}
> Standard Output
> http://git-wip-us.apache.org/repos/asf/cassandra.git git:7eac22dd41cb09e6d64fb5ac48b2cca3c8840cc8
> Unexpected error in node2 log, error:
> ERROR [Native-Transport-Requests-2] 2016-12-08 03:20:04,861 Message.java:617 - Unexpected exception during request; channel = [id: 0xf4c13f2c, L:/127.0.0.2:9042 - R:/127.0.0.1:52112]
> java.io.IOError: java.io.IOException: Corrupt empty row found in unfiltered partition
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:224) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:212) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.processPartition(SelectStatement.java:779) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:741) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:408) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:273) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:232) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:219) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:204) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) [apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) [apache-cassandra-3.9.jar:3.9]
> at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.39.Final.jar:4.0.39.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) [netty-all-4.0.39.Final.jar:4.0.39.Final]
> at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) [netty-all-4.0.39.Final.jar:4.0.39.Final]
> at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:357) [netty-all-4.0.39.Final.jar:4.0.39.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.9.jar:3.9]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> Caused by: java.io.IOException: Corrupt empty row found in unfiltered partition
> at org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:430) ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:219) ~[apache-cassandra-3.9.jar:3.9]
> ... 23 common frames omitted
> {code}
> Related failures: (~25)
> http://cassci.datastax.com/job/cassandra-3.X_dtest_upgrade/28/testReport/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)