You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Marten Kenbeek (Jira)" <ji...@apache.org> on 2021/10/14 11:53:00 UTC
[jira] [Commented] (CASSANDRA-14113) AssertionError while trying to
upgrade 2.2.11 -> 3.11.1
[ https://issues.apache.org/jira/browse/CASSANDRA-14113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428772#comment-17428772 ]
Marten Kenbeek commented on CASSANDRA-14113:
--------------------------------------------
I've looked into the case with column_metadata some more. Seems that on the thrift side, the column_metadata overrides the validation type of specific subcolumns, but it's still possible to add data for dynamic subcolumns, not just the statically defined ones. On CQL's side, however, only the static subcolumns defined in columns_metadata are accessible. Either way, it's these statically defined subcolumns in column_metadata that we need to account for in decodeForSuperColumn().
So I think the solution for this should be:
* If the table is dense, always return the compact value column with the subcolumn as the collection element.
* If the table is sparse, check for a statically defined subcolumn. This means it is a regular column, and is not the compact value column (i.e. the internal map column) or super column value column (i.e. the "value" column by default). If a statically defined subcolumn is found, return that column, otherwise fall back to the compact value column with collection element.
[~blerer] If you've got some time to take a look, let me know what you think. We've decided to temporarily patch Cassandra during the upgrade for this issue and another one specific to our usecase, so it's not blocking for us, but extra eyes are more than welcome.
> AssertionError while trying to upgrade 2.2.11 -> 3.11.1
> -------------------------------------------------------
>
> Key: CASSANDRA-14113
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14113
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
> Environment: Tables have been created in 2.2.11 using thrift and have supercolumns
> Reporter: Guillaume Herail
> Assignee: Benjamin Lerer
> Priority: Normal
> Labels: supercolumns
> Attachments: data.tar.gz
>
>
> We're trying to upgrade a test cluster from Cassandra 2.2.11 to Cassandra 3.11.1. The tables have been created using thrift and have supercolumns. When I try to run {{nodetool upgradesstables}} I get the following:
> {noformat}error: null
> -- StackTrace --
> java.lang.AssertionError
> at org.apache.cassandra.db.rows.BufferCell.<init>(BufferCell.java:42)
> at org.apache.cassandra.db.LegacyLayout$CellGrouper.addCell(LegacyLayout.java:1242)
> at org.apache.cassandra.db.LegacyLayout$CellGrouper.addAtom(LegacyLayout.java:1185)
> at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:498)
> at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:472)
> at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:306)
> at org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:188)
> at org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:140)
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:122)
> at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:484)
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:499)
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:359)
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:75)
> at org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:26)
> at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
> at org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:233)
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:196)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85)
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:428)
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:315)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> We also tried to upgrade to 3.0.15 instead and had a different error:
> {noformat}
> ERROR 11:00:40 Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.IllegalStateException: [ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=, type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.BytesType,org.apache.cassandra.db.marshal.BytesType), kind=REGULAR, position=-1}] is not a subset of []
> at org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:532) ~[main/:na]
> at org.apache.cassandra.db.Columns$Serializer.serializedSubsetSize(Columns.java:484) ~[main/:na]
> at org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:290) ~[main/:na]
> at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:169) ~[main/:na]
> at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114) ~[main/:na]
> at org.apache.cassandra.db.ColumnIndex$Builder.add(ColumnIndex.java:144) ~[main/:na]
> at org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:112) ~[main/:na]
> at org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52) ~[main/:na]
> at org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149) ~[main/:na]
> at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125) ~[main/:na]
> at org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88) ~[main/:na]
> at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109) ~[main/:na]
> at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195) ~[main/:na]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na]
> at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89) ~[main/:na]
> at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) ~[main/:na]
> at org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424) ~[main/:na]
> at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311) ~[main/:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_151]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_151]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) [main/:na]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_151]
> {noformat}
> Attached are a set of sstables that reproduce the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org