You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Christoph Schnepf (Jira)" <ji...@apache.org> on 2025/06/25 11:29:00 UTC
[jira] [Comment Edited] (CASSANDRA-18282) Exception message for RTBoundValidator
[ https://issues.apache.org/jira/browse/CASSANDRA-18282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17983637#comment-17983637 ]
Christoph Schnepf edited comment on CASSANDRA-18282 at 6/25/25 11:28 AM:
-------------------------------------------------------------------------
Any news on this topic please? :)
I'm seeing this on my cluster running Cassandra 4.1.9 frequently now:
{code:java}
ERROR [ReadStage-6] 2025-06-17 00:22:41,412 JVMStabilityInspector.java:68 - Exception in thread Thread[ReadStage-6,5,SharedPool]
java.lang.IllegalStateException: UnfilteredRowIterator for mykeyspace."mytable" has an open RT bound as its last item
at org.apache.cassandra.db.transform.RTBoundCloser$RowsTransformation.moreContents(RTBoundCloser.java:106)
at org.apache.cassandra.db.transform.RTBoundCloser$RowsTransformation.moreContents(RTBoundCloser.java:62)
at org.apache.cassandra.db.transform.BaseIterator.tryGetMoreContents(BaseIterator.java:142)
at org.apache.cassandra.db.transform.BaseIterator.hasMoreContents(BaseIterator.java:132)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:159)
at org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:210)
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:299)
at org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:131)
at org.apache.cassandra.db.ReadResponse.createDigestResponse(ReadResponse.java:72)
at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:349)
at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:92)
at org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:97)
at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:45)
at org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:430)
at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:142)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Suppressed: java.lang.IllegalStateException: PROCESSED UnfilteredRowIterator for mykeyspace."mytable" has an illegal RT bounds sequence: expected all RTs to be closed, but the last one is open
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.onPartitionClose(RTBoundValidator.java:113)
at org.apache.cassandra.db.transform.BaseRows.runOnClose(BaseRows.java:91)
at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:95)
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:300)
... 12 common frames omitted {code}
Is there anything I could do to solve this problem? Or any debug command to find the problematic partition(s)? At least with the current config I do not see anything suspicious in the debug.log.
I already tried to scrub and repair the table, but it didn't change anything. Or could we try to re-delete the data to maybe place the tombstones accordingly?
Edit:
I just applied the patch mentioned in this ticket on my cluster for further investigation.
was (Author: JIRAUSER306570):
Any news on this topic please? :)
I'm seeing this on my cluster running Cassandra 4.1.9 frequently now:
{code:java}
ERROR [ReadStage-6] 2025-06-17 00:22:41,412 JVMStabilityInspector.java:68 - Exception in thread Thread[ReadStage-6,5,SharedPool]
java.lang.IllegalStateException: UnfilteredRowIterator for mykeyspace."mytable" has an open RT bound as its last item
at org.apache.cassandra.db.transform.RTBoundCloser$RowsTransformation.moreContents(RTBoundCloser.java:106)
at org.apache.cassandra.db.transform.RTBoundCloser$RowsTransformation.moreContents(RTBoundCloser.java:62)
at org.apache.cassandra.db.transform.BaseIterator.tryGetMoreContents(BaseIterator.java:142)
at org.apache.cassandra.db.transform.BaseIterator.hasMoreContents(BaseIterator.java:132)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:159)
at org.apache.cassandra.db.rows.UnfilteredRowIterators.digest(UnfilteredRowIterators.java:210)
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:299)
at org.apache.cassandra.db.ReadResponse.makeDigest(ReadResponse.java:131)
at org.apache.cassandra.db.ReadResponse.createDigestResponse(ReadResponse.java:72)
at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:349)
at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:92)
at org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:97)
at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:45)
at org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:430)
at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:142)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Suppressed: java.lang.IllegalStateException: PROCESSED UnfilteredRowIterator for mykeyspace."mytable" has an illegal RT bounds sequence: expected all RTs to be closed, but the last one is open
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.onPartitionClose(RTBoundValidator.java:113)
at org.apache.cassandra.db.transform.BaseRows.runOnClose(BaseRows.java:91)
at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:95)
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators.digest(UnfilteredPartitionIterators.java:300)
... 12 common frames omitted {code}
Is there anything I could do to solve this problem? Or any debug command to find the problematic partition(s)? At least with the current config I do not see anything suspicious in the debug.log.
I already tried to scrub and repair the table, but it didn't change anything. Or could we try to re-delete the data to maybe place the tombstones accordingly?
> Exception message for RTBoundValidator
> --------------------------------------
>
> Key: CASSANDRA-18282
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18282
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Observability/Logging
> Reporter: Brad Schoening
> Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Attachments: 18282-logging.patch
>
>
> From the exception RTBoundValidator its not clear what 'RT' refers to.
> 1.) Exception subject
> Instead of: 'has an illegal RT bounds sequence'
> change to: 'has an illegal Range Tombstone bounds sequence'
> 2.) Why doesn't the (key: %s omdt: [%s]) display in the exception message?
> Example:
> 2023-01-31 18:26:27,480 [ERROR] [ReadStage-9] cluster_id=5 ip_address=127.1.1.1 AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread Thread[ReadStage-9,5,main]
> java.lang.IllegalStateException: MERGED UnfilteredRowIterator for rdhprod.instrument_alias_map has an illegal RT bounds sequence: unexpected end bound or boundary Marker INCL_END_BOUND(T, TBG)@1612002648782856/1612002648
> at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120) ~[apache-cassandra-3.11.11.jar:3.11.11]
> at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87) ~[apache-cassandra-3.11.11.jar:3.11.11]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:148) ~[apache-cassandra-3.11.11.jar:3.11.11]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) ~[apache-cassandra-3.11.11.jar:3.11.11]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org