You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Chris Herron (Created) (JIRA)" <ji...@apache.org> on 2012/02/28 17:51:46 UTC

[jira] [Created] (CASSANDRA-3975) Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily

Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily
----------------------------------------------------------------------

                 Key: CASSANDRA-3975
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3975
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.0.7
            Reporter: Chris Herron
             Fix For: 1.0.9, 1.1.0


If hints have accumulated for a CF that has been deleted, Hinted Handoff repeatedly fails until manual intervention removes those hints. For 1.0.7, UnserializableColumnFamilyException is thrown only when a CFid is unknown on the sending node. As discussed on #cassandra-dev, if the schema is in agreement, the affected hint(s) should be deleted to avoid indefinite repeat failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3975) Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily

Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218695#comment-13218695 ] 

Jonathan Ellis commented on CASSANDRA-3975:
-------------------------------------------

Patch posted to https://github.com/jbellis/cassandra/branches/3975
                
> Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-3975
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3975
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Chris Herron
>              Labels: datastax_qa
>             Fix For: 1.0.9, 1.1.0
>
>
> If hints have accumulated for a CF that has been deleted, Hinted Handoff repeatedly fails until manual intervention removes those hints. For 1.0.7, UnserializableColumnFamilyException is thrown only when a CFid is unknown on the sending node. As discussed on #cassandra-dev, if the schema is in agreement, the affected hint(s) should be deleted to avoid indefinite repeat failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3975) Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily

Posted by "Chris Herron (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218348#comment-13218348 ] 

Chris Herron commented on CASSANDRA-3975:
-----------------------------------------

Stack trace from 1.0.7:

 INFO [HintedHandoff:854] 2012-02-27 22:39:51,183 org.apache.cassandra.db.HintedHandOffManager Started hinted handoff for token: Token(bytes[7f]) with IP: /XX.XX.XX.XXX
ERROR [HintedHandoff:854] 2012-02-27 22:39:51,186 org.apache.cassandra.service.AbstractCassandraDaemon Fatal exception in thread Thread[HintedHandoff:854,1,main]
java.lang.RuntimeException: org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=2391
        at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:689)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=2391
        at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:129)
        at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:401)
        at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:409)
        at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:344)
        at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:248)
        at org.apache.cassandra.db.HintedHandOffManager.access$200(HintedHandOffManager.java:84)
        at org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:416)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
                
> Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-3975
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3975
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.7
>            Reporter: Chris Herron
>             Fix For: 1.0.9, 1.1.0
>
>
> If hints have accumulated for a CF that has been deleted, Hinted Handoff repeatedly fails until manual intervention removes those hints. For 1.0.7, UnserializableColumnFamilyException is thrown only when a CFid is unknown on the sending node. As discussed on #cassandra-dev, if the schema is in agreement, the affected hint(s) should be deleted to avoid indefinite repeat failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3975) Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily

Posted by "Chris Herron (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218353#comment-13218353 ] 

Chris Herron commented on CASSANDRA-3975:
-----------------------------------------

Suggest that for 1.0.x, UnserializableColumnFamilyException might be better named UnknownColumnFamilyException (since unknown cfId is the only time its ever thrown. It could then be caught within HHOM.deliverHintsToEndpointInternal and the affected hint deleted. Its unclear to me if this needs to be tackled differently for 1.1.
                
> Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-3975
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3975
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.7
>            Reporter: Chris Herron
>             Fix For: 1.0.9, 1.1.0
>
>
> If hints have accumulated for a CF that has been deleted, Hinted Handoff repeatedly fails until manual intervention removes those hints. For 1.0.7, UnserializableColumnFamilyException is thrown only when a CFid is unknown on the sending node. As discussed on #cassandra-dev, if the schema is in agreement, the affected hint(s) should be deleted to avoid indefinite repeat failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-3975) Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily

Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218763#comment-13218763 ] 

Brandon Williams commented on CASSANDRA-3975:
---------------------------------------------

+1
                
> Hints Should Be Dropped When Missing CFid Implies Deleted ColumnFamily
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-3975
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3975
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Chris Herron
>            Assignee: Jonathan Ellis
>              Labels: datastax_qa
>             Fix For: 1.0.9, 1.1.0
>
>
> If hints have accumulated for a CF that has been deleted, Hinted Handoff repeatedly fails until manual intervention removes those hints. For 1.0.7, UnserializableColumnFamilyException is thrown only when a CFid is unknown on the sending node. As discussed on #cassandra-dev, if the schema is in agreement, the affected hint(s) should be deleted to avoid indefinite repeat failures.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira