You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David B (JIRA)" <ji...@apache.org> on 2012/07/18 22:34:34 UTC

[jira] [Created] (CASSANDRA-4444) Failure to delete column families

David B created CASSANDRA-4444:
----------------------------------

             Summary: Failure to delete column families
                 Key: CASSANDRA-4444
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.1.2
         Environment: 2 node cluster running on Ubuntu Precise
            Reporter: David B


I have a two node cluster, and one keyspace defined as follows:
create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};

I then create a column family as follows:
create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }

I stream SSTables through SStableLoader.

After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  

Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Comment Edited] (CASSANDRA-4444) Failure to delete column families

Posted by "David B (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417758#comment-13417758 ] 

David B edited comment on CASSANDRA-4444 at 7/18/12 10:25 PM:
--------------------------------------------------------------

In the most recent test, it did delete the metadata.  I wasn't aware of the modifications introduced in CASSANDRA-4221--thanks for the pointer. 

Running the same test multiple times yesterday, however, got the server into a state where it would not delete the CF from keyspace metadata.  I had to delete and rebuild the system keyspace for the system to return to "normal operation".  Symptoms were very similar to what is reported in https://issues.apache.org/jira/browse/CASSANDRA-4431.  Any chance they could be related?

In the meantime I'll try to reproduce the erroneous state. Thanks for the quick response! 
                
      was (Author: sj.climber):
    In the most recent test, it did delete the metadata.  I wasn't aware of the modifications introduced in CASSANDRA-4221--thanks for the pointer. 

Running the same test multiple times yesterday, however, got the server into a state where it would not delete the CF from keyspace metadata.  I had to delete and rebuild the system keyspace for the system to return to "normal operation".  Symptoms were very similar to what is reported in https://issues.apache.org/jira/browse/CASSANDRA-4431.  Any chance they could be related.

In the meantime I'll try to reproduce the erroneous state. Thanks for the quick response! 
                  
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "David B (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417758#comment-13417758 ] 

David B commented on CASSANDRA-4444:
------------------------------------

In the most recent test, it did delete the metadata.  I wasn't aware of the modifications introduced in CASSANDRA-4221--thanks for the pointer. 

Running the same test multiple times yesterday, however, got the server into a state where it would not delete the CF from keyspace metadata.  I had to delete and rebuild the system keyspace for the system to return to "normal operation".  Symptoms were very similar to what is reported in https://issues.apache.org/jira/browse/CASSANDRA-4431.  Any chance they could be related.

In the meantime I'll try to reproduce the erroneous state. Thanks for the quick response! 
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Resolved] (CASSANDRA-4444) Failure to delete column families

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pavel Yaskevich resolved CASSANDRA-4444.
----------------------------------------

    Resolution: Duplicate

Resolving as duplicate to CASSANDRA-4432 until further notice.
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417764#comment-13417764 ] 

Pavel Yaskevich commented on CASSANDRA-4444:
--------------------------------------------

Yeah, this was my second guess, CASSANDRA-4432 applies for all of schema operations, can you try to apply CASSANDRA-4432 before running your test? 
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "David B (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417806#comment-13417806 ] 

David B commented on CASSANDRA-4444:
------------------------------------

You mention CASSANDRA-4432 applies for all schema operations--doesn't the issue manifest only randomly, though, based on the results returned by nanoTime()?  Are there any workarounds, instead of applying the patch?

At the moment I'm unfortunately unable to replace the current install (from the 1.1.2 deb packages) with a patched source build.  I can probably arrange to do this in the next few days, though.
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417674#comment-13417674 ] 

Pavel Yaskevich commented on CASSANDRA-4444:
--------------------------------------------

So did it actually delete your CF from keyspace metadata? You can also take a look at CASSANDRA-4221 discussion about why SSTable files still in place if you try to delete CF while compaction is running.
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417814#comment-13417814 ] 

Pavel Yaskevich commented on CASSANDRA-4444:
--------------------------------------------

bq. You mention CASSANDRA-4432 applies for all schema operations--doesn't the issue manifest only randomly, though, based on the results returned by nanoTime()?

Yes, that manifests randomly, that is why we weren't able to track the issue sooner. I don't think that there any workarounds available except applying the patch.
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4444) Failure to delete column families

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421485#comment-13421485 ] 

Pavel Yaskevich commented on CASSANDRA-4444:
--------------------------------------------

Any news on this one, David?
                
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (CASSANDRA-4444) Failure to delete column families

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis reassigned CASSANDRA-4444:
-----------------------------------------

    Assignee: Pavel Yaskevich
    
> Failure to delete column families
> ---------------------------------
>
>                 Key: CASSANDRA-4444
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4444
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: 2 node cluster running on Ubuntu Precise
>            Reporter: David B
>            Assignee: Pavel Yaskevich
>
> I have a two node cluster, and one keyspace defined as follows:
> create keyspace SampleKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:2};
> I then create a column family as follows:
> create column family SampleFamily with caching = 'keys_only' and key_validation_class = 'LongType' and compression_options = { sstable_compression: SnappyCompressor, chunk_length_kb: 64 }
> I stream SSTables through SStableLoader.
> After the load is complete, compaction begins.  During this time, I request a drop of the family through cassandra-cli using "drop column family SampleFamily".  
> Cassandra-cli responds that schemas are in agreement.  Looking on the file system, however, the full set of data files are still found under data/SampleFamily (in addition to the snapshot created on drop family).  There are no errors in either system or output logs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira