You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "André Cruz (JIRA)" <ji...@apache.org> on 2012/05/09 12:29:51 UTC

[jira] [Created] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

André Cruz created CASSANDRA-4230:
-------------------------------------

             Summary: Deleting a CF always produces an error and that CF remains in an unknown state
                 Key: CASSANDRA-4230
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.1.0
         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
            Reporter: André Cruz


>From the CLI perspective:

[default@Disco] drop column family client; 
null
org.apache.thrift.transport.TTransportException
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)

Log:

 INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
 INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
 INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
 INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
 INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
 INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
 INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
 INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
 INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
 INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
 INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
com.sun.jna.LastErrorException: errno was 17
        at org.apache.cassandra.utils.CLibrary.link(Native Method)
        at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
        at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
        at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
        at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
        at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
        at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
        at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
        at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
        at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
        at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
        at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
        at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
        ... 11 more
Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
        at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
        at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
        at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
        at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        ... 3 more
Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
        at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
        ... 10 more
ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
        at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
        at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
        at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
        at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
        at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
        at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
        ... 10 more
 INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
 INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.

Schema:

CREATE COLUMN FAMILY Client WITH
       key_validation_class = UUIDType AND
       comparator = UTF8Type AND
       column_metadata = [ { column_name: key, validation_class: BytesType }
                           { column_name: name, validation_class: UTF8Type }
                           { column_name: userid, validation_class: UUIDType, index_type: KEYS }
                         ] AND
       compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
       compaction_strategy = LeveledCompactionStrategy AND
       compaction_strategy_options = { sstable_size_in_mb: 10 } AND
       gc_grace = 432000;

State of data dir after deletion attempt:

# ls -lah /var/lib/cassandra/data/Disco/Client/ 
total 76K
drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
-rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
-rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
-rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
-rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
-rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
-rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
-rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
-rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
-rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
-rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
-rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
-rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
-rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
-rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271589#comment-13271589 ] 

Brandon Williams commented on CASSANDRA-4230:
---------------------------------------------

snapshot on compaction is one way it can happen.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271596#comment-13271596 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

Yeah, it's only called in snapshotWithoutFlush so if you try to drop CF in the middle of compaction snapshoting files it would be the only way to lead to such behavior. I think what we need to do here is to stop compaction and run drop after that...
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "André Cruz (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271682#comment-13271682 ] 

André Cruz commented on CASSANDRA-4230:
---------------------------------------

I suppose you mean this:
{noformat}
$ cat /etc/cassandra/cassandra.yaml |grep snapshot |grep compac
# Whether or not to take a snapshot before each compaction.  Be
snapshot_before_compaction: false
{noformat}

I did not enable it.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271648#comment-13271648 ] 

Jonathan Ellis commented on CASSANDRA-4230:
-------------------------------------------

bq. snapshot on compaction is one way it can happen.

I suppose there's a really small chance of that happening if you happen to time it just right, but that wouldn't explain it being easily reproducible (since the compaction and drop snapshots generate their snapshot name independently).

Also, snapshot-on-compaction is off by default, I doubt André has it enabled.  André, can you confirm?
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271548#comment-13271548 ] 

Brandon Williams commented on CASSANDRA-4230:
---------------------------------------------

Since snapshot dirs include a timestamp, perhaps we should just move along if it already exists.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274389#comment-13274389 ] 

Jonathan Ellis commented on CASSANDRA-4230:
-------------------------------------------

Do we need this in 1.0 too?
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274270#comment-13274270 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

I have figured out that this problem is caused only when LeveledCompaction is used for Secondary Index, it seems like when index's leveled manifest is snapshoted it uses the wrong name.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271558#comment-13271558 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

Yeah, we probably should do just that.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271570#comment-13271570 ] 

Jonathan Ellis commented on CASSANDRA-4230:
-------------------------------------------

But why would it attempt the same snapshot twice?  Sounds like there's a real bug here.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Updated] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4230:
--------------------------------------

    Attachment: 4230-v2.txt

simpler v2 attached
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: 4230-v2.txt, CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271534#comment-13271534 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

>From the exception message I see that it's related to the JNA hardlinks while it does a snapshot:

{noformat}
ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
com.sun.jna.LastErrorException: errno was 17
...
Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
{noformat}

and errno of 17 means "File exists" so this is something with snapshot or the system rather then migrations.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274751#comment-13274751 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

Andre, can you please test and confirm that everything works as expected before we commit it?
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: 4230-v2.txt, CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Assigned] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis reassigned CASSANDRA-4230:
-----------------------------------------

    Assignee: Pavel Yaskevich

probably related to CASSANDRA-4219
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "André Cruz (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13275981#comment-13275981 ] 

André Cruz commented on CASSANDRA-4230:
---------------------------------------

It works, thanks!
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: 4230-v2.txt, CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274391#comment-13274391 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

No, this bug came together with directory-per-CF which was added into 1.1.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Updated] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pavel Yaskevich updated CASSANDRA-4230:
---------------------------------------

    Attachment: CASSANDRA-4230.patch
    
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (CASSANDRA-4230) Deleting a CF always produces an error and that CF remains in an unknown state

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274748#comment-13274748 ] 

Pavel Yaskevich commented on CASSANDRA-4230:
--------------------------------------------

Heh, I overlooked that possibility, +1.
                
> Deleting a CF always produces an error and that CF remains in an unknown state
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4230
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4230
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Debian Linux Squeeze with the cassandra debian package from Apache.
>            Reporter: André Cruz
>            Assignee: Pavel Yaskevich
>             Fix For: 1.1.1
>
>         Attachments: 4230-v2.txt, CASSANDRA-4230.patch
>
>
> From the CLI perspective:
> [default@Disco] drop column family client; 
> null
> org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> 	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> 	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_column_family(Cassandra.java:1222)
> 	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_column_family(Cassandra.java:1209)
> 	at org.apache.cassandra.cli.CliClient.executeDelColumnFamily(CliClient.java:1301)
> 	at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:234)
> 	at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:219)
> 	at org.apache.cassandra.cli.CliMain.main(CliMain.java:346)
> Log:
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,686 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,687 Memtable.java (line 266) Writing Memtable-schema_columnfamilies@225225949(978/1222 serialized/live bytes, 21 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,748 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-34-Data.db (1041 bytes)
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,749 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,750 Memtable.java (line 266) Writing Memtable-schema_columns@213209572(586/732 serialized/live bytes, 12 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,812 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db (649 bytes)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,814 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-27-Data.db'), SSTableReader
> (path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-25-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-26-Data.db'), SSTableReader(path
> ='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-28-Data.db')]
>  INFO [MigrationStage:1] 2012-05-09 11:25:35,918 ColumnFamilyStore.java (line 634) Enqueuing flush of Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [FlushWriter:3] 2012-05-09 11:25:35,919 Memtable.java (line 266) Writing Memtable-Client@864320066(372/465 serialized/live bytes, 6 ops)
>  INFO [CompactionExecutor:20] 2012-05-09 11:25:35,945 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-29-Data.db,].  22,486 to 20,621 (~91% of orig
> inal) bytes for 2 keys at 0.150120MB/s.  Time: 131ms.
>  INFO [FlushWriter:3] 2012-05-09 11:25:36,013 Memtable.java (line 307) Completed flushing /var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db (407 bytes)
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,043 CLibrary.java (line 158) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
>         at org.apache.cassandra.utils.CLibrary.link(Native Method)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:3] 2012-05-09 11:25:36,048 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
>         at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
>         at org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
>         at org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
>         at org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
>         ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
> ERROR [MigrationStage:1] 2012-05-09 11:25:36,051 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>         at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
>         at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
>         at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
>         at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/Disco/Client/snapshots/1336559135918-Client/Client.json (errno 17)
>         at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>         at org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>         at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>         ... 10 more
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,052 CompactionTask.java (line 114) Compacting [SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-4-Data.db')]
>  INFO [CompactionExecutor:22] 2012-05-09 11:25:36,187 CompactionTask.java (line 225) Compacted to [/var/lib/cassandra/data/Disco/Client/Disco-Client-hc-6-Data.db,].  728 to 458 (~62% of original) bytes for 8 keys at 0.003235MB/s.  Time: 135ms.
> Schema:
> CREATE COLUMN FAMILY Client WITH
>        key_validation_class = UUIDType AND
>        comparator = UTF8Type AND
>        column_metadata = [ { column_name: key, validation_class: BytesType }
>                            { column_name: name, validation_class: UTF8Type }
>                            { column_name: userid, validation_class: UUIDType, index_type: KEYS }
>                          ] AND
>        compression_options = { sstable_compression:SnappyCompressor, chunk_length_kb:64 } AND
>        compaction_strategy = LeveledCompactionStrategy AND
>        compaction_strategy_options = { sstable_size_in_mb: 10 } AND
>        gc_grace = 432000;
> State of data dir after deletion attempt:
> # ls -lah /var/lib/cassandra/data/Disco/Client/ 
> total 76K
> drwxr-xr-x  3 cassandra cassandra 4.0K May  9 11:25 .
> drwxr-xr-x 17 cassandra cassandra 4.0K May  3 12:34 ..
> -rw-r--r--  2 cassandra cassandra  420 May  9 11:25 Client-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx-old.json
> -rw-r--r--  1 cassandra cassandra  418 May  7 18:04 Client.Client_userid_idx.json
> -rw-r--r--  1 cassandra cassandra  418 May  9 11:25 Client.json
> -rw-r--r--  1 cassandra cassandra   46 May  9 11:25 Disco-Client-hc-6-CompressionInfo.db
> -rw-r--r--  1 cassandra cassandra  458 May  9 11:25 Disco-Client-hc-6-Data.db
> -rw-r--r--  1 cassandra cassandra  976 May  9 11:25 Disco-Client-hc-6-Filter.db
> -rw-r--r--  1 cassandra cassandra  208 May  9 11:25 Disco-Client-hc-6-Index.db
> -rw-r--r--  1 cassandra cassandra 4.3K May  9 11:25 Disco-Client-hc-6-Statistics.db
> -rw-r--r--  4 cassandra cassandra   46 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-CompressionInfo.db
> -rw-r--r--  4 cassandra cassandra   92 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Data.db
> -rw-r--r--  4 cassandra cassandra  496 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Filter.db
> -rw-r--r--  4 cassandra cassandra   26 May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Index.db
> -rw-r--r--  4 cassandra cassandra 4.3K May  7 18:04 Disco-Client.Client_userid_idx-hc-2-Statistics.db
> drwxr-xr-x  6 cassandra cassandra 4.0K May  9 11:25 snapshots

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira