You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Pierre Chalamet (JIRA)" <ji...@apache.org> on 2012/04/30 23:29:49 UTC

[jira] [Created] (CASSANDRA-4201) Node crash after truncating CF

Pierre Chalamet created CASSANDRA-4201:
------------------------------------------

             Summary: Node crash after truncating CF
                 Key: CASSANDRA-4201
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.1.0
         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
            Reporter: Pierre Chalamet
         Attachments: cassandra.yaml

1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:

2. run the following commands in cli:
{code}
create keyspace toto;
use toto;
create column family titi;
truncate titi;
{code}

3. the node dies with this error:
{code}
ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
java.io.IOError: java.io.IOException: Map failed
        at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
        at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
        at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
        at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(Unknown Source)
        at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
        ... 5 more
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        ... 7 more
 INFO 23:23:02,122 Stop listening to thrift clients
 INFO 23:23:02,123 Waiting for messaging service to quiesce
 INFO 23:23:02,125 MessagingService shutting down server thread.
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Pierre Chalamet (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265700#comment-13265700 ] 

Pierre Chalamet commented on CASSANDRA-4201:
--------------------------------------------

Yes it is a 32 bits jvm running on x64 system.
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265810#comment-13265810 ] 

Jonathan Ellis commented on CASSANDRA-4201:
-------------------------------------------

How big is your heap?
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Preserve commitlog size cap when recycling segments at startup

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4201:
--------------------------------------

    Summary: Preserve commitlog size cap when recycling segments at startup  (was: Node crash after truncating CF)
    
> Preserve commitlog size cap when recycling segments at startup
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>            Assignee: Jonathan Ellis
>              Labels: commitlog
>             Fix For: 1.1.1
>
>         Attachments: 4201.txt, cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Pierre Chalamet (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Chalamet updated CASSANDRA-4201:
---------------------------------------

    Attachment: cassandra.yaml
    
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Preserve commitlog size cap when recycling segments at startup

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4201:
--------------------------------------

    Priority: Minor  (was: Major)
    
> Preserve commitlog size cap when recycling segments at startup
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>            Assignee: Jonathan Ellis
>            Priority: Minor
>              Labels: commitlog
>             Fix For: 1.1.1
>
>         Attachments: 4201.txt, cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Pierre Chalamet (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265837#comment-13265837 ] 

Pierre Chalamet commented on CASSANDRA-4201:
--------------------------------------------

1 gig. Using jvm options in cassandra.bat :

set JAVA_OPTS=-ea^
 -javaagent:"%CASSANDRA_HOME%\lib\jamm-0.2.5.jar"^
 -Xms1G^
 -Xmx1G^
 -XX:+HeapDumpOnOutOfMemoryError^
 -XX:+UseParNewGC^
 -XX:+UseConcMarkSweepGC^
 -XX:+CMSParallelRemarkEnabled^
 -XX:SurvivorRatio=8^
 -XX:MaxTenuringThreshold=1^
 -XX:CMSInitiatingOccupancyFraction=75^
 -XX:+UseCMSInitiatingOccupancyOnly^
 -Dcom.sun.management.jmxremote.port=7199^
 -Dcom.sun.management.jmxremote.ssl=false^
 -Dcom.sun.management.jmxremote.authenticate=false^
 -Dlog4j.configuration=log4j-server.properties^
 -Dlog4j.defaultInitOverride=true
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4201:
--------------------------------------

    Attachment: 4201.txt

The problem is you don't have enough contiguous address space to mmap the commitlog segements.

1.1.1 already has a configuration parameter to change the commitlog segment size; dropping that to 16, and setting commitlog_total_space_in_mb to the same, works for me.

I did find a related bug, that Cassandra keeps extra commitlog segments around at startup time.  Patch attached to fix that.
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>              Labels: commitlog
>             Fix For: 1.1.1
>
>         Attachments: 4201.txt, cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4201:
--------------------------------------

    Reviewer: vijay2win@yahoo.com  (was: rbranson)
    
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>            Assignee: Jonathan Ellis
>              Labels: commitlog
>             Fix For: 1.1.1
>
>         Attachments: 4201.txt, cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Preserve commitlog size cap when recycling segments at startup

Posted by "Vijay (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278449#comment-13278449 ] 

Vijay commented on CASSANDRA-4201:
----------------------------------

+1 
                
> Preserve commitlog size cap when recycling segments at startup
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>            Assignee: Jonathan Ellis
>            Priority: Minor
>              Labels: commitlog
>             Fix For: 1.1.1
>
>         Attachments: 4201.txt, cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265407#comment-13265407 ] 

Jonathan Ellis commented on CASSANDRA-4201:
-------------------------------------------

I can't reproduce.  Are you using a 32bit jvm?
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Pierre Chalamet (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Chalamet updated CASSANDRA-4201:
---------------------------------------

    Attachment: log.txt
    
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml, log.txt
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4201) Node crash after truncating CF

Posted by "Pierre Chalamet (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266161#comment-13266161 ] 

Pierre Chalamet commented on CASSANDRA-4201:
--------------------------------------------

The first time truncate is called, CommitLogAllocator (line 104) calls createFreshSegment() : a new CommitLogSegment is created. It is working fine.

It seems that everything collapes when the segment is recycled. CommitLogSegment failed line 119
{code}
     buffer = logFileAccessor.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, CommitLog.SEGMENT_SIZE);
{code}
The map() function is failing with "java.io.IOException: Map failed" with an OutOfMemory as inner exception.

I've also attached a log in debug mode.
                
> Node crash after truncating CF
> ------------------------------
>
>                 Key: CASSANDRA-4201
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4201
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: Windows 7 x64, 4Gb, JRE 1.6.0_31 (x86)
>            Reporter: Pierre Chalamet
>         Attachments: cassandra.yaml
>
>
> 1. Create a single node cluster, use default configuration, use cassandra.bat to start the server:
> 2. run the following commands in cli:
> {code}
> create keyspace toto;
> use toto;
> create column family titi;
> truncate titi;
> {code}
> 3. the node dies with this error:
> {code}
> ERROR 23:23:02,118 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Map failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:202)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:159)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(Unknown Source)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:119)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         ... 7 more
>  INFO 23:23:02,122 Stop listening to thrift clients
>  INFO 23:23:02,123 Waiting for messaging service to quiesce
>  INFO 23:23:02,125 MessagingService shutting down server thread.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira