You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Michael Yeh (JIRA)" <ji...@apache.org> on 2012/10/24 22:14:15 UTC

[jira] [Created] (CASSANDRA-4856) Thrift daemon is crashing the server

Michael Yeh created CASSANDRA-4856:
--------------------------------------

             Summary: Thrift daemon is crashing the server
                 Key: CASSANDRA-4856
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
             Project: Cassandra
          Issue Type: Bug
    Affects Versions: 1.1.6
         Environment: MacOS 10.8.2
            Reporter: Michael Yeh


I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
INFO 12:55:37,147 Listening for thrift clients...
ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:658)
	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)

After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483661#comment-13483661 ] 

Brandon Williams commented on CASSANDRA-4856:
---------------------------------------------

Does this reproduce reliably for you?
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484031#comment-13484031 ] 

Brandon Williams commented on CASSANDRA-4856:
---------------------------------------------

Try this: set rpc_max_threads to something reasonable like 1000 and see if it repros.  If not, it's the client code freaking out and creating connections until the OS won't let the server create more threads for them.

Alternately, you could try it with the hsha server so it's not thread-per-connection.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484853#comment-13484853 ] 

Brandon Williams commented on CASSANDRA-4856:
---------------------------------------------

bq. Would it be better to set an artificially high limit vs unlimited number of threads as the default? IMHO, I would prefer to deal with some tuning on the server

If the environment is artificially limiting the amount of threads you can create, solving that limitation instead of moving it into the server seems to make more sense.  You can always use the hsha server if that's something you want to avoid.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483709#comment-13483709 ] 

Michael Yeh commented on CASSANDRA-4856:
----------------------------------------

Upgrading the OS is a red herring.  What it did was it wiped out my cassandra directory and I basically started with a fresh install.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483703#comment-13483703 ] 

Michael Yeh commented on CASSANDRA-4856:
----------------------------------------

Yes it does.  If I delete /var/lib/cassandra and don't create the namespace I'll get the same error message.

If you really think it's specific to my machine, point me to a cassandra server that I can access and I can point my thrift client to to it and see if it repros.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Yeh reopened CASSANDRA-4856:
------------------------------------


Brandon's comment suggesting it was environment specific was incorrect.  It worked correctly once the namespace and column family was created.  The server should never crash even if was not initialized correctly.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Closed] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams closed CASSANDRA-4856.
---------------------------------------

    
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams resolved CASSANDRA-4856.
-----------------------------------------

    Resolution: Invalid

"unable to create new native thread" means the OS denied java.  It was probably related to your OS upgrade somehow.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483588#comment-13483588 ] 

Michael Yeh edited comment on CASSANDRA-4856 at 10/24/12 9:00 PM:
------------------------------------------------------------------

Why did it work again once I recreated the keyspace and column family then?  Did you at least try to repro it?
                
      was (Author: myeh):
    Why did it work again once I recreated the keyspace and column family then?
                  
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483588#comment-13483588 ] 

Michael Yeh commented on CASSANDRA-4856:
----------------------------------------

Why did it work again once I recreated the keyspace and column family then?
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Michael Yeh (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484428#comment-13484428 ] 

Michael Yeh commented on CASSANDRA-4856:
----------------------------------------

Setting rpc_max_threads worked.  I'm seeing: 

WARN 12:35:13,255 Maximum number of clients 1000 reached

instead of the server crashing.  Would it be better to set an artificially high limit vs unlimited number of threads as the default?  IMHO, I would prefer to deal with some tuning on the server than one that crashes because of misbehaving clients which may or may not be under your control.  Thanks for the tip.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-4856) Thrift daemon is crashing the server

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis resolved CASSANDRA-4856.
---------------------------------------

    Resolution: Invalid

Brandon's diagnosis of your error message is accurate.
                
> Thrift daemon is crashing the server
> ------------------------------------
>
>                 Key: CASSANDRA-4856
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4856
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.6
>         Environment: MacOS 10.8.2
>            Reporter: Michael Yeh
>
> I'm new to cassandra and was looking at using the kundera thrift client.  I was able to get the sample code working without any issues.  After upgrading to mountain lion on my mac, I didn't realize I had lost the /var/lib/cassandra directory.  When I tried to run the previously working sample code, it would crash cassandra with the following stack trace:
> INFO 12:55:37,147 Listening for thrift clients...
> ERROR 12:55:43,545 Exception in thread Thread[Thread-3,5,main]
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
> 	at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:103)
> 	at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)
> After realizing that I needed to recreate the keyspace and column families, everything started to work again.  Although I'm no longer blocked with my issue, no clients should ever be able to crash a server even if it is user error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira