You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Harald Falzberger <h....@gmail.com> on 2011/12/04 18:38:21 UTC

OutOfMemory Exception during bootstrap

Hi,

I'm trying to set up a test environment with 2 nodes on one physical
machine with two ips. I configured both as adviced in the
documentation:

cluster_name: 'MyDemoCluster'
initial_token: 0
seed_provider:
- seeds: "IP1"
listen_address: IP1
rpc_address: IP1

cluster_name: 'MyDemoCluster'
initial_token: 85070591730234615865843651857942052864
seed_provider:
- seeds: "IP1"
listen_address: IP2
rpc_address: IP2

Node1 uses 7199 as JMX port, Node2 7198 because JMX by default is
listening on all interfaces.

When I bootstrap node2, on node1 following exception is thrown and
node1 terminates. the same error occurs again if I try to restart
node1 and node2 is still running.

Does anyone of you have an idea why this happens? I'm starting each
cassandra instance with 16GB RAM and my database is empty.

Exception on Node1
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
at java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1384)
at org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:77)
at org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:65)
at org.apache.cassandra.concurrent.StageManager.multiThreadedStage(StageManager.java:58)
at org.apache.cassandra.concurrent.StageManager.<clinit>(StageManager.java:44)
at org.apache.cassandra.net.MessagingService.receive(MessagingService.java:512)
at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:159)

Re: OutOfMemory Exception during bootstrap

Posted by Harald Falzberger <h....@gmail.com>.
You are right Samal, configuring cassandra-env.sh a little bit and let
it calculate heap size solved the problem! thank you!

On Mon, Dec 5, 2011 at 11:23, Harald Falzberger <h....@gmail.com> wrote:
> I don't think that this is the problem because I'm testing on a
> supercomputer with 16T RAM.
>
> Is it possible, that cassandra runs into this bug of the jvm?
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6399443
>
> On Mon, Dec 5, 2011 at 06:29, samal <sa...@gmail.com> wrote:
>> Lower your heap size, if you are testing multiple instance with single node.
>>
>> https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L64
>>
>>
>> On Sun, Dec 4, 2011 at 11:08 PM, Harald Falzberger <h....@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to set up a test environment with 2 nodes on one physical
>>> machine with two ips. I configured both as adviced in the
>>> documentation:
>>>
>>> cluster_name: 'MyDemoCluster'
>>> initial_token: 0
>>> seed_provider:
>>> - seeds: "IP1"
>>> listen_address: IP1
>>> rpc_address: IP1
>>>
>>> cluster_name: 'MyDemoCluster'
>>> initial_token: 85070591730234615865843651857942052864
>>> seed_provider:
>>> - seeds: "IP1"
>>> listen_address: IP2
>>> rpc_address: IP2
>>>
>>> Node1 uses 7199 as JMX port, Node2 7198 because JMX by default is
>>> listening on all interfaces.
>>>
>>> When I bootstrap node2, on node1 following exception is thrown and
>>> node1 terminates. the same error occurs again if I try to restart
>>> node1 and node2 is still running.
>>>
>>> Does anyone of you have an idea why this happens? I'm starting each
>>> cassandra instance with 16GB RAM and my database is empty.
>>>
>>> Exception on Node1
>>> java.lang.OutOfMemoryError: unable to create new native thread
>>> at java.lang.Thread.start0(Native Method)
>>> at java.lang.Thread.start(Thread.java:597)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1384)
>>> at
>>> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:77)
>>> at
>>> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:65)
>>> at
>>> org.apache.cassandra.concurrent.StageManager.multiThreadedStage(StageManager.java:58)
>>> at
>>> org.apache.cassandra.concurrent.StageManager.<clinit>(StageManager.java:44)
>>> at
>>> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:512)
>>> at
>>> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:159)
>>
>>

Re: OutOfMemory Exception during bootstrap

Posted by Harald Falzberger <h....@gmail.com>.
I don't think that this is the problem because I'm testing on a
supercomputer with 16T RAM.

Is it possible, that cassandra runs into this bug of the jvm?
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6399443

On Mon, Dec 5, 2011 at 06:29, samal <sa...@gmail.com> wrote:
> Lower your heap size, if you are testing multiple instance with single node.
>
> https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L64
>
>
> On Sun, Dec 4, 2011 at 11:08 PM, Harald Falzberger <h....@gmail.com>
> wrote:
>>
>> Hi,
>>
>> I'm trying to set up a test environment with 2 nodes on one physical
>> machine with two ips. I configured both as adviced in the
>> documentation:
>>
>> cluster_name: 'MyDemoCluster'
>> initial_token: 0
>> seed_provider:
>> - seeds: "IP1"
>> listen_address: IP1
>> rpc_address: IP1
>>
>> cluster_name: 'MyDemoCluster'
>> initial_token: 85070591730234615865843651857942052864
>> seed_provider:
>> - seeds: "IP1"
>> listen_address: IP2
>> rpc_address: IP2
>>
>> Node1 uses 7199 as JMX port, Node2 7198 because JMX by default is
>> listening on all interfaces.
>>
>> When I bootstrap node2, on node1 following exception is thrown and
>> node1 terminates. the same error occurs again if I try to restart
>> node1 and node2 is still running.
>>
>> Does anyone of you have an idea why this happens? I'm starting each
>> cassandra instance with 16GB RAM and my database is empty.
>>
>> Exception on Node1
>> java.lang.OutOfMemoryError: unable to create new native thread
>> at java.lang.Thread.start0(Native Method)
>> at java.lang.Thread.start(Thread.java:597)
>> at
>> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
>> at
>> java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1384)
>> at
>> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:77)
>> at
>> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:65)
>> at
>> org.apache.cassandra.concurrent.StageManager.multiThreadedStage(StageManager.java:58)
>> at
>> org.apache.cassandra.concurrent.StageManager.<clinit>(StageManager.java:44)
>> at
>> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:512)
>> at
>> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:159)
>
>

Re: OutOfMemory Exception during bootstrap

Posted by samal <sa...@gmail.com>.
Lower your heap size, if you are testing multiple instance with single
node.

https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L64


On Sun, Dec 4, 2011 at 11:08 PM, Harald Falzberger
<h....@gmail.com>wrote:

> Hi,
>
> I'm trying to set up a test environment with 2 nodes on one physical
> machine with two ips. I configured both as adviced in the
> documentation:
>
> cluster_name: 'MyDemoCluster'
> initial_token: 0
> seed_provider:
> - seeds: "IP1"
> listen_address: IP1
> rpc_address: IP1
>
> cluster_name: 'MyDemoCluster'
> initial_token: 85070591730234615865843651857942052864
> seed_provider:
> - seeds: "IP1"
> listen_address: IP2
> rpc_address: IP2
>
> Node1 uses 7199 as JMX port, Node2 7198 because JMX by default is
> listening on all interfaces.
>
> When I bootstrap node2, on node1 following exception is thrown and
> node1 terminates. the same error occurs again if I try to restart
> node1 and node2 is still running.
>
> Does anyone of you have an idea why this happens? I'm starting each
> cassandra instance with 16GB RAM and my database is empty.
>
> Exception on Node1
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:597)
> at
> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
> at
> java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1384)
> at
> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:77)
> at
> org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:65)
> at
> org.apache.cassandra.concurrent.StageManager.multiThreadedStage(StageManager.java:58)
> at
> org.apache.cassandra.concurrent.StageManager.<clinit>(StageManager.java:44)
> at
> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:512)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:159)
>