You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by ruslan usifov <ru...@gmail.com> on 2011/02/16 14:19:21 UTC
memory consuption
Hello
Here is my output from ps aux:
root 737 0.0 0.0 8940 804 ? S 04:34 0:00 nimbus(hdb)
root 768 0.0 0.0 0 0 ? S 04:34 0:00
[flush-202:0]
root 1368 43.1 70.5 8248976 5935852 ? Sl 04:45 297:47
/usr/local/jdk1.6.0_23/bin/java -ea -XX:+UseThreadPriorities -XX:Thr
<<<<<< cassandra daemon
root 1633 0.0 0.0 70584 3212 ? Ss 15:49 0:00 sshd:
ruslan [priv]
Is it normal that cassandra daemon eat so many memory? I look on it around
12 hours and memory consumption is only grows.
ps:
uname -a
Linux slv004 2.6.32-311-ec2 #23-Ubuntu SMP Thu Dec 2 11:14:35 UTC 2010
x86_64 GNU/Linux
java -version
java version "1.6.0_23"
Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
My working machine is 8 CPU 8GB of RAM
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
Victor, I'm a bit confused here as we've now got two conversations in this thread.
Can you please start a new thread about your memory problem, I'm not clear on the details. Please include...
- cassandra version
- JVM heap settings from bin/cassandra.in.sh or conf/cassandra-envsh
- disk_access_mode
- result from top or ps
- why what you are seeing is different from what you expect.
Aaron
On 18 Feb, 2011,at 01:17 PM, Victor Kabdebon <vi...@gmail.com> wrote:
Already done. The disk access mode is standard in storage-conf.xml (I am using 0.6.6 at the moment, I will upgrade to 0.7.x later). But this memory consumption is a real issue.
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
Looks like you are using virtual memory for memmapped files. Change the disk_access_mode to standard if you want to reduce the overall memory usage.
Aaron
On 18 Feb, 2011,at 09:34 AM, Victor Kabdebon <vi...@gmail.com> wrote:
Sorry I forgot to say that this is the partial result of :
ps aux | grep cassandra
Best regards
2011/2/17 Victor Kabdebon <vi...@gmail.com>
Oh right but Cassandra doesn't really respect that, I thought there was another option to set that.
Just for your information, I set xms and xmx very low with a small amount of data. I am waiting to be able to connect jconsole, I don't know why it is not reachable at the moment. Here is my result :
105 26115 0.2 273 1125328 755316 ? Sl Feb09 23:58 /usr/bin/java -ea -Xms64M -Xmx128M
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
bin/cassandra.in.sh
set Xms and Xmx in the JVM_OPTS
Aaron
On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com> wrote:
Is it possible to change the maximum JVM heap memory use in 0.6.X ?
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
What are you using for disk_access_mode ?
Have you tried reducing the JVM head size?
Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory.
Aaron
On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/16 Aaron Morton <aa...@thelastpickle.com>
JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in my case 4GB, here is a part of cassandra-env.sh:
calculate_heap_size()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
return 0
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
return 0
;;
*)
MAX_HEAP_SIZE=1024M
return 1
;;
esac
}
I set all this options by default. All my nodes have 8GB of memory. And i affraid that after some time all my nodes goes to hard swap, and only reboot help them :-(((
PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by Victor Kabdebon <vi...@gmail.com>.
Already done. The disk access mode is standard in storage-conf.xml (I am
using 0.6.6 at the moment, I will upgrade to 0.7.x later). But this memory
consumption is a real issue.
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
> Looks like you are using virtual memory for memmapped files. Change the
> disk_access_mode to standard if you want to reduce the overall memory
> usage.
>
> Aaron
>
> On 18 Feb, 2011,at 09:34 AM, Victor Kabdebon <vi...@gmail.com>
> wrote:
>
> Sorry I forgot to say that this is the partial result of :
> ps aux | grep cassandra
>
> Best regards
>
> 2011/2/17 Victor Kabdebon <vi...@gmail.com>
>
>> Oh right but Cassandra doesn't really respect that, I thought there was
>> another option to set that.
>>
>> Just for your information, I set xms and xmx very low with a small amount
>> of data. I am waiting to be able to connect jconsole, I don't know why it is
>> not reachable at the moment. Here is my result :
>>
>>
>> 105 26115 0.2 273 1125328 755316 ? Sl Feb09 23:58
>> /usr/bin/java -ea -Xms64M -Xmx128M
>>
>>
>>
>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>
>>> bin/cassandra.in.sh
>>> set Xms and Xmx in the JVM_OPTS
>>>
>>> Aaron
>>>
>>>
>>>
>>> On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com>
>>> wrote:
>>>
>>>
>>> Is it possible to change the maximum JVM heap memory use in 0.6.X ?
>>>
>>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>>
>>>> What are you using for disk_access_mode ?
>>>> Have you tried reducing the JVM head size?
>>>> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to
>>>> lock the JVM memory.
>>>>
>>>>
>>>> Aaron
>>>>
>>>>
>>>>
>>>> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2011/2/16 Aaron Morton < <aa...@thelastpickle.com>
>>>> aaron@thelastpickle.com>
>>>>
>>>>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>>>>
>>>>> Memory mapped files will use additional virtual memory, is controlled
>>>>> in conf/Cassandra.yaml disk_access_mode
>>>>>
>>>>>
>>>> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
>>>> system in my case 4GB, here is a part of cassandra-env.sh:
>>>>
>>>> calculate_heap_size()
>>>> {
>>>> case "`uname`" in
>>>> Linux)
>>>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
>>>> return 0
>>>> ;;
>>>> FreeBSD)
>>>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print
>>>> $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
>>>> return 0
>>>> ;;
>>>> *)
>>>> MAX_HEAP_SIZE=1024M
>>>> return 1
>>>> ;;
>>>> esac
>>>> }
>>>>
>>>>
>>>>
>>>> I set all this options by default. All my nodes have 8GB of memory. And
>>>> i affraid that after some time all my nodes goes to hard swap, and only
>>>> reboot help them :-(((
>>>>
>>>> PS: as i understand that down sometime of cassandra is normal?
>>>>
>>>>
>>>
>>
>
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
Thanks Peter for the extra detail.
I thought there may have been something more mysterious going on. But it sounds like it was just the semantics of the term "use".
Cheers
Aaron
On 18/02/2011, at 9:25 PM, Peter Schuller <pe...@infidyne.com> wrote:
>> main argument for using mmap() instead of standard I/O is the fact
>> that reading entails just touching memory - in the case of the memory
>> being resident, you just read it - you don't even take a page fault
>> (so no overhead in entering the kernel and doing a semi-context
>> switch).
>
> Oh and in the case of Java/Cassandra, as Jonathan clued me in on
> earlier, there is also the issue that byte[] arrays are mandated to be
> zeroed when allocated which causes overhead typically because there
> has to be a loop[1] somewhere writing a bunch of zeroes in, that
> you're then just going to replace immediately. Mapping a file has no
> such implications as long as you read directly from the underlying
> direct ByteBuffer.
>
> [1] Not *necessarily*; a JVM could theoretically do byte[] allocations
> in such a way that it already knows the contents is zeroed, but it
> would be highly dependent on the GC/memory management technique used
> by the JVM whether this is practical. (It just occurred to me that
> Azul should get this for 'free' in their GC. Wonder if that's true.)
>
> --
> / Peter Schuller
Re: memory consuption
Posted by Peter Schuller <pe...@infidyne.com>.
> main argument for using mmap() instead of standard I/O is the fact
> that reading entails just touching memory - in the case of the memory
> being resident, you just read it - you don't even take a page fault
> (so no overhead in entering the kernel and doing a semi-context
> switch).
Oh and in the case of Java/Cassandra, as Jonathan clued me in on
earlier, there is also the issue that byte[] arrays are mandated to be
zeroed when allocated which causes overhead typically because there
has to be a loop[1] somewhere writing a bunch of zeroes in, that
you're then just going to replace immediately. Mapping a file has no
such implications as long as you read directly from the underlying
direct ByteBuffer.
[1] Not *necessarily*; a JVM could theoretically do byte[] allocations
in such a way that it already knows the contents is zeroed, but it
would be highly dependent on the GC/memory management technique used
by the JVM whether this is practical. (It just occurred to me that
Azul should get this for 'free' in their GC. Wonder if that's true.)
--
/ Peter Schuller
Re: memory consuption
Posted by Peter Schuller <pe...@infidyne.com>.
> Jonathan,
> When you get time could you please explain that a little more. Got a feeling
> I'm about to learn something :)
I'm not Jonathan, but: The operating system's virtual memory system
supports mapping files into a process' address space. This will "use"
virtual memory; i.e. address space. On 32 bit systems this was a
concern recently since running out of address space was a practical
concern; with 64 bit (even if you can't address the full 64 bit) this
is no longer an issue for a while - making virtual address space
essentially "free".
What matters from the perspective of "memory use" in the sense as it
is normally meant, is the amount of data allocated on brk():ed or
mmap():ed /dev/zero, which represent real memory used (or possibly
swap space, but unless the memory is never again accessed that's
usually not interesting from the point of view of "how much memory do
I need").
The key issue is that for a mmap():ed file, there is never a need to
retain the data in physical memory (=resident). Thus, whatever you do
keep resident in physical memory is essentially just there as a cache,
in the same way as normal I/O will cause the kernel page cache to
retain data that you read/write. The different between the normal I/O
and mmap() is that in the mmap() case the memory is actually mapped to
the process, thus affecting the virtual size as reported by top. The
main argument for using mmap() instead of standard I/O is the fact
that reading entails just touching memory - in the case of the memory
being resident, you just read it - you don't even take a page fault
(so no overhead in entering the kernel and doing a semi-context
switch).
A downside with mmap() is that you have less control over how I/O is
done (you can't say "read 60 MB from here", but instead traverse pages
hoping prefetch and/or read-ahead will help you; this can be mitigated
with posix_fadvise(), but then you're back to doing syscalls).
The other effect with mmap() is that it seems to affect the sense the
kernel has of the priority of different pages in terms of what to drop
or swap out, such that mmap() has a tendency to cause swapping out of
the JVM heap. But this is not because the process actually uses more
memory as such.
I didn't read it now but scrolling through it seems the wikipedia
article is a pretty good intro:
http://en.wikipedia.org/wiki/Virtual_memory
--
/ Peter Schuller
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
Jonathan,
When you get time could you please explain that a little more. Got a feeling I'm about to learn something :)
Thanks
Aaron
On 18 Feb, 2011,at 01:36 PM, Jonathan Ellis <jb...@gmail.com> wrote:
Please note that this won't affect the actual memory in use, only how
the OS reports it.
On Thu, Feb 17, 2011 at 6:15 PM, Aaron Morton <aa...@thelastpickle.com> wrote:
> Looks like you are using virtual memory for memmapped files. Change the
> disk_access_mode to standard if you want to reduce the overall memory
> usage.
> Aaron
> On 18 Feb, 2011,at 09:34 AM, Victor Kabdebon <vi...@gmail.com>
> wrote:
>
> Sorry I forgot to say that this is the partial result of :
> ps aux | grep cassandra
> Best regards
>
> 2011/2/17 Victor Kabdebon <vi...@gmail.com>
>>
>> Oh right but Cassandra doesn't really respect that, I thought there was
>> another option to set that.
>> Just for your information, I set xms and xmx very low with a small amount
>> of data. I am waiting to be able to connect jconsole, I don't know why it is
>> not reachable at the moment. Here is my result :
>>
>> 105 26115 0.2 273 1125328 755316 ? Sl Feb09 23:58
>> /usr/bin/java -ea -Xms64M -Xmx128M
>>
>>
>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>>
>>> bin/cassandra.in.sh
>>> set Xms and Xmx in the JVM_OPTS
>>> Aaron
>>>
>>>
>>> On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com>
>>> wrote:
>>>
>>>
>>> Is it possible to change the maximum JVM heap memory use in 06.X ?
>>>
>>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>>>
>>>> What are you using for disk_access_mode ?
>>>> Have you tried reducing the JVM head size?
>>>> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to
>>>> lock the JVM memory.
>>>>
>>>> Aaron
>>>>
>>>>
>>>> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2011/2/16 Aaron Morton <aa...@thelastpickle.com>
>>>>>
>>>>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>>>>
>>>>> Memory mapped files will use additional virtual memory, is controlled
>>>>> in conf/Cassandra.yaml disk_access_mode
>>>>>
>>>>
>>>> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
>>>> system in my case 4GB, here is a part of cassandra-env.sh:
>>>>
>>>> calculate_heap_size()
>>>> {
>>>> case "`uname`" in
>>>> Linux)
>>>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
>>>> return 0
>>>> ;;
>>>> FreeBSD)
>>>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print
>>>> $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
>>>> return 0
>>>> ;;
>>>> *)
>>>> MAX_HEAP_SIZE=1024M
>>>> return 1
>>>> ;;
>>>> esac
>>>> }
>>>>
>>>>
>>>>
>>>> I set all this options by default. All my nodes have 8GB of memory. And
>>>> i affraid that after some time all my nodes goes to hard swap, and only
>>>> reboot help them :-(((
>>>>
>>>> PS: as i understand that down sometime of cassandra is normal?
>>>
>>
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: memory consuption
Posted by Jonathan Ellis <jb...@gmail.com>.
Please note that this won't affect the actual memory in use, only how
the OS reports it.
On Thu, Feb 17, 2011 at 6:15 PM, Aaron Morton <aa...@thelastpickle.com> wrote:
> Looks like you are using virtual memory for memmapped files. Change the
> disk_access_mode to standard if you want to reduce the overall memory
> usage.
> Aaron
> On 18 Feb, 2011,at 09:34 AM, Victor Kabdebon <vi...@gmail.com>
> wrote:
>
> Sorry I forgot to say that this is the partial result of :
> ps aux | grep cassandra
> Best regards
>
> 2011/2/17 Victor Kabdebon <vi...@gmail.com>
>>
>> Oh right but Cassandra doesn't really respect that, I thought there was
>> another option to set that.
>> Just for your information, I set xms and xmx very low with a small amount
>> of data. I am waiting to be able to connect jconsole, I don't know why it is
>> not reachable at the moment. Here is my result :
>>
>> 105 26115 0.2 273 1125328 755316 ? Sl Feb09 23:58
>> /usr/bin/java -ea -Xms64M -Xmx128M
>>
>>
>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>>
>>> bin/cassandra.in.sh
>>> set Xms and Xmx in the JVM_OPTS
>>> Aaron
>>>
>>>
>>> On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com>
>>> wrote:
>>>
>>>
>>> Is it possible to change the maximum JVM heap memory use in 0.6.X ?
>>>
>>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>>>
>>>> What are you using for disk_access_mode ?
>>>> Have you tried reducing the JVM head size?
>>>> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to
>>>> lock the JVM memory.
>>>>
>>>> Aaron
>>>>
>>>>
>>>> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2011/2/16 Aaron Morton <aa...@thelastpickle.com>
>>>>>
>>>>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>>>>
>>>>> Memory mapped files will use additional virtual memory, is controlled
>>>>> in conf/Cassandra.yaml disk_access_mode
>>>>>
>>>>
>>>> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
>>>> system in my case 4GB, here is a part of cassandra-env.sh:
>>>>
>>>> calculate_heap_size()
>>>> {
>>>> case "`uname`" in
>>>> Linux)
>>>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
>>>> return 0
>>>> ;;
>>>> FreeBSD)
>>>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print
>>>> $2}'`
>>>> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
>>>> return 0
>>>> ;;
>>>> *)
>>>> MAX_HEAP_SIZE=1024M
>>>> return 1
>>>> ;;
>>>> esac
>>>> }
>>>>
>>>>
>>>>
>>>> I set all this options by default. All my nodes have 8GB of memory. And
>>>> i affraid that after some time all my nodes goes to hard swap, and only
>>>> reboot help them :-(((
>>>>
>>>> PS: as i understand that down sometime of cassandra is normal?
>>>
>>
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
Looks like you are using virtual memory for memmapped files. Change the disk_access_mode to standard if you want to reduce the overall memory usage.
Aaron
On 18 Feb, 2011,at 09:34 AM, Victor Kabdebon <vi...@gmail.com> wrote:
Sorry I forgot to say that this is the partial result of :
ps aux | grep cassandra
Best regards
2011/2/17 Victor Kabdebon <vi...@gmail.com>
Oh right but Cassandra doesn't really respect that, I thought there was another option to set that
Just for your information, I set xms and xmx very low with a small amount of data. I am waiting to be able to connect jconsole, I don't know why it is not reachable at the moment. Here is my result :
105 26115 0.2 27.3 1125328 755316 ? Sl Feb09 23:58 /usr/bin/java -ea -Xms64M -Xmx128M
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
bin/cassandra.in.sh
set Xms and Xmx in the JVM_OPTS
Aaron
On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com> wrote:
Is it possible to change the maximum JVM heap memory use in 0.6.X ?
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
What are you using for disk_access_mode ?
Have you tried reducing the JVM head size?
Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory.
Aaron
On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/16 Aaron Morton <aa...@thelastpickle.com>
JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in my case 4GB, here is a part of cassandra-env.sh:
calculate_heap_size()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
return 0
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
return 0
;;
*)
MAX_HEAP_SIZE=1024M
return 1
;;
esac
}
I set all this options by default. All my nodes have 8GB of memory. And i affraid that after some time all my nodes goes to hard swap, and only reboot help them :-(((
PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by Victor Kabdebon <vi...@gmail.com>.
Sorry I forgot to say that this is the partial result of :
ps aux | grep cassandra
Best regards
2011/2/17 Victor Kabdebon <vi...@gmail.com>
> Oh right but Cassandra doesn't really respect that, I thought there was
> another option to set that.
>
> Just for your information, I set xms and xmx very low with a small amount
> of data. I am waiting to be able to connect jconsole, I don't know why it is
> not reachable at the moment. Here is my result :
>
>
> 105 26115 0.2 27.3 1125328 755316 ? Sl Feb09 23:58
> /usr/bin/java -ea -Xms64M -Xmx128M
>
>
> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>
>> bin/cassandra.in.sh
>> set Xms and Xmx in the JVM_OPTS
>>
>> Aaron
>>
>>
>> On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com>
>> wrote:
>>
>> Is it possible to change the maximum JVM heap memory use in 0.6.X ?
>>
>> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>>
>>> What are you using for disk_access_mode ?
>>> Have you tried reducing the JVM head size?
>>> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to
>>> lock the JVM memory.
>>>
>>>
>>> Aaron
>>>
>>>
>>>
>>> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>
>>>
>>> 2011/2/16 Aaron Morton < <aa...@thelastpickle.com>
>>> aaron@thelastpickle.com>
>>>
>>>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>>>
>>>> Memory mapped files will use additional virtual memory, is controlled in
>>>> conf/Cassandra.yaml disk_access_mode
>>>>
>>>>
>>> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
>>> system in my case 4GB, here is a part of cassandra-env.sh:
>>>
>>> calculate_heap_size()
>>> {
>>> case "`uname`" in
>>> Linux)
>>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>>> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
>>> return 0
>>> ;;
>>> FreeBSD)
>>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
>>> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
>>> return 0
>>> ;;
>>> *)
>>> MAX_HEAP_SIZE=1024M
>>> return 1
>>> ;;
>>> esac
>>> }
>>>
>>>
>>>
>>> I set all this options by default. All my nodes have 8GB of memory. And i
>>> affraid that after some time all my nodes goes to hard swap, and only reboot
>>> help them :-(((
>>>
>>> PS: as i understand that down sometime of cassandra is normal?
>>>
>>>
>>
>
Re: memory consuption
Posted by Victor Kabdebon <vi...@gmail.com>.
Oh right but Cassandra doesn't really respect that, I thought there was
another option to set that.
Just for your information, I set xms and xmx very low with a small amount of
data. I am waiting to be able to connect jconsole, I don't know why it is
not reachable at the moment. Here is my result :
105 26115 0.2 27.3 1125328 755316 ? Sl Feb09 23:58
/usr/bin/java -ea -Xms64M -Xmx128M
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
> bin/cassandra.in.sh
> set Xms and Xmx in the JVM_OPTS
>
> Aaron
>
>
> On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com>
> wrote:
>
> Is it possible to change the maximum JVM heap memory use in 0.6.X ?
>
> 2011/2/17 Aaron Morton <aa...@thelastpickle.com>
>
>> What are you using for disk_access_mode ?
>> Have you tried reducing the JVM head size?
>> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to
>> lock the JVM memory.
>>
>>
>> Aaron
>>
>>
>>
>> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
>>
>>
>>
>>
>>
>> 2011/2/16 Aaron Morton < <aa...@thelastpickle.com>aaron@thelastpickle.com
>> >
>>
>>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>>
>>> Memory mapped files will use additional virtual memory, is controlled in
>>> conf/Cassandra.yaml disk_access_mode
>>>
>>>
>> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
>> system in my case 4GB, here is a part of cassandra-env.sh:
>>
>> calculate_heap_size()
>> {
>> case "`uname`" in
>> Linux)
>> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
>> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
>> return 0
>> ;;
>> FreeBSD)
>> system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
>> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
>> return 0
>> ;;
>> *)
>> MAX_HEAP_SIZE=1024M
>> return 1
>> ;;
>> esac
>> }
>>
>>
>>
>> I set all this options by default. All my nodes have 8GB of memory. And i
>> affraid that after some time all my nodes goes to hard swap, and only reboot
>> help them :-(((
>>
>> PS: as i understand that down sometime of cassandra is normal?
>>
>>
>
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
bin/cassandra.in.sh
set Xms and Xmx in the JVM_OPTS
Aaron
On 18 Feb, 2011,at 09:10 AM, Victor Kabdebon <vi...@gmail.com> wrote:
Is it possible to change the maximum JVM heap memory use in 0.6.X ?
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
What are you using for disk_access_mode ?
Have you tried reducing the JVM head size?
Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory.
Aaron
On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/16 Aaron Morton <aa...@thelastpickle.com>
JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in my case 4GB, here is a part of cassandra-env.sh:
calculate_heap_size()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
return 0
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
return 0
;;
*)
MAX_HEAP_SIZE=1024M
return 1
;;
esac
}
I set all this options by default. All my nodes have 8GB of memory. And i affraid that after some time all my nodes goes to hard swap, and only reboot help them :-(((
PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by Victor Kabdebon <vi...@gmail.com>.
Is it possible to change the maximum JVM heap memory use in 0.6.X ?
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
> What are you using for disk_access_mode ?
> Have you tried reducing the JVM head size?
> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock
> the JVM memory.
>
>
> Aaron
>
>
> On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
>
>
>
> 2011/2/16 Aaron Morton < <aa...@thelastpickle.com>
>
>> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>>
>> Memory mapped files will use additional virtual memory, is controlled in
>> conf/Cassandra.yaml disk_access_mode
>>
>>
> And??? JVM memory heap in cassandra 0.7 is by default half of memory is
> system in my case 4GB, here is a part of cassandra-env.sh:
>
> calculate_heap_size()
> {
> case "`uname`" in
> Linux)
> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
> return 0
> ;;
> FreeBSD)
> system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
> return 0
> ;;
> *)
> MAX_HEAP_SIZE=1024M
> return 1
> ;;
> esac
> }
>
>
>
> I set all this options by default. All my nodes have 8GB of memory. And i
> affraid that after some time all my nodes goes to hard swap, and only reboot
> help them :-(((
>
> PS: as i understand that down sometime of cassandra is normal?
>
>
Re: memory consuption
Posted by ruslan usifov <ru...@gmail.com>.
2011/2/17 Aaron Morton <aa...@thelastpickle.com>
> What are you using for disk_access_mode ?
>
I set config default: auto.
Have you tried reducing the JVM head size?
>
No i set default settings
> Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock
> the JVM memory.
>
> No, and what does this mean lock the JVM memory? Can you explain
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
Ruslan,
Change your disk_access_mode standard to remove the virtual memory usage.
Some info on JNA
http://www.datastax.com/blog/whats-new-cassandra-066
http://journal.paul.querna.org/articles/2010/11/11/enabling-jna-in-cassandra/
Aaron
On 18 Feb, 2011,at 09:00 AM, Aaron Morton <aa...@thelastpickle.com> wrote:
What are you using for disk_access_mode ?
Have you tried reducing the JVM head size?
Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory.
Aaron
On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/16 Aaron Morton <aa...@thelastpickle.com>
JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in my case 4GB, here is a part of cassandra-env.sh:
calculate_heap_size()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
return 0
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
return 0
;;
*)
MAX_HEAP_SIZE=1024M
return 1
;;
esac
}
I set all this options by default. All my nodes have 8GB of memory. And i affraid that after some time all my nodes goes to hard swap, and only reboot help them :-(((
PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
What are you using for disk_access_mode ?
Have you tried reducing the JVM head size?
Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory.
Aaron
On 17/02/2011, at 9:20 PM, ruslan usifov <ru...@gmail.com> wrote:
>
>
> 2011/2/16 Aaron Morton <aa...@thelastpickle.com>
> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>
> Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
>
>
> And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in my case 4GB, here is a part of cassandra-env.sh:
>
> calculate_heap_size()
> {
> case "`uname`" in
> Linux)
> system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
> MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
> return 0
> ;;
> FreeBSD)
> system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
> MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
> return 0
> ;;
> *)
> MAX_HEAP_SIZE=1024M
> return 1
> ;;
> esac
> }
>
>
>
> I set all this options by default. All my nodes have 8GB of memory. And i affraid that after some time all my nodes goes to hard swap, and only reboot help them :-(((
>
> PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by ruslan usifov <ru...@gmail.com>.
2011/2/16 Aaron Morton <aa...@thelastpickle.com>
> JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
>
> Memory mapped files will use additional virtual memory, is controlled in
> conf/Cassandra.yaml disk_access_mode
>
>
And??? JVM memory heap in cassandra 0.7 is by default half of memory is
system in my case 4GB, here is a part of cassandra-env.sh:
calculate_heap_size()
{
case "`uname`" in
Linux)
system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_mb / 2))M
return 0
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
MAX_HEAP_SIZE=$((system_memory_in_bytes / 1024 / 1024 / 2))M
return 0
;;
*)
MAX_HEAP_SIZE=1024M
return 1
;;
esac
}
I set all this options by default. All my nodes have 8GB of memory. And i
affraid that after some time all my nodes goes to hard swap, and only reboot
help them :-(((
PS: as i understand that down sometime of cassandra is normal?
Re: memory consuption
Posted by Aaron Morton <aa...@thelastpickle.com>.
JVM heap memory is controlled by the settings in conf/Cassandra-env.sh
Memory mapped files will use additional virtual memory, is controlled in conf/Cassandra.yaml disk_access_mode
Aaron
On 17/02/2011, at 2:19 AM, ruslan usifov <ru...@gmail.com> wrote:
> Hello
>
> Here is my output from ps aux:
>
> root 737 0.0 0.0 8940 804 ? S 04:34 0:00 nimbus(hdb)
> root 768 0.0 0.0 0 0 ? S 04:34 0:00 [flush-202:0]
> root 1368 43.1 70.5 8248976 5935852 ? Sl 04:45 297:47 /usr/local/jdk1.6.0_23/bin/java -ea -XX:+UseThreadPriorities -XX:Thr <<<<<< cassandra daemon
> root 1633 0.0 0.0 70584 3212 ? Ss 15:49 0:00 sshd: ruslan [priv]
>
>
> Is it normal that cassandra daemon eat so many memory? I look on it around 12 hours and memory consumption is only grows.
>
>
> ps:
> uname -a
> Linux slv004 2.6.32-311-ec2 #23-Ubuntu SMP Thu Dec 2 11:14:35 UTC 2010 x86_64 GNU/Linux
>
> java -version
> java version "1.6.0_23"
> Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
> Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
>
>
> My working machine is 8 CPU 8GB of RAM