You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Amit Sinha <am...@gmail.com> on 2010/07/22 00:37:25 UTC
Cassandra committing massive virtual memory
We are seeing cassandra using very high virtual memory. One server in the
cluster shows 90GB and the other shows about 70GB of committed virtual
memory.
The real memory used is less than 1GB. The Xmx is 4GB. The physical memory
on the machine is 32GB and the swap space on the machine is around 20GB.
I understand that the virtual memory is not really used by the process and
is just reserved. My question is why does cassandra need to reserve such
huge memory. What is the logic that is used internally. How is it related to
usage pattern or data storage pattern. Should we be concerned?
thanks
Amit
Re: Cassandra committing massive virtual memory
Posted by Peter Schuller <pe...@infidyne.com>.
> We are seeing cassandra using very high virtual memory. One server in the
> cluster shows 90GB and the other shows about 70GB of committed virtual
> memory.
> The real memory used is less than 1GB. The Xmx is 4GB. The physical memory
> on the machine is 32GB and the swap space on the machine is around 20GB.
>
> I understand that the virtual memory is not really used by the process and
> is just reserved. My question is why does cassandra need to reserve such
> huge memory. What is the logic that is used internally. How is it related to
> usage pattern or data storage pattern. Should we be concerned?
Virtual size in and of itself is not a concern because it is mmap():ed
address space. AFAIK you may see sizes up to the entire database size.
What people have had issues with is swapping resulting from the
operating system considering mmap():ed area contributing to memory
pressure and swapping out the Java heap. But unless you have issues
like that, I don't believe the virtual address space size in and of
itself should be a concern.
--
/ Peter Schuller
Re: Cassandra committing massive virtual memory
Posted by Amit Sinha <am...@gmail.com>.
The DiskAccessMode is set to auto.
I am going to try with standard and see, but i am concerned how much that
would affect performance negatively.
thx
Amit
On Wed, Jul 21, 2010 at 3:59 PM, Aaron Morton <aa...@thelastpickle.com>wrote:
> Are you using mmap or auto DiskAccessMode ?
>
> There is a known issue with memory mapped file access taking up a lot of
> memory. See CASSANDRA-1214
> https://issues.apache.org/jira/browse/CASSANDRA-1214 there is also some
> discussion in the mail list here.
>
> Try setting the DiskAccessMode to standard
>
> Aaron
>
>
> On 22 Jul, 2010,at 10:37 AM, Amit Sinha <am...@gmail.com> wrote:
>
> We are seeing cassandra using very high virtual memory. One server in the
> cluster shows 90GB and the other shows about 70GB of committed virtual
> memory.
> The real memory used is less than 1GB. The Xmx is 4GB. The physical memory
> on the machine is 32GB and the swap space on the machine is around 20GB.
>
> I understand that the virtual memory is not really used by the process and
> is just reserved. My question is why does cassandra need to reserve such
> huge memory. What is the logic that is used internally. How is it related to
> usage pattern or data storage pattern. Should we be concerned?
>
> thanks
> Amit
>
>
>
>
--
thanks
Amitabh
Re: Cassandra committing massive virtual memory
Posted by Aaron Morton <aa...@thelastpickle.com>.
Are you using mmap or auto DiskAccessMode ?
There is a known issue with memory mapped file access taking up a lot of memory. See CASSANDRA-1214 https://issues.apache.org/jira/browse/CASSANDRA-1214 there is also some discussion in the mail list here.
Try setting the DiskAccessMode to standard
Aaron
On 22 Jul, 2010,at 10:37 AM, Amit Sinha <am...@gmail.com> wrote:
> We are seeing cassandra using very high virtual memory. One server in the cluster shows 90GB and the other shows about 70GB of committed virtual memory.
> The real memory used is less than 1GB. The Xmx is 4GB. The physical memory on the machine is 32GB and the swap space on the machine is around 20GB.
>
> I understand that the virtual memory is not really used by the process and is just reserved My question is why does cassandra need to reserve such huge memory. What is the logic that is used internally. How is it related to usage pattern or data storage pattern. Should we be concerned?
>
> thanks
> Amit
>
>
>