You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by nokia ceph <no...@gmail.com> on 2017/03/16 11:37:37 UTC
Cassandra + NUMA integration
Hello,
Recently we tried top integrate cassandra with NUMA
==
commented out the numactl --interleave in bin/cassandra and add
-XX:+UseNUMA to cassandra-env.sh.
==
But after inplementing, we saw HEAP and STACK values were changed to null.
#/usr/bin/numactl --cpunodebind=1 -- /usr/sbin/cassandra -p
/run/cassandra/cassandra.pid
#numastat -s cassandra
Per-node process memory usage (in MBs) for PID 13258 (java)
Node 0 Node 1 Total
--------------- --------------- ---------------
*Private 4205.18 4508.09* 8713.28
Stack 0.00 219.61 219.61
Heap 0.00 30.52 30.52
Huge 0.00 0.00 0.00
---------------- --------------- --------------- ---------------
Total 4205.18 4758.23 8963.41
Here you can see both Node 0 and Node 1 utilising 4GB. As per our
expectation NODE 0 Private should be zero. I
Is there any way to specify NODE-0 Private value change to 0?
Env,
RHEL7.3
3.10.0-514.2.2.el7.x86_64
cassandra30-3.0.6-1.noarch
Thanks
Re: Cassandra + NUMA integration
Posted by nokia ceph <no...@gmail.com>.
Hello,
In addition to my previous comment.
Regarding the NUMA integration with Cassadra, we configured NODE-1 only
utilize the resources. However after enabling this functionality, we
noticed on NODE-1 Private allocated to 4G .
What we did
<<
#tail -n1 /etc/cassandra/conf/cassandra-env.sh
JVM_OPTS="$JVM_OPTS -XX:+UseNUMA"
#grep -A3 numactl /usr/sbin/cassandra
# If numactl is available, use it. For Cassandra, the priority is to
# avoid disk I/O. Even for the purpose of CPU efficiency, we don't
# really have CPU<->data affinity anyway. Also, empirically test that
numactl
# works before trying to use it (CASSANDRA-3245).
#NUMACTL_ARGS="--interleave=all" ===>>> Here we commented this parameter.
After triggering below command
#/usr/bin/numactl --cpunodebind=1 -- /usr/sbin/cassandra -p
/run/cassandra/cassandra.pid
We noticed on NODE-1 Stack,Heap,Huge values changed to null .
We need to know why Private on NODE-0 utilizing memory even after we routed
traffic to NODE-1?
Please let us know do we miss anything here?
Can anyone please explain what below parameters will do as per numactl
output.
---
*Private*
*Stack*
*Heap*
*Huge*
*--*
//
# numastat -s cassandra
Per-node process memory usage (in MBs) for PID 5732 (java)
Node 0 Node 1 Total
--------------- --------------- ---------------
Private 4205.57 4570.72 8776.30
Stack 0.00 210.09 210.09
Heap 0.00 32.59 32.59
Huge 0.00 0.00 0.00
---------------- --------------- --------------- ---------------
Total 4205.57 4813.40 9018.98
Env,
RHEL7.3
3.10.0-514.2.2.el7.x86_64
cassandra30-3.0.6-1.noarch
Awaiting for comments
Thanks
On Thu, Mar 16, 2017 at 5:07 PM, nokia ceph <no...@gmail.com>
wrote:
> Hello,
>
> Recently we tried top integrate cassandra with NUMA
>
> ==
> commented out the numactl --interleave in bin/cassandra and add
> -XX:+UseNUMA to cassandra-env.sh.
> ==
>
> But after inplementing, we saw HEAP and STACK values were changed to null.
>
> #/usr/bin/numactl --cpunodebind=1 -- /usr/sbin/cassandra -p
> /run/cassandra/cassandra.pid
> #numastat -s cassandra
>
> Per-node process memory usage (in MBs) for PID 13258 (java)
> Node 0 Node 1 Total
> --------------- --------------- ---------------
> *Private 4205.18 4508.09* 8713.28
> Stack 0.00 219.61 219.61
> Heap 0.00 30.52 30.52
> Huge 0.00 0.00 0.00
> ---------------- --------------- --------------- ---------------
> Total 4205.18 4758.23 8963.41
>
>
> Here you can see both Node 0 and Node 1 utilising 4GB. As per our
> expectation NODE 0 Private should be zero. I
>
> Is there any way to specify NODE-0 Private value change to 0?
>
> Env,
>
> RHEL7.3
> 3.10.0-514.2.2.el7.x86_64
> cassandra30-3.0.6-1.noarch
>
> Thanks
>
>