You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Panagiotis Atmatzidis <at...@convalesco.org> on 2016/03/02 20:08:51 UTC
hdfs swapping
Hello,
I have a hadoop cluster where each server has 128 GB RAM. I noticed that in some servers hdfs (java process) swaps. To give an idea:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
$ for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done|grep java
java 14352 kB
java 39956 kB
java 148416 kB
java 16848 kB
$ free -m
total used free shared buffers cached
Mem: 128835 128181 653 11 105 90162
-/+ buffers/cache: 37913 90921
Swap: 511 233 278
$ vmstat -w -S M
procs ---------------memory-------------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 233 673 105 90140 0 0 863 1644 0 0 6 2 91 1 0
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
In cloudera’s documentation says[1] that swappiness should be setup to ‘0’ while a more recent blogpost[1] says it’s safer to set it up to ‘1’.
Are there any recommendations and/or best practices? I’d like to know what other people use in their clusters.
According to this SO[3] thread if a java app is swapping, there’s not much I can do. In my case there’s lots of extra RAM, are there any other options I missed, I can tweak to avoid such behaviour while keeping swampiness to ‘1’?
Thanks,
[1] https://www.cloudera.com/documentation/enterprise/5-2-x/topics/cdh_admin_performance.html#xd_583c10bfdbd326ba-7dae4aa6-147c30d0933--7fd5__section_ovb_wcy_kq <https://www.cloudera.com/documentation/enterprise/5-2-x/topics/cdh_admin_performance.html#xd_583c10bfdbd326ba-7dae4aa6-147c30d0933--7fd5__section_ovb_wcy_kq>
[2] http://blog.cloudera.com/blog/2015/01/how-to-deploy-apache-hadoop-clusters-like-a-boss/ <http://blog.cloudera.com/blog/2015/01/how-to-deploy-apache-hadoop-clusters-like-a-boss/>
[3] http://stackoverflow.com/a/9306054
Panagiotis (atmosx) Atmatzidis
email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0x1A7BFEC5
gpg --keyserver pgp.mit.edu --recv-keys 1A7BFEC5
"Everyone thinks of changing the world, but no one thinks of changing himself.” - Leo Tolstoy