svn commit: r1079606 - in /hadoop/common/branches/branch-0.22: CHANGES.txt bin/hadoop-config.sh

[to...@apache.org]

Author: todd
Date: Tue Mar  8 23:51:40 2011
New Revision: 1079606

URL: http://svn.apache.org/viewvc?rev=1079606&view=rev
Log:
HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh. Contributed by Todd Lipcon.

Modified:
    hadoop/common/branches/branch-0.22/CHANGES.txt
    hadoop/common/branches/branch-0.22/bin/hadoop-config.sh

Modified: hadoop/common/branches/branch-0.22/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/CHANGES.txt?rev=1079606&r1=1079605&r2=1079606&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/CHANGES.txt (original)
+++ hadoop/common/branches/branch-0.22/CHANGES.txt Tue Mar  8 23:51:40 2011
@@ -221,6 +221,8 @@ Release 0.22.0 - Unreleased
  
     HADOOP-6970. SecurityAuth.audit should be generated under /build (boryas)
 
+    HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh (todd)
+
   OPTIMIZATIONS
 
     HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..).

Modified: hadoop/common/branches/branch-0.22/bin/hadoop-config.sh
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/bin/hadoop-config.sh?rev=1079606&r1=1079605&r2=1079606&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/bin/hadoop-config.sh (original)
+++ hadoop/common/branches/branch-0.22/bin/hadoop-config.sh Tue Mar  8 23:51:40 2011
@@ -78,6 +78,11 @@ then
   exit 1
 fi
 
+# Newer versions of glibc use an arena memory allocator that causes virtual
+# memory usage to explode. This interacts badly with the many threads that
+# we use in Hadoop. Tune the variable down to prevent vmem explosion.
+export MALLOC_ARENA_MAX=${MALLOC_ARENA_MAX:-4}
+
 # some Java parameters
 if [ "$JAVA_HOME" != "" ]; then
   #echo "run java in $JAVA_HOME"