hadoop distcp container killed

[adu <du...@hzduozhun.com>]

hi all,

I use distcp to copy a large file (20g+) in two clusters. I got the error.

Container [pid=12876,containerID=container_1411625661257_0156_01_000002]
is running beyond virtual memory limits. Current usage: 157.6 MB of 1 GB
physical memory used; 12.1 GB of 10 GB virtual memory used. Killing
container.
Dump of the process-tree for container_1411625661257_0156_01_000002 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 12881 12876 12876 12876 (java) 891 65 12923498496 40050
/usr/java/latest/bin/java -Djava.net.preferIPv4Stack=true
-Dhadoop.metrics.log.level=WARN -Xmx2048m
-Djava.io.tmpdir=/alidata1/data/
hdfs/node_manager/usercache/root/appcache/application_1411625661257_0156/container_1411625661257_0156_01_000002/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir=/
alidata1/data/hdfs/nodemanager_log/application_1411625661257_0156/container_1411625661257_0156_01_000002
-Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
org.apache.hadoop.mapred. YarnChild 10.162.39.154 49876
attempt_1411625661257_0156_m_000000_0 2
|- 12876 15020 12876 12876 (bash) 0 0 108646400 299 /bin/bash -c
/usr/java/latest/bin/java -Djava.net.preferIPv4Stack=true
-Dhadoop.metrics.log.level=WARN -Xmx2048m -Djava.io.tmpdir=/alidata1/
data/hdfs/node_manager/usercache/root/appcache/application_1411625661257_0156/container_1411625661257_0156_01_000002/tmp
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.container.log.dir=/
alidata1/data/hdfs/nodemanager_log/application_1411625661257_0156/container_1411625661257_0156_01_000002
-Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
org.apache.hadoop.mapred. YarnChild 10.162.39.154 49876
attempt_1411625661257_0156_m_000000_0 2
1>/alidata1/data/hdfs/nodemanager_log/application_1411625661257_0156/container_1411625661257_0156_01_000002/stdout
2>/alidata1/
data/hdfs/nodemanager_log/application_1411625661257_0156/container_1411625661257_0156_01_000002/stderr

Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143


The source hadoop version is 2.2.0 . and the destination is 2.4.1. I
test -m (50. 100. 100) to increase the number of map task. But it
doesn't work. It seems the file isn't spited into small parts.

I need help , Thanks.