[jira] Commented: (WHIRR-227) CDH and Hadoop integration tests are failing

["Tibor Kiss (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/WHIRR-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989667#comment-12989667 ] 

Tibor Kiss commented on WHIRR-227:
----------------------------------

I am following this bug and I see that changing the short rolenames can be the problem because on the end of 
http://whirr.s3.amazonaws.com/0.4.0-incubating-SNAPSHOT/apache/hadoop/post-configure 
there are the old short rolenames!

{code}

for role in $(echo "$ROLES" | tr "," "\n"); do
  case $role in
  nn)
    setup_web
    start_namenode
    ;;
  snn)
    start_daemon secondarynamenode
    ;;
  jt)
    start_daemon jobtracker
    ;;
  dn)
    start_daemon datanode
    ;;
  tt)
    start_daemon tasktracker
    ;;
  esac
done
{code}

Here is what I see in the node when I am running manually.... does nothing at the end!
{code}
[root@domU-12-31-39-0E-CD-63 computeserv]# ./post-configure hadoop-namenode,hadoop-jobtracker -n ec2-184-73-150-247.compute-1.amazonaws.com -j ec2-184-73-150-247.compute-1.amazonaws.com -c ec2
+ set -e
+ ROLES=hadoop-namenode,hadoop-jobtracker
+ shift
+ NN_HOST=
+ JT_HOST=
+ CLOUD_PROVIDER=
+ getopts n:j:c: OPTION
+ case $OPTION in
+ NN_HOST=ec2-184-73-150-247.compute-1.amazonaws.com
+ getopts n:j:c: OPTION
+ case $OPTION in
+ JT_HOST=ec2-184-73-150-247.compute-1.amazonaws.com
+ getopts n:j:c: OPTION
+ case $OPTION in
+ CLOUD_PROVIDER=ec2
+ getopts n:j:c: OPTION
+ case $CLOUD_PROVIDER in
++ wget -q -O - http://169.254.169.254/latest/meta-data/public-hostname
+ SELF_HOST=ec2-184-73-150-247.compute-1.amazonaws.com
+ HADOOP_VERSION=0.20.2
+ HADOOP_HOME=/usr/local/hadoop-0.20.2
+ HADOOP_CONF_DIR=/usr/local/hadoop-0.20.2/conf
+ configure_hadoop
+ case $CLOUD_PROVIDER in
+ MOUNT=/mnt
+ FIRST_MOUNT=/mnt
+ DFS_NAME_DIR=/mnt/hadoop/hdfs/name
+ FS_CHECKPOINT_DIR=/mnt/hadoop/hdfs/secondary
+ DFS_DATA_DIR=/mnt/hadoop/hdfs/data
+ MAPRED_LOCAL_DIR=/mnt/hadoop/mapred/local
+ MAX_MAP_TASKS=2
+ MAX_REDUCE_TASKS=1
+ CHILD_OPTS=-Xmx550m
+ CHILD_ULIMIT=1126400
+ mkdir -p /mnt/hadoop
+ chown hadoop:hadoop /mnt/hadoop
+ '[' '!' -e /mnt/tmp ']'
+ mkdir /etc/hadoop
+ ln -s /usr/local/hadoop-0.20.2/conf /etc/hadoop/conf
+ cat
+ sed -i -e 's|# export HADOOP_PID_DIR=.*|export HADOOP_PID_DIR=/var/run/hadoop|' /usr/local/hadoop-0.20.2/conf/hadoop-env.sh
+ mkdir -p /var/run/hadoop
+ chown -R hadoop:hadoop /var/run/hadoop
+ sed -i -e 's|# export HADOOP_SSH_OPTS=.*|export HADOOP_SSH_OPTS="-o StrictHostKeyChecking=no"|' /usr/local/hadoop-0.20.2/conf/hadoop-env.sh
+ sed -i -e 's|# export HADOOP_OPTS=.*|export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true"|' /usr/local/hadoop-0.20.2/conf/hadoop-env.sh
+ sed -i -e 's|# export HADOOP_LOG_DIR=.*|export HADOOP_LOG_DIR=/var/log/hadoop/logs|' /usr/local/hadoop-0.20.2/conf/hadoop-env.sh
+ rm -rf /var/log/hadoop
+ mkdir /mnt/hadoop/logs
+ chown hadoop:hadoop /mnt/hadoop/logs
+ ln -s /mnt/hadoop/logs /var/log/hadoop
+ chown -R hadoop:hadoop /var/log/hadoop
++ echo hadoop-namenode,hadoop-jobtracker
++ tr , '\n'
+ for role in '$(echo "$ROLES" | tr "," "\n")'
+ case $role in
+ for role in '$(echo "$ROLES" | tr "," "\n")'
+ case $role in
{code}

> CDH and Hadoop integration tests are failing 
> ---------------------------------------------
>
>                 Key: WHIRR-227
>                 URL: https://issues.apache.org/jira/browse/WHIRR-227
>             Project: Whirr
>          Issue Type: Bug
>            Reporter: Andrei Savu
>            Priority: Critical
>             Fix For: 0.4.0
>
>
> I have tried multiple times (even using different internet connections and cloud providers) to run the integration tests for cdh and hadoop and they always fail with the same error message:
> {code}
> -------------------------------------------------------------------------------
> Test set: org.apache.whirr.service.cdh.integration.CdhHadoopServiceTest
> -------------------------------------------------------------------------------
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 336.63 sec <<< FAILURE!
> test(org.apache.whirr.service.cdh.integration.CdhHadoopServiceTest)  Time elapsed: 336.53 sec  <<< ERROR!
> java.io.IOException: Call to ec2-50-16-169-138.compute-1.amazonaws.com/50.16.169.138:8021 failed on local exception: java.net.SocketException: Malformed reply from SOCKS server
>   at org.apache.hadoop.ipc.Client.wrapException(Client.java:1089)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1057)
>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>   at org.apache.hadoop.mapred.$Proxy76.getProtocolVersion(Unknown Source)
>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:369)
>   at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:486)
>   at org.apache.hadoop.mapred.JobClient.init(JobClient.java:471)
>   at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:456)
>   at org.apache.whirr.service.cdh.integration.CdhHadoopServiceTest.test(CdhHadoopServiceTest.java:87)
> {code}
> I believe this is somehow related to one of the recently committed patches.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira