[jira] Commented: (MAPREDUCE-1713) Utilities for system tests specific.

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864857#action_12864857 ] 

Konstantin Boudnik commented on MAPREDUCE-1713:
-----------------------------------------------

There was a cross-posting of these comment, so repeating them to make sure they are not missed:
 - use {{$(TO_DO_GLOBAL_TMP_DIR)/localconf}}
instead of
+  {{<value>$HADOOP_HOME/conf/localconf</value>}}
because $HADOOP_HOME mighe have root only writable permissions in some setups.

- The value of this variable is _wrong_.
{noformat}
+  public final static String CONF_HADOOP_LOCAL_DIR =
+    "test.system.hadoop.local.confdir";
{noformat}
It is suppose to be 
{{test.system.hdrc.hadoop.local.confdir}} according to the naming convention and the name in the system-test.xml file

 - also this code seems very enefficient:
{noformat}
+  public void waitForClusterToStart() throws 
+      Exception {
+    int counter;
+    for (List<AbstractDaemonClient> set : daemons.values()) {
+      for (AbstractDaemonClient daemon : set) {
+        counter = 0;
+        while (counter < 30) {
{noformat}
on a cluster of like 10 nodes it takes about 5 minutes to restart a cluster (RPC tries to reconnect to a remove server for about 50 seconds or so). It is too much. Instead of doing this sequentially you need to start a number of threads (one per a daemon) which will be waiting for them to come up. All wait can be done in about a minute or so instead of 1 min x # nodes.


> Utilities for system tests specific.
> ------------------------------------
>
>                 Key: MAPREDUCE-1713
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1713
>             Project: Hadoop Map/Reduce
>          Issue Type: Task
>          Components: test
>            Reporter: Vinay Kumar Thota
>            Assignee: Vinay Kumar Thota
>         Attachments: 1713-ydist-security.patch, systemtestutils_MR1713.patch, utilsforsystemtest_1713.patch
>
>
> 1.  A method for restarting  the daemon with new configuration.
>       public static  void restartCluster(Hashtable<String,Long> props, String confFile) throws Exception;
> 2.  A method for resetting the daemon with default configuration.
>       public void resetCluster() throws Exception;
> 3.  A method for waiting until daemon to stop.
>       public  void waitForClusterToStop() throws Exception;
> 4.  A method for waiting until daemon to start.
>       public  void waitForClusterToStart() throws Exception;
> 5.  A method for checking the job whether it has started or not.
>       public boolean isJobStarted(JobID id) throws IOException;
> 6.  A method for checking the task whether it has started or not.
>       public boolean isTaskStarted(TaskInfo taskInfo) throws IOException;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.