[jira] Commented: (HADOOP-2888) Enhancements to gridmix scripts

["Chris Douglas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575549#action_12575549 ] 

Chris Douglas commented on HADOOP-2888:
---------------------------------------

While this is being updated, it might make sense to change the defaults away from 0.15.2-dev builds of the test, streaming, and example jars, probably to hadoop-*-test.jar, hadoop-*-examples.jar, etc. as in:

{noformat}
# set var only if it has not already been set externally
if [ -z "${EXAMPLE_JAR}" ] ; then
  export EXAMPLE_JAR="${HADOOP_HOME}/hadoop-*-examples.jar"
fi
{noformat}

Also, since the USE_REAL_DATASET change, the comments describing the generated data no longer match the adjacent var; these might be more readable if each were moved within the "real" block.

+1 overall

> Enhancements to gridmix scripts
> -------------------------------
>
>                 Key: HADOOP-2888
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2888
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.17.0
>            Reporter: Mukund Madhugiri
>            Assignee: Mukund Madhugiri
>             Fix For: 0.17.0
>
>         Attachments: gridmix_patch, gridmix_patch
>
>
> I would like to propose enhancements to the gridmix scripts to make it:
> 1. easier to setup parameters for the test run and data generation (makes it easier to automate the runs using something like hudson)
> 2. ensure the benchmarks wait until they are completed (makes it easier to automate the runs using something like hudson)
> Here are the details:
> Ability to override these parameters in gridmix-env
> * HADOOP_HOME
> * GRID_MIX_HOME
> * EXAMPLE_JAR
> * APP_JAR
> * STREAM_JAR
> * GRID_MIX_DATA
> * GRID_MIX_PROG
> Ability to override these parameters in generateData.sh
> * COMPRESSED_DATA_BYTES
> * UNCOMPRESSED_DATA_BYTES
> * INDIRECT_DATA_BYTES
> Ability for the tests submitted to the same cluster to wait until they are done. Changes will be in:
> * submissionScripts/monsterQueriesToSameCluster
> * submissionScripts/maxentToSameCluster
> * submissionScripts/textSortToSameCluster
> * submissionScripts/webdataScanToSameCluster
> * submissionScripts/webdataSortToSameCluster

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.