You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Andrei Savu (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 12:02:32 UTC
[jira] [Created] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Let users specify the CDH release (cdh3u1, cdh3u2)
--------------------------------------------------
Key: WHIRR-415
URL: https://issues.apache.org/jira/browse/WHIRR-415
Project: Whirr
Issue Type: Bug
Components: service/cdh
Reporter: Andrei Savu
Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152812#comment-13152812 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
Now working on this. I'm trying also to find the root cause for WHIRR-427.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Heneveld updated WHIRR-415:
--------------------------------
Attachment: WHIRR-415-whirr.env.patch
revised patch uses *{{whirr.}}*{{env.XXX}} syntax, and tests an explicit version separately with explicit CDH versions in pom ... though integration test ultimately still fails due to WHIRR-427
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146989#comment-13146989 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
Personally I don't like overloading {{tarball.url}} in this way.
What if we had a more general mechanism for allowing environment variables to be specified in the config file?
I am thinking something like:
{code}
env.someVar=value
{code}
Then {{SOME_VAR=value}} is exported and available to the scripts, so we could say {{env.repo=cdh3u1}} to solve this problem.
Interested to know people's thoughts, if this is a good idea, if {{env.SOME_VAR}} syntax is more intuitive (and if this should be a separate jira).
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-415:
------------------------------
Fix Version/s: 0.7.0
It shouldn't be that hard to address this as part of 0.7.0.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Adrian Cole (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-415:
------------------------------
Priority: Critical (was: Major)
Issue Type: Improvement (was: Bug)
marking this as critical as it is very important to paul
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Heneveld updated WHIRR-415:
--------------------------------
Attachment: WHIRR-415-env.patch
This attachment ({{WHIRR-415-env}} implements the {{env.someVar}} syntax above, with unit test, and with a corresponding integraton test for {{cdh3u1}} (although I am having trouble getting any of the CDH integration tests to pass so not sure if this integration test works, although I have confirmed it is installing the correct 3u1 version).
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152882#comment-13152882 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
PS: integration tests working as expected on aws-ec2 now testing on cloudservers.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Paul Baclace (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146472#comment-13146472 ]
Paul Baclace commented on WHIRR-415:
------------------------------------
This is essential for automation reasons. Whirr cannot be used reproducibly unless the launched bits are known from prior testing.
What is needed to get this fixed?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152887#comment-13152887 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
Also working fine with cloudservers. +1 from me. We can address any comments from Tom regarding adding the ability to set arbitrary environment variables as part of 0.8.0.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152994#comment-13152994 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
Patch looks sensible, might not have the chance to test it for a few days.
Combining version test with regular test is simpler, although my thinking was that it was better to have integration test which tests a specific version which is testably distinct from what you get when you request the latest.
Although that said, I was (with all the WHIRR-427 errors) wondering whether we might hit version errors at the pom level -- locally we are locked to one version for the project, so could be problematic testing multiple versions on the other end if they aren't compatible with the local one.
So fine either way.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-415) Let users specify the CDH
release (cdh3u1, cdh3u2)
Posted by "Paul Baclace (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146561#comment-13146561 ]
Paul Baclace edited comment on WHIRR-415 at 11/8/11 9:01 PM:
-------------------------------------------------------------
setting:
export REPO=cdh3u1
and then running whirr 0.6.0 was insufficient, at least for Debian flavor. This must be on the right track, however.
was (Author: pbaclace):
setting:
export REPO=cdh3u1
and then running whirr was insufficient, at least for Debian flavor. This must be on the right track, however.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by Andrei Savu <sa...@gmail.com>.
On Wed, Nov 9, 2011 at 5:20 AM, Paul Baclace <pa...@gmail.com> wrote:
> I can try it out by setting REPO under $WHIRR_HOME/functions/... by
> overriding all cdh scripts, but that seems crude.
Can you give it a try?
How does one set an env var that is defined before any scripts are run?
There is no generic mechanism for this.
Re: [jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by Paul Baclace <pa...@gmail.com>.
Perhaps my test did not work because env that runs whirr is not the same
one that runs the *_cdh*.sh scripts on the target. I don't see where
REPO is normally set. I can try it out by setting REPO under
$WHIRR_HOME/functions/... by overriding all cdh scripts, but that seems
crude. How does one set an env var that is defined before any scripts
are run?
Paul
On 20111108 12:55 , Paul Baclace (Commented) (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146561#comment-13146561 ]
>
> Paul Baclace commented on WHIRR-415:
> ------------------------------------
>
> setting:
>
> export REPO=cdh3u1
>
> and then running whirr was insufficient, at least for Debian flavor. This must be on the right track, however.
>
>
>> Let users specify the CDH release (cdh3u1, cdh3u2)
>> --------------------------------------------------
>>
>> Key: WHIRR-415
>> URL: https://issues.apache.org/jira/browse/WHIRR-415
>> Project: Whirr
>> Issue Type: Improvement
>> Components: service/cdh
>> Reporter: Andrei Savu
>> Priority: Critical
>> Fix For: 0.7.0
>>
>>
>> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Paul Baclace (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146561#comment-13146561 ]
Paul Baclace commented on WHIRR-415:
------------------------------------
setting:
export REPO=cdh3u1
and then running whirr was insufficient, at least for Debian flavor. This must be on the right track, however.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu reassigned WHIRR-415:
---------------------------------
Assignee: Alex Heneveld
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Assignee: Alex Heneveld
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147111#comment-13147111 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
See WHIRR-427 for failing CDH integration tests.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153144#comment-13153144 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
+1
Had the chance to review patch, looks good, and integration tests all passing for me on AWS. Hurrah!
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147318#comment-13147318 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
I think it makes sense to have the ability to add arbitrary environment variables in the .properties file (e.g. whirr.env.REPO=chd3u1) but I would add this in another JIRA. For this one I would go for something like whirr.cdh.repo=cdh3u1.
Guys what do you think if we move this to 0.8.0 (next release) and only implement env vars because that feature can be used as a workaround so that Paul can do his work as needed? Tom why do you worry about setting arbitrary env variables?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-415) Let users specify the CDH
release (cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146533#comment-13146533 ]
Alex Heneveld edited comment on WHIRR-415 at 11/8/11 8:25 PM:
--------------------------------------------------------------
>From the code it looks like the {{functions/\*\_cdh\_\*.sh}} pay attention to the environment variable {{REPO}} which defaults to {{cdh3}} (version {{3}}, symlinked to {{3u2}}; other versions listed e.g. at [http://archive.cloudera.com/redhat/cdh/]).
Does it work to, say force 3u1, if you *{{export REPO=cdh3u1}}* before whirring?
was (Author: alex.heneveld):
From the code it looks like the {{functions/*_cdh_*.sh}} pay attention to the environment variable {{REPO}} which defaults to {{cdh3}} (version {{3}}, symlinked to {{3u2}}; other versions listed e.g. at [http://archive.cloudera.com/redhat/cdh/]).
Does it work to, say force 3u1, if you {{export REPO=cdh3u1}} before whirring?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146366#comment-13146366 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
Tom can you point me to some relevant documentation pages?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Paul Baclace (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139477#comment-13139477 ]
Paul Baclace commented on WHIRR-415:
------------------------------------
This issue should be Priority==Critical.
Really.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Bug
> Components: service/cdh
> Reporter: Andrei Savu
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150019#comment-13150019 ]
Andrei Savu commented on WHIRR-415:
-----------------------------------
Looks good. Why do we need a new test class CdhHadoopServiceVersionTest? I think we can add just a new method to CdhHadoopServiceTest and make setUp @BeforeClass and tearDown @AfterClass. What do you think?
Tom you've raised some concerns about adding the ability to setup arbitrary environment variables. Can you elaborate?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Tom White (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-415:
----------------------------
Attachment: WHIRR-415.patch
The REPO variable is the right way to do it, but unfortunately it's not being passed through. The current workaround would be to copy install_cdh_hadoop.sh and edit it (see http://whirr.apache.org/faq.html#How_can_I_modify_the_instance_installation_and_configuration_scripts).
Here's an untested patch that should allow you to specify the repo via the "tarball" property:
{noformat}
whirr.hadoop.tarball.url=cdh3u1
{noformat}
However this overloads the "tarball" property, so we might want to come up with a better way.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Tom White (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147161#comment-13147161 ]
Tom White commented on WHIRR-415:
---------------------------------
The tarball.url overload was a hack to get ideas on how to do this better! Alex, your patch is much better.
However, the decision to use environment variables vs. properties is an implementation detail, which I'm not sure is something the user cares about. (I also worry about the ability to set arbitrary environment variables.) E.g. in this case the setting should be "whirr.hadoop.repo=cdh3u1" or similar. Perhaps we can just have another property for that?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147286#comment-13147286 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
A whirr property would be nicer, I quite agree -- something in me just objects to three new roles+ClusterActionHandler classes for CDH flavours of hadoop/hbase/zookeep, just to have the property called something nicer. WDYT?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146533#comment-13146533 ]
Alex Heneveld commented on WHIRR-415:
-------------------------------------
>From the code it looks like the {{functions/*_cdh_*.sh}} pay attention to the environment variable {{REPO}} which defaults to {{cdh3}} (version {{3}}, symlinked to {{3u2}}; other versions listed e.g. at [http://archive.cloudera.com/redhat/cdh/]).
Does it work to, say force 3u1, if you {{export REPO=cdh3u1}} before whirring?
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu resolved WHIRR-415.
-------------------------------
Resolution: Fixed
Committed. Thanks Alex!
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Assignee: Alex Heneveld
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-415) Let users specify the CDH release
(cdh3u1, cdh3u2)
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-415:
------------------------------
Attachment: WHIRR-415.patch
Alex I have made the following changes to your patch:
* moved the version test to CdhHadoopServiceTest to avoid starting a new cluster from scratch - it takes too long
* locked all cdh tests to chd3u2
* change hadoop-jobtracker+hadoop-namenode to hadoop-namenode+hadoop-jobtracker to make the integration test work
Let me know what you think.
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415-whirr.env.patch, WHIRR-415.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-415) Let users specify the CDH
release (cdh3u1, cdh3u2)
Posted by "Alex Heneveld (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146990#comment-13146990 ]
Alex Heneveld edited comment on WHIRR-415 at 11/9/11 12:39 PM:
---------------------------------------------------------------
This attachment {{whirr-415-env}} implements the {{env.someVar}} syntax above, with unit test, and with a corresponding integraton test for {{cdh3u1}} (although I am having trouble getting any of the CDH integration tests to pass so not sure if this integration test works, although I have confirmed it is installing the correct 3u1 version).
was (Author: alex.heneveld):
This attachment ({{WHIRR-415-env}} implements the {{env.someVar}} syntax above, with unit test, and with a corresponding integraton test for {{cdh3u1}} (although I am having trouble getting any of the CDH integration tests to pass so not sure if this integration test works, although I have confirmed it is installing the correct 3u1 version).
> Let users specify the CDH release (cdh3u1, cdh3u2)
> --------------------------------------------------
>
> Key: WHIRR-415
> URL: https://issues.apache.org/jira/browse/WHIRR-415
> Project: Whirr
> Issue Type: Improvement
> Components: service/cdh
> Reporter: Andrei Savu
> Priority: Critical
> Fix For: 0.7.0
>
> Attachments: WHIRR-415-env.patch, WHIRR-415.patch
>
>
> Allow explicit selection of the CDH release you want to use. The current behaviour is to use the latest one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira