You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Konstantin Boudnik (JIRA)" <ji...@apache.org> on 2011/02/27 01:14:58 UTC
[jira] Created: (HADOOP-7157) Create build hosts configuratoins to
prevent degradation of patch validation and snapshot build environment
Create build hosts configuratoins to prevent degradation of patch validation and snapshot build environment
-----------------------------------------------------------------------------------------------------------
Key: HADOOP-7157
URL: https://issues.apache.org/jira/browse/HADOOP-7157
Project: Hadoop Common
Issue Type: Improvement
Environment: Apache Hadoop build hosts
Reporter: Konstantin Boudnik
Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029436#comment-13029436 ]
Konstantin Boudnik commented on HADOOP-7157:
--------------------------------------------
Thanks for the review. I will fix the patch to make it more compact with resource array and will post it shortly.
As for applying: your #3 is pretty goes pretty much along with what I had in mind by saying "to be done say once a day". Storing it along with rest of Hadoop source base seems to be a logical thing to do too.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7157) Create build hosts configurations to
prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-7157:
---------------------------------------
Component/s: build
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002056#comment-13002056 ]
Allen Wittenauer commented on HADOOP-7157:
------------------------------------------
Why are they being reinstalled and/or in need of configuration updates? Is something being updated just to be updated or is there a purpose?
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029083#comment-13029083 ]
Konstantin Boudnik commented on HADOOP-7157:
--------------------------------------------
The following steps are needed to bootstrap puppet and keeping build configuration in check:
- download puppet [2.6.8|http://puppetlabs.com/downloads/puppet/puppet-2.6.8.tar.gz]. Unpack and install it by running puppet/install.rb
- run recipe (with superuser privileges):
{noformat}
puppet apply --debug src/build.env/default-env.puppet
{noformat}
Running this with {{--noop}} will effectively execute dry-run
I propose these steps are to be done say once a day on every build machine.
The set of packages/versions proposed is what we have on Apache Hadoop build machines at the moment.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7157) Create build hosts configurations to
prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-7157:
---------------------------------------
Attachment: HADOOP-7157.patch
Initial puppet recipe to keep the configuration of build system in check.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik resolved HADOOP-7157.
----------------------------------------
Resolution: Won't Fix
Seeminly not of interest to anyone.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch, HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-7157) Create build hosts configurations to
prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-7157:
---------------------------------------
Summary: Create build hosts configurations to prevent degradation of patch validation and snapshot build environment (was: Create build hosts configuratoins to prevent degradation of patch validation and snapshot build environment)
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029090#comment-13029090 ]
Roman Shaposhnik commented on HADOOP-7157:
------------------------------------------
In general -- a strong +1 on the approach. H/W tends to go down, being decommissioned and tinkered with so having a reliable way to re-deploy a particular build/test environment is extremely important. Puppet fits the bill perfectly.
Now, a couple of points on Puppet usage:
1. Not sure you need precise version strings for the kinds of packages that are specified (and not all of them are specified, actually)
2. Minor nit: you can simplify an implementation by using an array syntax for resource instantiation
3. What's your plan on applying proposed recipe? I think it would be nice to have a cron job that would check it out from the apache repo and apply
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7157) Create build hosts configurations to
prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HADOOP-7157:
---------------------------------------
Attachment: HADOOP-7157.patch
Using resource array instead of decl. list.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Attachments: HADOOP-7157.patch, HADOOP-7157.patch
>
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik reassigned HADOOP-7157:
------------------------------------------
Assignee: Konstantin Boudnik
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7157) Create build hosts configurations
to prevent degradation of patch validation and snapshot build environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002462#comment-13002462 ]
Konstantin Boudnik commented on HADOOP-7157:
--------------------------------------------
That I don't know, Allen. It has been pointed out that recently some machines were re-flashed (again, I have no idea why). During this op. patch packages got missed which cases builds failing for some days. Thus this JIRA.
> Create build hosts configurations to prevent degradation of patch validation and snapshot build environment
> -----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7157
> URL: https://issues.apache.org/jira/browse/HADOOP-7157
> Project: Hadoop Common
> Issue Type: Improvement
> Environment: Apache Hadoop build hosts
> Reporter: Konstantin Boudnik
>
> Build hosts are being re-jumped, in need for configuration updates, etc. It is hard to maintain the same configuration across a 10+ hosts. A specialized service such as Puppet can be used to maintain build machines software and configurations at a desired level.
> Such configs should be checked into SCM system along with the of sources code.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira