You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2008/03/07 07:34:47 UTC

[Hadoop Wiki] Update of "HudsonBuildServer" by NigelDaley

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by NigelDaley:
http://wiki.apache.org/hadoop/HudsonBuildServer

The comment on the change is:
Updated for new hudson.zones.apache.org build machine

------------------------------------------------------------------------------
  
  We use [https://hudson.dev.java.net/ Hudson] as our continuous integration server:
  
-  * [http://lucene.zones.apache.org:8080/hudson/]
+  * [http://hudson.zones.apache.org/hudson/]
  
- Hudson performs nightly builds for Lucene, Solr, Nutch, and Hadoop.  In addition is runs automated patch verification builds for Hadoop.
+ Hudson runs automated patch verification builds for Hadoop.
  
  = Administering Hudson =
  
- Hudson is run by the {{{hudson}}} user on {{{lucene.zones.apache.org}}}.  Any Lucene committer may have an account on this machine.  To manage the Hudson server, log into this machine and
+ Hudson is run by the {{{hudson}}} user on {{{hudson.zones.apache.org}}}.  Any Apache committer may have an account on this machine.  To manage the Hudson server, log into this machine and
  
  {{{sudo su - hudson}}}
  
  This will give you the proper shell environment to manage the Hudson server.  Hudson uses a number of tools for itself and for the builds it runs.  All tools are located in {{{~hudson/tools}}}.  Hudson stores all of it's config files and builds in {{{$HUDSON_HOME}}} which is set to {{{~hudson/hudson}}}.
  
- '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net) before you start manipulating Hudson, to make sure that you're not stepping on someone else's toes.  Any substantial configuration changes should be discussed on the developer mailing list.
+ '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net) before you start manipulating Hudson, to make sure that you're not stepping on someone else's toes.  Any substantial configuration changes to Hadoop builds should be discussed on the developer mailing list.
  
  == Tasks ==
  
- Most tasks, such as creating a new build or modifying an existing build, are performed through the Hudson web interface.  Once you login to Hudson, you are provided with an expanded number of administration options.  To create a login on Hudson, see below.  The following tasks must be performed after logging into {{{lucene.zones.apache.org}}} and switching to the {{{hudson}}} user.
+ Most tasks, such as creating a new build or modifying an existing build, are performed through the Hudson web interface.  Once you login to Hudson, you are provided with an expanded number of administration options.  To create a login on Hudson, see below.  The following tasks must be performed after logging into {{{hudson.zones.apache.org}}} and switching to the {{{hudson}}} user.
  
     1. '''Stopping a nightly build:'''
-       When logged into Hudson, a red X button is displayed next to each running build.  Clicking the button will end the build as far as Hudson is concerned, but the underlying processes sometimes don't get properly terminated.  A better approach is to kill the {{{build-test.sh}}} process (used by some builds) or, if that doesn't exist, the {{{ant}}} process:
+       When logged into Hudson, a red X button is displayed next to each running build.  Clicking the button will end the build as far as Hudson is concerned, but the underlying processes sometimes don't get properly terminated.  A better approach is to kill the {{{hudsonBuildHadoopNightly.sh}}} process or, if that doesn't exist, the {{{ant}}} process:
-          1. Login to {{{lucene.zones.apache.org}}}
+          1. Login to {{{hudson.zones.apache.org}}}
           1. Switch to the {{{hudson}}} user
-          1. {{{ps -ef | grep hudson}}} to get the {{{build-test.sh}}} or {{{ant}}} process id
+          1. {{{ps -ef | grep hudson}}} to get the {{{hudsonBuildHadoopNightly.sh}}} or {{{ant}}} process id
-          1. {{{kill [-9] <build-test.sh or ant process id>}}}
+          1. {{{kill [-9] <hudsonBuildHadoopNightly.sh or ant process id>}}}
-          1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are gone.  The only remaining java process should be the Tomcat process ({{{org.apache.catalina.startup.Bootstrap}}}).
+          1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are gone.
        When this process dies, Hudson will fail the build.
- 
-    1. '''Stopping a patch build:'''
-       Follow the same advice for stopping a nightly build (above) but include these additional steps once the running patch build is terminated:
-          1. {{{cd ~hudson/hudson/jobs/Hadoop-Patch/}}}
-          1. {{{ls -al current}}} to see which directory the {{{current}}} link is pointing to
-          1. {{{rm -rf <directory that current points to> current}}} to delete both the directory and the {{{current}}} link
-       Once the {{{current}}} link is gone, a waiting {{{processEmail.sh}}} process will create a new {{{current}}} link and start the next patch build.
  
     1. '''Killing a hung test:'''
        Sometimes it is desirable to just kill a hung test and allow the rest of the build to continue. Although the build will still fail, it can sometimes provide useful information if you can get a thread dump on the hung test and, if the build is in the first test pass, the second pass might succeed as some tests hang only intermittently. You can tell a test is hung by looking at the console log for the build and if the last entry is some time ago (at least 15 minutes) it is safe to assume the test is hung.
-          1. Login to {{{lucene.zones.apache.org}}}
+          1. Login to {{{hudson.zones.apache.org}}}
           1. Switch to the {{{hudson}}} user
           1. Find the hung test: {{{ps -laxww | grep hudson | grep test.build.data | grep -v grep'}}}
              For example:
@@ -81, +74 @@

           1. {{{kill -QUIT 17700}}} to try to get a thread dump. Although this does not always work (because the test is just too locked up), if it does work, it can provide useful debugging information.
           1. {{{kill -9 17700}}} to kill the hung test. If you are watching the console (and you should be), you should see the console output resume.
  
-    1. '''Creating a login on lucene.zones.apache.org:'''
+    1. '''Creating a login on hudson.zones.apache.org and the Hudson web UI:'''
+       Committers may have a login on the hudson.zones.apache.org build machine.  Follow [http://wiki.apache.org/general/Hudson these] directions.
-       Committers may have a login on the lucene.zones.apache.org build machine.  To create a new user, follow these directions:
-          {{{cat /etc/passwd}}} and look for an unused userid (the 3rd column)
  
-          {{{sudo useradd -u <unique userid> -g other -d /export/home/<user name> -m -s /usr/bin/bash <user name>}}}
- 
-          {{{sudo visudo}}} to give the new user sudo permission
- 
-          {{{sudo passwd <user name>}}} to set the new password that you will pass along to the new user; they should change it of course
- 
-    1. '''Creating a login on Hudson:'''
-       Committers can create a Hudson login for themselves by adding an entry to the Tomcat users list located in {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/conf/tomcat-users.xml}}}.  Simply copy and edit an existing entry.  Once your entry is added, Tomcat must be restarted (see below for instructions to perform this task).  Once Tomcat is restarted, you can then login to the web interface.
- 
-    1. '''Starting and shutting down Hudson (and Tomcat):'''
-       Hudson is a web app run inside of Tomcat.  '''Please only shutdown Hudson when there are no builds in process.'''  Tomcat is started and shutdown with these scripts:
-          {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/bin/startup.sh}}}
- 
-          {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/bin/shutdown.sh}}}
- 
-    1. '''Installing a new version of Hudson:'''
-       The {{{hudson.war}}} file goes in {{{~hudson/tools/tomcat/apache-tomcat-5.5.20/webapp/}}}
-          1. Shutdown Hudson
-          1. Download the new {{{hudson.war}}} file
-          1. Save the old {{{hudson.war}}} file
-          1. Copy in the new {{{hudson.war}}} file into the {{{webapp}}} directory
-          1. Startup Hudson
- 
-    1. '''Installing a new plugin:'''
-       Hudson plugins extend the functionality of Hudson.  Plugins are used to collect and present data for Findbugs and Checkstyle, to name a few.  These plugins are packaged as {{{.hpi}}} files and are available from [https://hudson.dev.java.net/].  The plugin file goes in {{{~hudson/hudson/plugins/}}}.  To install or upgrade a plugin, follow these steps:
-          1. Download the new plugin file
-          1. Save the old plugin file
-          1. Copy in the new plugin file into the {{{plugins}}} directory
-          1. Shutdown and startup Hudson
-