You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Brock Noland (JIRA)" <ji...@apache.org> on 2013/11/04 20:16:18 UTC
[jira] [Updated] (HADOOP-9765) Precommit Admin job chokes on issues
without an attachment
[ https://issues.apache.org/jira/browse/HADOOP-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HADOOP-9765:
---------------------------------
Attachment: HADOOP-9765.patch
The attached patch changes the script to python. I did not use lxml because it's not installed on all machines by default. ElementTree is slower but more widely available. Other changes:
* Limit the number of entries in the historical log to 5000
* Only submit jobs to Jenkins if --live is specified
Should we choose to deploy this, the process would be as follows:
1) Test on a hadoop slave to ensure it works:
{noformat}
python jenkinsPrecommitAdmin.py \
--jira-filter="https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/12323182/SearchRequest-12323182.xml?tempMax=50" \
--jenkins-url="https://builds.apache.org"
{noformat}
2) Commit patch to nightly directory
3) Change Precommit-Admin job script to:
{noformat}
python $WORKSPACE/nightly/jenkinsPrecommitAdmin.py \
--jira-filter="https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/12323182/SearchRequest-12323182.xml?tempMax=50" \
--jenkins-url="https://builds.apache.org" \
--jenkins-token=<REPLACE ME> \
--live
{noformat}
The jenkins-token needs to be places where <REPLACE ME> is currently located. The token is available in the current Precommit admin job script.
> Precommit Admin job chokes on issues without an attachment
> ----------------------------------------------------------
>
> Key: HADOOP-9765
> URL: https://issues.apache.org/jira/browse/HADOOP-9765
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: HADOOP-9765.patch
>
>
> Check out this file: https://builds.apache.org/job/PreCommit-Admin/lastSuccessfulBuild/artifact/patch_tested.txt
> It has "corrupt" data:
> {noformat}
> HIVE-4877HDFS-5010,12593214
> HIVE-4877HBASE-8693,12593082
> HIVE-4877YARN-919,12593107
> YARN-905,12593225
> HIVE-4877HBASE-8752,12588069
> {noformat}
> which resulted in the Hive precommit job being called with the ISSUE_NUM of 5010, 8693, 919, and 8752.
> Looking at the script and some output, I pulled from the last run, it looks like it gets hosed up when there is a JIRA which is PA but doesn't have an attachment (as ZK-1402 is currently sitting). For example:
> This is the bad data the script is encountering:
> {noformat}
> $ grep -A 2 'ZOOKEEPER-1402' patch_available2.elements
> ZOOKEEPER-1402
> HBASE-8348
> id="12592318"
> {noformat}
> This is where it screws up:
> {noformat}
> $ awk '{ printf "%s", $0 }' patch_available2.elements | sed -e "s/\W*id=\"/,/g" | perl -pe "s/\"/\n/g" | grep ZOOKEEPER-1402
> ZOOKEEPER-1402HBASE-8348 ,12592318
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)