You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Allen Wittenauer <aw...@altiscale.com> on 2015/12/01 00:04:45 UTC
Re: Disable some of the Hudson integration comments on JIRA
> On Nov 30, 2015, at 11:33 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> Good point Allen. So I guess the broader question is, do we find the
> per-commit tracking build and test useful? With our current flakiness
> levels, there isn't much signal from a FAILED on one of these integration
> jobs. I think Hadoop-trunk-Commit is still nice as a compilation check.
… except it doesn’t compile everything ...
Re: Disable some of the Hudson integration comments on JIRA
Posted by Steve Loughran <st...@hortonworks.com>.
> On 1 Dec 2015, at 00:03, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>> On Nov 30, 2015, at 3:46 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>>
>> To be exhaustive, we'd also add the various -Drequire options for full
>> inclusion of all optional native components. Unfortunately, that gets
>> tricky because of the various build pre-requisites which would need to be
>> present on the Jenkins hosts. This is where the Dockerfile is helpful,
>> because it encapsulates all of that. Maybe this job would benefit from
>> using the Dockerfile.
>>
>
>
> I’ve been noodling over the past few weeks about a tool called ‘qbt’ that would basically manage builds like Yetus manages tests. It’d be able to use the Yetus plug-ins to qualify the build, do reporting, etc.
>
>
>
I've been doing some work again on some databricks-originating code to go through jenkins histories and ID flaky tests, including automated result publishing as a google doc.
https://github.com/steveloughran/spark-test-failures/tree/stevel-version
The original release was hard code to the spark jenkins and builds; I'm fixing my branch to work with ASF jenkins.
Although the original design's publishing to google docs is cute, we could have some other reporters. Generating an HTML page would be ideal for jenkins itself. So jenkins could be used to run analytics workloads against other job histories
Re: Disable some of the Hudson integration comments on JIRA
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 30, 2015, at 3:46 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>
> To be exhaustive, we'd also add the various -Drequire options for full
> inclusion of all optional native components. Unfortunately, that gets
> tricky because of the various build pre-requisites which would need to be
> present on the Jenkins hosts. This is where the Dockerfile is helpful,
> because it encapsulates all of that. Maybe this job would benefit from
> using the Dockerfile.
>
I’ve been noodling over the past few weeks about a tool called ‘qbt’ that would basically manage builds like Yetus manages tests. It’d be able to use the Yetus plug-ins to qualify the build, do reporting, etc.
Re: Disable some of the Hudson integration comments on JIRA
Posted by Chris Nauroth <cn...@hortonworks.com>.
To be exhaustive, we'd also add the various -Drequire options for full
inclusion of all optional native components. Unfortunately, that gets
tricky because of the various build pre-requisites which would need to be
present on the Jenkins hosts. This is where the Dockerfile is helpful,
because it encapsulates all of that. Maybe this job would benefit from
using the Dockerfile.
--Chris Nauroth
On 11/30/15, 3:26 PM, "Andrew Wang" <an...@cloudera.com> wrote:
>I went ahead and turned off JIRA notification for all the "other" jobs I
>could find, if you see more than one Hudson email from now on feel free to
>ping me or turn it off yourself. The jobs are also still running, just not
>JIRA commenting anymore.
>
>I did also add the native nvn args to Hadoop-trunk-Commit, feel free to
>append more stuff as desired.
>
>On Mon, Nov 30, 2015 at 3:11 PM, Andrew Wang <an...@cloudera.com>
>wrote:
>
>>
>> On Mon, Nov 30, 2015 at 3:04 PM, Allen Wittenauer <aw...@altiscale.com>
>> wrote:
>>
>>>
>>> > On Nov 30, 2015, at 11:33 AM, Andrew Wang <an...@cloudera.com>
>>> wrote:
>>> >
>>> > Good point Allen. So I guess the broader question is, do we find the
>>> > per-commit tracking build and test useful? With our current flakiness
>>> > levels, there isn't much signal from a FAILED on one of these
>>> integration
>>> > jobs. I think Hadoop-trunk-Commit is still nice as a compilation
>>>check.
>>>
>>> Š except it doesn¹t compile everything ...
>>>
>>> So I see this in the shell step at the top level:
>>
>> $MAVEN_HOME/bin/mvn clean install -DskipTests
>>
>> It's not compiling native code if that was your concern, and I can add
>> -Pnative. Anything else to tack on?
>>
Re: Disable some of the Hudson integration comments on JIRA
Posted by Andrew Wang <an...@cloudera.com>.
I went ahead and turned off JIRA notification for all the "other" jobs I
could find, if you see more than one Hudson email from now on feel free to
ping me or turn it off yourself. The jobs are also still running, just not
JIRA commenting anymore.
I did also add the native nvn args to Hadoop-trunk-Commit, feel free to
append more stuff as desired.
On Mon, Nov 30, 2015 at 3:11 PM, Andrew Wang <an...@cloudera.com>
wrote:
>
> On Mon, Nov 30, 2015 at 3:04 PM, Allen Wittenauer <aw...@altiscale.com>
> wrote:
>
>>
>> > On Nov 30, 2015, at 11:33 AM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> >
>> > Good point Allen. So I guess the broader question is, do we find the
>> > per-commit tracking build and test useful? With our current flakiness
>> > levels, there isn't much signal from a FAILED on one of these
>> integration
>> > jobs. I think Hadoop-trunk-Commit is still nice as a compilation check.
>>
>> … except it doesn’t compile everything ...
>>
>> So I see this in the shell step at the top level:
>
> $MAVEN_HOME/bin/mvn clean install -DskipTests
>
> It's not compiling native code if that was your concern, and I can add
> -Pnative. Anything else to tack on?
>
Re: Disable some of the Hudson integration comments on JIRA
Posted by Andrew Wang <an...@cloudera.com>.
On Mon, Nov 30, 2015 at 3:04 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> > On Nov 30, 2015, at 11:33 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > Good point Allen. So I guess the broader question is, do we find the
> > per-commit tracking build and test useful? With our current flakiness
> > levels, there isn't much signal from a FAILED on one of these integration
> > jobs. I think Hadoop-trunk-Commit is still nice as a compilation check.
>
> … except it doesn’t compile everything ...
>
> So I see this in the shell step at the top level:
$MAVEN_HOME/bin/mvn clean install -DskipTests
It's not compiling native code if that was your concern, and I can add
-Pnative. Anything else to tack on?