You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Karthik Kambatla <ka...@cloudera.com> on 2015/03/31 18:21:46 UTC
Uncertainty of pre-commit builds
Hi devs,
I am sure people must have noticed, but thought I would bring this up.
I have noticed that our pre-commit builds could end up running the "wrong"
set of unit tests for patches. For instance, YARN-3412
<https://issues.apache.org/jira/browse/YARN-3412> changes only YARN files,
but the test were run against one of the MR modules.
I suspect there is a race condition when there are multiple builds
executing on the same node, or remnants from a previous run are getting
picked up.
Filed HADOOP-11779 for this.
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
BTW, I think HADOOP-11746 is pretty much ready for review. I’ve done some Jenkins simulations, but really need someone more familiar with our build systems to do some tests.
On Mar 31, 2015, at 10:14 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
> improving our test-patch scripts, Allen.
>
> On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Very likely. One of the things I noticed during HADOOP-11746 is
>> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
>> environment correctly or leaks variables between runs. shellcheck prints
>> out so many messages on the current code I’m surprised it doesn’t crash.
>>
>> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>>
>>> Hi devs,
>>>
>>> I am sure people must have noticed, but thought I would bring this up.
>>>
>>> I have noticed that our pre-commit builds could end up running the
>> "wrong"
>>> set of unit tests for patches. For instance, YARN-3412
>>> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
>> files,
>>> but the test were run against one of the MR modules.
>>>
>>> I suspect there is a race condition when there are multiple builds
>>> executing on the same node, or remnants from a previous run are getting
>>> picked up.
>>>
>>> Filed HADOOP-11779 for this.
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>
>>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
BTW, I think HADOOP-11746 is pretty much ready for review. I’ve done some Jenkins simulations, but really need someone more familiar with our build systems to do some tests.
On Mar 31, 2015, at 10:14 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
> improving our test-patch scripts, Allen.
>
> On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Very likely. One of the things I noticed during HADOOP-11746 is
>> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
>> environment correctly or leaks variables between runs. shellcheck prints
>> out so many messages on the current code I’m surprised it doesn’t crash.
>>
>> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>>
>>> Hi devs,
>>>
>>> I am sure people must have noticed, but thought I would bring this up.
>>>
>>> I have noticed that our pre-commit builds could end up running the
>> "wrong"
>>> set of unit tests for patches. For instance, YARN-3412
>>> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
>> files,
>>> but the test were run against one of the MR modules.
>>>
>>> I suspect there is a race condition when there are multiple builds
>>> executing on the same node, or remnants from a previous run are getting
>>> picked up.
>>>
>>> Filed HADOOP-11779 for this.
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>
>>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
BTW, I think HADOOP-11746 is pretty much ready for review. I’ve done some Jenkins simulations, but really need someone more familiar with our build systems to do some tests.
On Mar 31, 2015, at 10:14 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
> improving our test-patch scripts, Allen.
>
> On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Very likely. One of the things I noticed during HADOOP-11746 is
>> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
>> environment correctly or leaks variables between runs. shellcheck prints
>> out so many messages on the current code I’m surprised it doesn’t crash.
>>
>> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>>
>>> Hi devs,
>>>
>>> I am sure people must have noticed, but thought I would bring this up.
>>>
>>> I have noticed that our pre-commit builds could end up running the
>> "wrong"
>>> set of unit tests for patches. For instance, YARN-3412
>>> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
>> files,
>>> but the test were run against one of the MR modules.
>>>
>>> I suspect there is a race condition when there are multiple builds
>>> executing on the same node, or remnants from a previous run are getting
>>> picked up.
>>>
>>> Filed HADOOP-11779 for this.
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>
>>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
BTW, I think HADOOP-11746 is pretty much ready for review. I’ve done some Jenkins simulations, but really need someone more familiar with our build systems to do some tests.
On Mar 31, 2015, at 10:14 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
> improving our test-patch scripts, Allen.
>
> On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>>
>> Very likely. One of the things I noticed during HADOOP-11746 is
>> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
>> environment correctly or leaks variables between runs. shellcheck prints
>> out so many messages on the current code I’m surprised it doesn’t crash.
>>
>> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>>
>>> Hi devs,
>>>
>>> I am sure people must have noticed, but thought I would bring this up.
>>>
>>> I have noticed that our pre-commit builds could end up running the
>> "wrong"
>>> set of unit tests for patches. For instance, YARN-3412
>>> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
>> files,
>>> but the test were run against one of the MR modules.
>>>
>>> I suspect there is a race condition when there are multiple builds
>>> executing on the same node, or remnants from a previous run are getting
>>> picked up.
>>>
>>> Filed HADOOP-11779 for this.
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>
>>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Karthik Kambatla <ka...@cloudera.com>.
Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
improving our test-patch scripts, Allen.
On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Very likely. One of the things I noticed during HADOOP-11746 is
> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
> environment correctly or leaks variables between runs. shellcheck prints
> out so many messages on the current code I’m surprised it doesn’t crash.
>
> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> > Hi devs,
> >
> > I am sure people must have noticed, but thought I would bring this up.
> >
> > I have noticed that our pre-commit builds could end up running the
> "wrong"
> > set of unit tests for patches. For instance, YARN-3412
> > <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
> files,
> > but the test were run against one of the MR modules.
> >
> > I suspect there is a race condition when there are multiple builds
> > executing on the same node, or remnants from a previous run are getting
> > picked up.
> >
> > Filed HADOOP-11779 for this.
> >
> > --
> > Karthik Kambatla
> > Software Engineer, Cloudera Inc.
> > --------------------------------------------
> > http://five.sentenc.es
>
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Karthik Kambatla <ka...@cloudera.com>.
Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
improving our test-patch scripts, Allen.
On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Very likely. One of the things I noticed during HADOOP-11746 is
> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
> environment correctly or leaks variables between runs. shellcheck prints
> out so many messages on the current code I’m surprised it doesn’t crash.
>
> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> > Hi devs,
> >
> > I am sure people must have noticed, but thought I would bring this up.
> >
> > I have noticed that our pre-commit builds could end up running the
> "wrong"
> > set of unit tests for patches. For instance, YARN-3412
> > <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
> files,
> > but the test were run against one of the MR modules.
> >
> > I suspect there is a race condition when there are multiple builds
> > executing on the same node, or remnants from a previous run are getting
> > picked up.
> >
> > Filed HADOOP-11779 for this.
> >
> > --
> > Karthik Kambatla
> > Software Engineer, Cloudera Inc.
> > --------------------------------------------
> > http://five.sentenc.es
>
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Karthik Kambatla <ka...@cloudera.com>.
Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
improving our test-patch scripts, Allen.
On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Very likely. One of the things I noticed during HADOOP-11746 is
> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
> environment correctly or leaks variables between runs. shellcheck prints
> out so many messages on the current code I’m surprised it doesn’t crash.
>
> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> > Hi devs,
> >
> > I am sure people must have noticed, but thought I would bring this up.
> >
> > I have noticed that our pre-commit builds could end up running the
> "wrong"
> > set of unit tests for patches. For instance, YARN-3412
> > <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
> files,
> > but the test were run against one of the MR modules.
> >
> > I suspect there is a race condition when there are multiple builds
> > executing on the same node, or remnants from a previous run are getting
> > picked up.
> >
> > Filed HADOOP-11779 for this.
> >
> > --
> > Karthik Kambatla
> > Software Engineer, Cloudera Inc.
> > --------------------------------------------
> > http://five.sentenc.es
>
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Karthik Kambatla <ka...@cloudera.com>.
Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on
improving our test-patch scripts, Allen.
On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Very likely. One of the things I noticed during HADOOP-11746 is
> that there is a HUGE, catastrophic race if Jenkins doesn’t setup the
> environment correctly or leaks variables between runs. shellcheck prints
> out so many messages on the current code I’m surprised it doesn’t crash.
>
> On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> > Hi devs,
> >
> > I am sure people must have noticed, but thought I would bring this up.
> >
> > I have noticed that our pre-commit builds could end up running the
> "wrong"
> > set of unit tests for patches. For instance, YARN-3412
> > <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN
> files,
> > but the test were run against one of the MR modules.
> >
> > I suspect there is a race condition when there are multiple builds
> > executing on the same node, or remnants from a previous run are getting
> > picked up.
> >
> > Filed HADOOP-11779 for this.
> >
> > --
> > Karthik Kambatla
> > Software Engineer, Cloudera Inc.
> > --------------------------------------------
> > http://five.sentenc.es
>
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic race if Jenkins doesn’t setup the environment correctly or leaks variables between runs. shellcheck prints out so many messages on the current code I’m surprised it doesn’t crash.
On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Hi devs,
>
> I am sure people must have noticed, but thought I would bring this up.
>
> I have noticed that our pre-commit builds could end up running the "wrong"
> set of unit tests for patches. For instance, YARN-3412
> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN files,
> but the test were run against one of the MR modules.
>
> I suspect there is a race condition when there are multiple builds
> executing on the same node, or remnants from a previous run are getting
> picked up.
>
> Filed HADOOP-11779 for this.
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic race if Jenkins doesn’t setup the environment correctly or leaks variables between runs. shellcheck prints out so many messages on the current code I’m surprised it doesn’t crash.
On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Hi devs,
>
> I am sure people must have noticed, but thought I would bring this up.
>
> I have noticed that our pre-commit builds could end up running the "wrong"
> set of unit tests for patches. For instance, YARN-3412
> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN files,
> but the test were run against one of the MR modules.
>
> I suspect there is a race condition when there are multiple builds
> executing on the same node, or remnants from a previous run are getting
> picked up.
>
> Filed HADOOP-11779 for this.
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic race if Jenkins doesn’t setup the environment correctly or leaks variables between runs. shellcheck prints out so many messages on the current code I’m surprised it doesn’t crash.
On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Hi devs,
>
> I am sure people must have noticed, but thought I would bring this up.
>
> I have noticed that our pre-commit builds could end up running the "wrong"
> set of unit tests for patches. For instance, YARN-3412
> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN files,
> but the test were run against one of the MR modules.
>
> I suspect there is a race condition when there are multiple builds
> executing on the same node, or remnants from a previous run are getting
> picked up.
>
> Filed HADOOP-11779 for this.
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: Uncertainty of pre-commit builds
Posted by Allen Wittenauer <aw...@altiscale.com>.
Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic race if Jenkins doesn’t setup the environment correctly or leaks variables between runs. shellcheck prints out so many messages on the current code I’m surprised it doesn’t crash.
On Mar 31, 2015, at 9:21 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Hi devs,
>
> I am sure people must have noticed, but thought I would bring this up.
>
> I have noticed that our pre-commit builds could end up running the "wrong"
> set of unit tests for patches. For instance, YARN-3412
> <https://issues.apache.org/jira/browse/YARN-3412> changes only YARN files,
> but the test were run against one of the MR modules.
>
> I suspect there is a race condition when there are multiple builds
> executing on the same node, or remnants from a previous run are getting
> picked up.
>
> Filed HADOOP-11779 for this.
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es