You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Carin Meier <ca...@gmail.com> on 2019/01/12 18:11:03 UTC
Question about notification on nightly test failures
The Clojure package is thinking of adding some nightly tests and I'd like
to understand how the notification works in case of failure.
The code in the nightly Jenkins file
https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
seems to indicate that the failure is emailed. But where does this email go?
If you are a contributor, do you get notified of this?
I don't remember seeing any notification of nightly failures, so I'm
wondering how this works and if there are any improvements to accessibility
that we can make, (like maybe posting it to a slack room)?
Thanks,
Carin
Re: Question about notification on nightly test failures
Posted by Pedro Larroy <pe...@gmail.com>.
Looks fine to me.
On Wed, Jan 16, 2019 at 11:44 PM Carin Meier <ca...@gmail.com> wrote:
>
> The Clojure package now has turned the examples into integration tests that
> we would like to have CI run. It takes about 15 min for them to complete.
>
> We have a PR in progress
> https://github.com/apache/incubator-mxnet/pull/13624
>
> I would like to propose that we run these as part of the regular CI until
> the notifications for nightly builds get implemented. We will run on a
> whitelist or have an exclude list so that any problematic tests can be
> disabled if needed.
>
> Please let me know any feedback on this or concerns.
>
> Thanks,
> Carin
>
>
> On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy <pe...@gmail.com>
> wrote:
>
> > Why don't we enable a slack notifier? I think it would be useful to
> > interact with notifications from slack directly, including the label
> > bot for example.
> >
> > Pedro.
> >
> > On Sun, Jan 13, 2019 at 1:55 AM Carin Meier <ca...@gmail.com> wrote:
> > >
> > > Thanks for the explanation Marco :)
> > >
> > > - Carin
> > >
> > > On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hi Carin,
> > > >
> > > > thanks for thinking about adding nightly tests to clojure, I'm sure
> > this
> > > > will be of big benefit!
> > > >
> > > > You're right, the email system is in place but we basically disabled
> > the
> > > > service December 2017 because it was flooding the inboxes of everybody.
> > > >
> > > > We've been thinking about various notification methods, but were always
> > > > afraid making the notifications meaningless if they come too frequently
> > > > (which is WAY better now, thanks to everybody's efforts around
> > stabilizing
> > > > the tests!) because people would filter them.
> > > >
> > > > I like the idea of a specific slack channel for the notifications. Are
> > > > there any alternatives the community could think of?
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > >
> > > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier <ca...@gmail.com>
> > > > geschrieben:
> > > >
> > > > > The Clojure package is thinking of adding some nightly tests and I'd
> > like
> > > > > to understand how the notification works in case of failure.
> > > > >
> > > > > The code in the nightly Jenkins file
> > > > >
> > > > >
> > > >
> > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
> > > > > seems to indicate that the failure is emailed. But where does this
> > email
> > > > > go?
> > > > > If you are a contributor, do you get notified of this?
> > > > >
> > > > > I don't remember seeing any notification of nightly failures, so I'm
> > > > > wondering how this works and if there are any improvements to
> > > > accessibility
> > > > > that we can make, (like maybe posting it to a slack room)?
> > > > >
> > > > > Thanks,
> > > > > Carin
> > > > >
> > > >
> >
Re: Question about notification on nightly test failures
Posted by Carin Meier <ca...@gmail.com>.
The Clojure package now has turned the examples into integration tests that
we would like to have CI run. It takes about 15 min for them to complete.
We have a PR in progress
https://github.com/apache/incubator-mxnet/pull/13624
I would like to propose that we run these as part of the regular CI until
the notifications for nightly builds get implemented. We will run on a
whitelist or have an exclude list so that any problematic tests can be
disabled if needed.
Please let me know any feedback on this or concerns.
Thanks,
Carin
On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy <pe...@gmail.com>
wrote:
> Why don't we enable a slack notifier? I think it would be useful to
> interact with notifications from slack directly, including the label
> bot for example.
>
> Pedro.
>
> On Sun, Jan 13, 2019 at 1:55 AM Carin Meier <ca...@gmail.com> wrote:
> >
> > Thanks for the explanation Marco :)
> >
> > - Carin
> >
> > On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu <ma...@gmail.com>
> > wrote:
> >
> > > Hi Carin,
> > >
> > > thanks for thinking about adding nightly tests to clojure, I'm sure
> this
> > > will be of big benefit!
> > >
> > > You're right, the email system is in place but we basically disabled
> the
> > > service December 2017 because it was flooding the inboxes of everybody.
> > >
> > > We've been thinking about various notification methods, but were always
> > > afraid making the notifications meaningless if they come too frequently
> > > (which is WAY better now, thanks to everybody's efforts around
> stabilizing
> > > the tests!) because people would filter them.
> > >
> > > I like the idea of a specific slack channel for the notifications. Are
> > > there any alternatives the community could think of?
> > >
> > > Best regards,
> > > Marco
> > >
> > >
> > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier <ca...@gmail.com>
> > > geschrieben:
> > >
> > > > The Clojure package is thinking of adding some nightly tests and I'd
> like
> > > > to understand how the notification works in case of failure.
> > > >
> > > > The code in the nightly Jenkins file
> > > >
> > > >
> > >
> https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
> > > > seems to indicate that the failure is emailed. But where does this
> email
> > > > go?
> > > > If you are a contributor, do you get notified of this?
> > > >
> > > > I don't remember seeing any notification of nightly failures, so I'm
> > > > wondering how this works and if there are any improvements to
> > > accessibility
> > > > that we can make, (like maybe posting it to a slack room)?
> > > >
> > > > Thanks,
> > > > Carin
> > > >
> > >
>
Re: Question about notification on nightly test failures
Posted by Pedro Larroy <pe...@gmail.com>.
Why don't we enable a slack notifier? I think it would be useful to
interact with notifications from slack directly, including the label
bot for example.
Pedro.
On Sun, Jan 13, 2019 at 1:55 AM Carin Meier <ca...@gmail.com> wrote:
>
> Thanks for the explanation Marco :)
>
> - Carin
>
> On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu <ma...@gmail.com>
> wrote:
>
> > Hi Carin,
> >
> > thanks for thinking about adding nightly tests to clojure, I'm sure this
> > will be of big benefit!
> >
> > You're right, the email system is in place but we basically disabled the
> > service December 2017 because it was flooding the inboxes of everybody.
> >
> > We've been thinking about various notification methods, but were always
> > afraid making the notifications meaningless if they come too frequently
> > (which is WAY better now, thanks to everybody's efforts around stabilizing
> > the tests!) because people would filter them.
> >
> > I like the idea of a specific slack channel for the notifications. Are
> > there any alternatives the community could think of?
> >
> > Best regards,
> > Marco
> >
> >
> > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier <ca...@gmail.com>
> > geschrieben:
> >
> > > The Clojure package is thinking of adding some nightly tests and I'd like
> > > to understand how the notification works in case of failure.
> > >
> > > The code in the nightly Jenkins file
> > >
> > >
> > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
> > > seems to indicate that the failure is emailed. But where does this email
> > > go?
> > > If you are a contributor, do you get notified of this?
> > >
> > > I don't remember seeing any notification of nightly failures, so I'm
> > > wondering how this works and if there are any improvements to
> > accessibility
> > > that we can make, (like maybe posting it to a slack room)?
> > >
> > > Thanks,
> > > Carin
> > >
> >
Re: Question about notification on nightly test failures
Posted by Carin Meier <ca...@gmail.com>.
Thanks for the explanation Marco :)
- Carin
On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu <ma...@gmail.com>
wrote:
> Hi Carin,
>
> thanks for thinking about adding nightly tests to clojure, I'm sure this
> will be of big benefit!
>
> You're right, the email system is in place but we basically disabled the
> service December 2017 because it was flooding the inboxes of everybody.
>
> We've been thinking about various notification methods, but were always
> afraid making the notifications meaningless if they come too frequently
> (which is WAY better now, thanks to everybody's efforts around stabilizing
> the tests!) because people would filter them.
>
> I like the idea of a specific slack channel for the notifications. Are
> there any alternatives the community could think of?
>
> Best regards,
> Marco
>
>
> Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier <ca...@gmail.com>
> geschrieben:
>
> > The Clojure package is thinking of adding some nightly tests and I'd like
> > to understand how the notification works in case of failure.
> >
> > The code in the nightly Jenkins file
> >
> >
> https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
> > seems to indicate that the failure is emailed. But where does this email
> > go?
> > If you are a contributor, do you get notified of this?
> >
> > I don't remember seeing any notification of nightly failures, so I'm
> > wondering how this works and if there are any improvements to
> accessibility
> > that we can make, (like maybe posting it to a slack room)?
> >
> > Thanks,
> > Carin
> >
>
Re: Question about notification on nightly test failures
Posted by Marco de Abreu <ma...@gmail.com>.
Hi Carin,
thanks for thinking about adding nightly tests to clojure, I'm sure this
will be of big benefit!
You're right, the email system is in place but we basically disabled the
service December 2017 because it was flooding the inboxes of everybody.
We've been thinking about various notification methods, but were always
afraid making the notifications meaningless if they come too frequently
(which is WAY better now, thanks to everybody's efforts around stabilizing
the tests!) because people would filter them.
I like the idea of a specific slack channel for the notifications. Are
there any alternatives the community could think of?
Best regards,
Marco
Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier <ca...@gmail.com>
geschrieben:
> The Clojure package is thinking of adding some nightly tests and I'd like
> to understand how the notification works in case of failure.
>
> The code in the nightly Jenkins file
>
> https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135
> seems to indicate that the failure is emailed. But where does this email
> go?
> If you are a contributor, do you get notified of this?
>
> I don't remember seeing any notification of nightly failures, so I'm
> wondering how this works and if there are any improvements to accessibility
> that we can make, (like maybe posting it to a slack room)?
>
> Thanks,
> Carin
>