You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Sid Anand <sa...@apache.org> on 2018/09/02 19:45:15 UTC

Apache Spark Interfering with Airflow Jira/PRs ??

Fellow committers... what happened around 11a PT today? I see a flood of
updates to our Jiras from "Apache Spark".

Some examples:

   1.
   https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
   2. It also updated https://issues.apache.org/jira/browse/AIRFLOW-2553 as
   resolved and closed https://github.com/apache/incubator-airflow/pull/3451
   PR

What's going on?
-s

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Holden Karau <ho...@pigscanfly.ca>.
On Mon, Sep 3, 2018 at 3:39 AM Ash Berlin-Taylor <as...@apache.org> wrote:

> Irrespective of the mis-configuration (Spark vs Airflow, bug causing loop
> etc.) is it possible to do the action that is triggering the new email? We
> already have Github creating some comments in Jira (which result in an
> email) and would like to keep the email volume down.
>
> Might be worth checking what settings Spark have for their Gitbox
> integration Jira and copying that?

so the easiest would be to just do it only for new PRs and avoid updating
old JIRA tickets. I’ll talk with josh about the config and see if there’s a
way to skip the emails entirely though.

>
>
> -ash
>
> > On 2 Sep 2018, at 21:57, Holden Karau <ho...@gmail.com> wrote:
> >
> > Really sorry for the noise on JIRA, I've shut down the app and I'll try
> and
> > figure out how it ended up doings this.
> >
> > On Sun, Sep 2, 2018 at 1:17 PM Holden Karau <ho...@gmail.com>
> wrote:
> >
> >> huh the one with "Apache Spark" in the name doesn't make any sense, I'll
> >> turn off the airflow dashboard but I don't have the credentials to the
> >> Apache Spark JIRA account so I'm super confused how that's happening (my
> >> guess would be there's an unprotected queue in the Spark version that's
> >> somehow gotten entries in it or someone put some credentials in the src
> >> repo is forked from).
> >>
> >> On Sun, Sep 2, 2018 at 1:12 PM Holden Karau <ho...@gmail.com>
> >> wrote:
> >>
> >>> Yup that's me, I've turned off the linking feature it was stuck failing
> >>> in a loop because of permissions (it had half of the permissions
> required).
> >>>
> >>> On Sun, Sep 2, 2018 at 1:07 PM Arthur Wiedmer <
> arthur.wiedmer@gmail.com>
> >>> wrote:
> >>>
> >>>> I am guessing Holden is testing the JIRA bot/integration, since some
> of
> >>>> the JIRA's are being assigned to her magical unicorn 🦄.
> >>>>
> >>>> Holden?
> >>>>
> >>>> Best,
> >>>> Arthur
> >>>>
> >>>> On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:
> >>>>
> >>>>> I am getting that too. Tons of emails as well about the same. Not
> sure
> >>>>> the
> >>>>> reason.
> >>>>>
> >>>>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
> >>>>>
> >>>>>> Fellow committers... what happened around 11a PT today? I see a
> flood
> >>>>> of
> >>>>>> updates to our Jiras from "Apache Spark".
> >>>>>>
> >>>>>> Some examples:
> >>>>>>
> >>>>>>   1.
> >>>>>>
> >>>>>>
> >>>>>
> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
> >>>>>>   2. It also updated
> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-2553
> >>>>>> as
> >>>>>>   resolved and closed
> >>>>>> https://github.com/apache/incubator-airflow/pull/3451
> >>>>>>   PR
> >>>>>>
> >>>>>> What's going on?
> >>>>>> -s
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>> --
> >>> Cell : 425-233-8271
> >>>
> >>
> >>
> >> --
> >> Cell : 425-233-8271
> >>
> >
> >
> > --
> > Cell : 425-233-8271
>
> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Ash Berlin-Taylor <as...@apache.org>.
Irrespective of the mis-configuration (Spark vs Airflow, bug causing loop etc.) is it possible to do the action that is triggering the new email? We already have Github creating some comments in Jira (which result in an email) and would like to keep the email volume down.

Might be worth checking what settings Spark have for their Gitbox integration Jira and copying that?

-ash

> On 2 Sep 2018, at 21:57, Holden Karau <ho...@gmail.com> wrote:
> 
> Really sorry for the noise on JIRA, I've shut down the app and I'll try and
> figure out how it ended up doings this.
> 
> On Sun, Sep 2, 2018 at 1:17 PM Holden Karau <ho...@gmail.com> wrote:
> 
>> huh the one with "Apache Spark" in the name doesn't make any sense, I'll
>> turn off the airflow dashboard but I don't have the credentials to the
>> Apache Spark JIRA account so I'm super confused how that's happening (my
>> guess would be there's an unprotected queue in the Spark version that's
>> somehow gotten entries in it or someone put some credentials in the src
>> repo is forked from).
>> 
>> On Sun, Sep 2, 2018 at 1:12 PM Holden Karau <ho...@gmail.com>
>> wrote:
>> 
>>> Yup that's me, I've turned off the linking feature it was stuck failing
>>> in a loop because of permissions (it had half of the permissions required).
>>> 
>>> On Sun, Sep 2, 2018 at 1:07 PM Arthur Wiedmer <ar...@gmail.com>
>>> wrote:
>>> 
>>>> I am guessing Holden is testing the JIRA bot/integration, since some of
>>>> the JIRA's are being assigned to her magical unicorn 🦄.
>>>> 
>>>> Holden?
>>>> 
>>>> Best,
>>>> Arthur
>>>> 
>>>> On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:
>>>> 
>>>>> I am getting that too. Tons of emails as well about the same. Not sure
>>>>> the
>>>>> reason.
>>>>> 
>>>>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>>>>> 
>>>>>> Fellow committers... what happened around 11a PT today? I see a flood
>>>>> of
>>>>>> updates to our Jiras from "Apache Spark".
>>>>>> 
>>>>>> Some examples:
>>>>>> 
>>>>>>   1.
>>>>>> 
>>>>>> 
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>>>>>>   2. It also updated
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-2553
>>>>>> as
>>>>>>   resolved and closed
>>>>>> https://github.com/apache/incubator-airflow/pull/3451
>>>>>>   PR
>>>>>> 
>>>>>> What's going on?
>>>>>> -s
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> --
>>> Cell : 425-233-8271
>>> 
>> 
>> 
>> --
>> Cell : 425-233-8271
>> 
> 
> 
> -- 
> Cell : 425-233-8271

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Holden Karau <ho...@gmail.com>.
Really sorry for the noise on JIRA, I've shut down the app and I'll try and
figure out how it ended up doings this.

On Sun, Sep 2, 2018 at 1:17 PM Holden Karau <ho...@gmail.com> wrote:

> huh the one with "Apache Spark" in the name doesn't make any sense, I'll
> turn off the airflow dashboard but I don't have the credentials to the
> Apache Spark JIRA account so I'm super confused how that's happening (my
> guess would be there's an unprotected queue in the Spark version that's
> somehow gotten entries in it or someone put some credentials in the src
> repo is forked from).
>
> On Sun, Sep 2, 2018 at 1:12 PM Holden Karau <ho...@gmail.com>
> wrote:
>
>> Yup that's me, I've turned off the linking feature it was stuck failing
>> in a loop because of permissions (it had half of the permissions required).
>>
>> On Sun, Sep 2, 2018 at 1:07 PM Arthur Wiedmer <ar...@gmail.com>
>> wrote:
>>
>>> I am guessing Holden is testing the JIRA bot/integration, since some of
>>> the JIRA's are being assigned to her magical unicorn 🦄.
>>>
>>> Holden?
>>>
>>> Best,
>>> Arthur
>>>
>>> On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:
>>>
>>>> I am getting that too. Tons of emails as well about the same. Not sure
>>>> the
>>>> reason.
>>>>
>>>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>>>>
>>>> > Fellow committers... what happened around 11a PT today? I see a flood
>>>> of
>>>> > updates to our Jiras from "Apache Spark".
>>>> >
>>>> > Some examples:
>>>> >
>>>> >    1.
>>>> >
>>>> >
>>>> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>>>> >    2. It also updated
>>>> https://issues.apache.org/jira/browse/AIRFLOW-2553
>>>> > as
>>>> >    resolved and closed
>>>> > https://github.com/apache/incubator-airflow/pull/3451
>>>> >    PR
>>>> >
>>>> > What's going on?
>>>> > -s
>>>> >
>>>>
>>>
>>
>> --
>> Cell : 425-233-8271
>>
>
>
> --
> Cell : 425-233-8271
>


-- 
Cell : 425-233-8271

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Holden Karau <ho...@gmail.com>.
huh the one with "Apache Spark" in the name doesn't make any sense, I'll
turn off the airflow dashboard but I don't have the credentials to the
Apache Spark JIRA account so I'm super confused how that's happening (my
guess would be there's an unprotected queue in the Spark version that's
somehow gotten entries in it or someone put some credentials in the src
repo is forked from).

On Sun, Sep 2, 2018 at 1:12 PM Holden Karau <ho...@gmail.com> wrote:

> Yup that's me, I've turned off the linking feature it was stuck failing in
> a loop because of permissions (it had half of the permissions required).
>
> On Sun, Sep 2, 2018 at 1:07 PM Arthur Wiedmer <ar...@gmail.com>
> wrote:
>
>> I am guessing Holden is testing the JIRA bot/integration, since some of
>> the JIRA's are being assigned to her magical unicorn 🦄.
>>
>> Holden?
>>
>> Best,
>> Arthur
>>
>> On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:
>>
>>> I am getting that too. Tons of emails as well about the same. Not sure
>>> the
>>> reason.
>>>
>>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>>>
>>> > Fellow committers... what happened around 11a PT today? I see a flood
>>> of
>>> > updates to our Jiras from "Apache Spark".
>>> >
>>> > Some examples:
>>> >
>>> >    1.
>>> >
>>> >
>>> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>>> >    2. It also updated
>>> https://issues.apache.org/jira/browse/AIRFLOW-2553
>>> > as
>>> >    resolved and closed
>>> > https://github.com/apache/incubator-airflow/pull/3451
>>> >    PR
>>> >
>>> > What's going on?
>>> > -s
>>> >
>>>
>>
>
> --
> Cell : 425-233-8271
>


-- 
Cell : 425-233-8271

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Holden Karau <ho...@gmail.com>.
Yup that's me, I've turned off the linking feature it was stuck failing in
a loop because of permissions (it had half of the permissions required).

On Sun, Sep 2, 2018 at 1:07 PM Arthur Wiedmer <ar...@gmail.com>
wrote:

> I am guessing Holden is testing the JIRA bot/integration, since some of
> the JIRA's are being assigned to her magical unicorn 🦄.
>
> Holden?
>
> Best,
> Arthur
>
> On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:
>
>> I am getting that too. Tons of emails as well about the same. Not sure the
>> reason.
>>
>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>>
>> > Fellow committers... what happened around 11a PT today? I see a flood of
>> > updates to our Jiras from "Apache Spark".
>> >
>> > Some examples:
>> >
>> >    1.
>> >
>> >
>> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>> >    2. It also updated
>> https://issues.apache.org/jira/browse/AIRFLOW-2553
>> > as
>> >    resolved and closed
>> > https://github.com/apache/incubator-airflow/pull/3451
>> >    PR
>> >
>> > What's going on?
>> > -s
>> >
>>
>

-- 
Cell : 425-233-8271

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Arthur Wiedmer <ar...@gmail.com>.
I am guessing Holden is testing the JIRA bot/integration, since some of the
JIRA's are being assigned to her magical unicorn 🦄.

Holden?

Best,
Arthur

On Sun, Sep 2, 2018, 12:48 Kaxil Naik <ka...@gmail.com> wrote:

> I am getting that too. Tons of emails as well about the same. Not sure the
> reason.
>
> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>
> > Fellow committers... what happened around 11a PT today? I see a flood of
> > updates to our Jiras from "Apache Spark".
> >
> > Some examples:
> >
> >    1.
> >
> >
> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
> >    2. It also updated https://issues.apache.org/jira/browse/AIRFLOW-2553
> > as
> >    resolved and closed
> > https://github.com/apache/incubator-airflow/pull/3451
> >    PR
> >
> > What's going on?
> > -s
> >
>

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Jakob Homan <jg...@gmail.com>.
Open a ticket against infra@. Either a misconfiguration or spam. 

> On Sep 2, 2018, at 12:48 PM, Kaxil Naik <ka...@gmail.com> wrote:
> 
> I am getting that too. Tons of emails as well about the same. Not sure the
> reason.
> 
>> On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:
>> 
>> Fellow committers... what happened around 11a PT today? I see a flood of
>> updates to our Jiras from "Apache Spark".
>> 
>> Some examples:
>> 
>>   1.
>> 
>> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>>   2. It also updated https://issues.apache.org/jira/browse/AIRFLOW-2553
>> as
>>   resolved and closed
>> https://github.com/apache/incubator-airflow/pull/3451
>>   PR
>> 
>> What's going on?
>> -s
>>

Re: Apache Spark Interfering with Airflow Jira/PRs ??

Posted by Kaxil Naik <ka...@gmail.com>.
I am getting that too. Tons of emails as well about the same. Not sure the
reason.

On Sun, 2 Sep 2018, 20:45 Sid Anand, <sa...@apache.org> wrote:

> Fellow committers... what happened around 11a PT today? I see a flood of
> updates to our Jiras from "Apache Spark".
>
> Some examples:
>
>    1.
>
> https://issues.apache.org/jira/browse/AIRFLOW-2408?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16601616#comment-16601616
>    2. It also updated https://issues.apache.org/jira/browse/AIRFLOW-2553
> as
>    resolved and closed
> https://github.com/apache/incubator-airflow/pull/3451
>    PR
>
> What's going on?
> -s
>