You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2024/06/26 20:12:43 UTC
Using AI / Dosu to help us with triaging issues
Hello everyone,
Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking into
employing AI / Natural Language processing to help us triage issues for
Apache Airflow. We do not want to go “all-in” into getting a chatbot to
respond to all our issues because we believe this is not how the community
is being built. We looked at various ways we can start exploring the
capabilities of the new ML/AI/Natural Language processing available.
We worked with the Dosu team. They are approved by the Apache Software
Foundation infrastructure as Github integration and few ASF projects
already use it (including our friends at Superset) - they have a fantastic
offer to provide free service for open-source projects like Airflow.
Together we evaluated what we can start with and initially we have a
proposal to use auto-labeling of issues created in the Airflow repository.
We have a number of rules that are established for the triage team [2] but
those rules are mundane and difficult to follow, and generally a lot of our
issues are either not classified or badly classified, and currently we
cannot rely on the classification.
What we want to start with is to re-classify our issues and apply the
labels retro-actively for all past issues as well as start applying them
automatically for new issues.
The risk of doing it is low, and it will allow us to explore integration
and follow up with more elaborated integration. We have some options such
as getting automated proposals for answers for similar questions, as well
as “chat-bot generated/maintainer approved” answers - but we definitely do
not want to have bots starting to answer automatically on PRs and issues.
We think that this will allow us to explore more ways how we can make
maintainers and triagers time more efficient - and help us while we are
focusing also on Airflow 3 development soon.
The Dosu founder - Devin, will send some more information soon and is
available for questions here and in the #triage-team channel on Slack.
Unless we hear some complaints, we will apply labelling changes in a few
days, I think this stage is not really controversial, and we will run a
LAZY CONSENSUS in a few days.
J. E. K. (and the Dosu team).
[1] https://dosu.dev/
[2]
https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
Re: Using AI / Dosu to help us with triaging issues
Posted by Poorvi Rohidekar <po...@gmail.com>.
The auto labelling would be a great idea as it would ensure that
issues are promptly classified without depending too much on the triage
team.
Regards,
Poorvi
On Thu, 27 Jun 2024 at 14:54, Ankit Chaurasia <su...@gmail.com> wrote:
> +1. Love this idea.
>
> *Ankit Chaurasia*
>
>
>
>
>
>
> On Thu, Jun 27, 2024 at 3:00 PM Amogh Desai <am...@gmail.com>
> wrote:
>
> > > The current idea we have, how we could still benefit from having AI/Bot
> > generated answers, is for maintainers and possibly triage teams to be
> able
> > to review the proposed answers and "approve" them before they are posted.
> > Such an answer might then be marked as "bot/automatically/AI generated"
> but
> > it will also get a "Maintainer approved" or "Triage team approved" badge,
> > This way anyone can see that the answer was reviewed by a
> maintainer/triage
> > team member. This way we can get way more efficiency in handling issues
> > (time for reviewing answers will be far less than writing it). Also
> > maintainers/triage team members will be able to correct the answers if
> they
> > are misleading / wrong. Who approved/corrected it will be visible for the
> > triage/maintainer team in the DoSu interface, but not seen in the Github
> UI
> > who actually approved/corrected it.
> >
> > Love this! One thing we need to be a little sure of is the "honesty"
> while
> > answering these bot suggestions, at least
> > initially, so that the bot settles well into our ecosystem.
> >
> > I am happy with the direction this is heading into
> >
> > Thanks & Regards,
> > Amogh Desai
> >
> >
> > On Thu, Jun 27, 2024 at 1:31 PM Wei Lee <we...@gmail.com> wrote:
> >
> > > +1 for this idea. I remember we had this discussion during PyCon US,
> > haha.
> > >
> > > So, in the first phase, we'll need a maintainer's approval before the
> AI
> > > response is sent. I'm not sure whether this will be a burden to
> > > maintainers. I'm thinking of adding a label to that AI response answer;
> > the
> > > user can decide whether to accept that even before the maintainers'
> > > approval. If maintainers check the response and it's ok, they can
> remove
> > > that label.
> > >
> > > Best,
> > > Wei
> > >
> > > > On Jun 27, 2024, at 3:07 PM, Jarek Potiuk <ja...@potiuk.com> wrote:
> > > >
> > > >>
> > > >>
> > > >> I guess it's too early to discuss what the workflow we're picturing
> > for
> > > it
> > > >> is. But I would like it if a reporter is unable to see the comment
> > until
> > > >> it's approved (which I guess is exactly what you're also suggesting
> > > Jarek).
> > > >>
> > > >
> > > > Absolutely. Actually - we (few of maintainers) can already see the
> > > answers
> > > > generated in a DoSU interface as "drafts" - and once we enable
> > labelling
> > > we
> > > > can on-board more maintainers and triage team and work with DoSu team
> > on
> > > > implementing a workflow that might be good to start with.
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > For additional commands, e-mail: dev-help@airflow.apache.org
> > >
> > >
> >
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Ankit Chaurasia <su...@gmail.com>.
+1. Love this idea.
*Ankit Chaurasia*
On Thu, Jun 27, 2024 at 3:00 PM Amogh Desai <am...@gmail.com>
wrote:
> > The current idea we have, how we could still benefit from having AI/Bot
> generated answers, is for maintainers and possibly triage teams to be able
> to review the proposed answers and "approve" them before they are posted.
> Such an answer might then be marked as "bot/automatically/AI generated" but
> it will also get a "Maintainer approved" or "Triage team approved" badge,
> This way anyone can see that the answer was reviewed by a maintainer/triage
> team member. This way we can get way more efficiency in handling issues
> (time for reviewing answers will be far less than writing it). Also
> maintainers/triage team members will be able to correct the answers if they
> are misleading / wrong. Who approved/corrected it will be visible for the
> triage/maintainer team in the DoSu interface, but not seen in the Github UI
> who actually approved/corrected it.
>
> Love this! One thing we need to be a little sure of is the "honesty" while
> answering these bot suggestions, at least
> initially, so that the bot settles well into our ecosystem.
>
> I am happy with the direction this is heading into
>
> Thanks & Regards,
> Amogh Desai
>
>
> On Thu, Jun 27, 2024 at 1:31 PM Wei Lee <we...@gmail.com> wrote:
>
> > +1 for this idea. I remember we had this discussion during PyCon US,
> haha.
> >
> > So, in the first phase, we'll need a maintainer's approval before the AI
> > response is sent. I'm not sure whether this will be a burden to
> > maintainers. I'm thinking of adding a label to that AI response answer;
> the
> > user can decide whether to accept that even before the maintainers'
> > approval. If maintainers check the response and it's ok, they can remove
> > that label.
> >
> > Best,
> > Wei
> >
> > > On Jun 27, 2024, at 3:07 PM, Jarek Potiuk <ja...@potiuk.com> wrote:
> > >
> > >>
> > >>
> > >> I guess it's too early to discuss what the workflow we're picturing
> for
> > it
> > >> is. But I would like it if a reporter is unable to see the comment
> until
> > >> it's approved (which I guess is exactly what you're also suggesting
> > Jarek).
> > >>
> > >
> > > Absolutely. Actually - we (few of maintainers) can already see the
> > answers
> > > generated in a DoSU interface as "drafts" - and once we enable
> labelling
> > we
> > > can on-board more maintainers and triage team and work with DoSu team
> on
> > > implementing a workflow that might be good to start with.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > For additional commands, e-mail: dev-help@airflow.apache.org
> >
> >
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Amogh Desai <am...@gmail.com>.
> The current idea we have, how we could still benefit from having AI/Bot
generated answers, is for maintainers and possibly triage teams to be able
to review the proposed answers and "approve" them before they are posted.
Such an answer might then be marked as "bot/automatically/AI generated" but
it will also get a "Maintainer approved" or "Triage team approved" badge,
This way anyone can see that the answer was reviewed by a maintainer/triage
team member. This way we can get way more efficiency in handling issues
(time for reviewing answers will be far less than writing it). Also
maintainers/triage team members will be able to correct the answers if they
are misleading / wrong. Who approved/corrected it will be visible for the
triage/maintainer team in the DoSu interface, but not seen in the Github UI
who actually approved/corrected it.
Love this! One thing we need to be a little sure of is the "honesty" while
answering these bot suggestions, at least
initially, so that the bot settles well into our ecosystem.
I am happy with the direction this is heading into
Thanks & Regards,
Amogh Desai
On Thu, Jun 27, 2024 at 1:31 PM Wei Lee <we...@gmail.com> wrote:
> +1 for this idea. I remember we had this discussion during PyCon US, haha.
>
> So, in the first phase, we'll need a maintainer's approval before the AI
> response is sent. I'm not sure whether this will be a burden to
> maintainers. I'm thinking of adding a label to that AI response answer; the
> user can decide whether to accept that even before the maintainers'
> approval. If maintainers check the response and it's ok, they can remove
> that label.
>
> Best,
> Wei
>
> > On Jun 27, 2024, at 3:07 PM, Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> >>
> >>
> >> I guess it's too early to discuss what the workflow we're picturing for
> it
> >> is. But I would like it if a reporter is unable to see the comment until
> >> it's approved (which I guess is exactly what you're also suggesting
> Jarek).
> >>
> >
> > Absolutely. Actually - we (few of maintainers) can already see the
> answers
> > generated in a DoSU interface as "drafts" - and once we enable labelling
> we
> > can on-board more maintainers and triage team and work with DoSu team on
> > implementing a workflow that might be good to start with.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Wei Lee <we...@gmail.com>.
+1 for this idea. I remember we had this discussion during PyCon US, haha.
So, in the first phase, we'll need a maintainer's approval before the AI response is sent. I'm not sure whether this will be a burden to maintainers. I'm thinking of adding a label to that AI response answer; the user can decide whether to accept that even before the maintainers' approval. If maintainers check the response and it's ok, they can remove that label.
Best,
Wei
> On Jun 27, 2024, at 3:07 PM, Jarek Potiuk <ja...@potiuk.com> wrote:
>
>>
>>
>> I guess it's too early to discuss what the workflow we're picturing for it
>> is. But I would like it if a reporter is unable to see the comment until
>> it's approved (which I guess is exactly what you're also suggesting Jarek).
>>
>
> Absolutely. Actually - we (few of maintainers) can already see the answers
> generated in a DoSU interface as "drafts" - and once we enable labelling we
> can on-board more maintainers and triage team and work with DoSu team on
> implementing a workflow that might be good to start with.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: Using AI / Dosu to help us with triaging issues
Posted by Jarek Potiuk <ja...@potiuk.com>.
>
>
> I guess it's too early to discuss what the workflow we're picturing for it
> is. But I would like it if a reporter is unable to see the comment until
> it's approved (which I guess is exactly what you're also suggesting Jarek).
>
Absolutely. Actually - we (few of maintainers) can already see the answers
generated in a DoSU interface as "drafts" - and once we enable labelling we
can on-board more maintainers and triage team and work with DoSu team on
implementing a workflow that might be good to start with.
Re: Using AI / Dosu to help us with triaging issues
Posted by Aritra Basu <ar...@gmail.com>.
Hmm I guess as long as there is an actual human approval before every
bot/ai answer (ie a triage team member or maintainer has actually verified
that the answer fits the question) I am onboard with the idea. It should
hopefully reduce some of the load on them.
I guess it's too early to discuss what the workflow we're picturing for it
is. But I would like it if a reporter is unable to see the comment until
it's approved (which I guess is exactly what you're also suggesting Jarek).
--
Regards,
Aritra Basu
On Thu, Jun 27, 2024, 12:01 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> Julian - anyone has a voice here :) .. We are not yet voting on it - I
> already see that lazy consensus will be enough - but even with voting,
> non-committers are encouraged to vote anyway (their votes are non-binding,
> but included in the results).
>
> Aritra/Amogh - I very much share your perspective there. Pure "bot" answers
> without human involvement are a bad way to build a community and it's
> "no-go" for me.
>
> The current idea we have, how we could still benefit from having AI/Bot
> generated answers, is for maintainers and possibly triage teams to be able
> to review the proposed answers and "approve" them before they are posted.
> Such an answer might then be marked as "bot/automatically/AI generated" but
> it will also get a "Maintainer approved" or "Triage team approved" badge,
> This way anyone can see that the answer was reviewed by a maintainer/triage
> team member. This way we can get way more efficiency in handling issues
> (time for reviewing answers will be far less than writing it). Also
> maintainers/triage team members will be able to correct the answers if they
> are misleading / wrong. Who approved/corrected it will be visible for the
> triage/maintainer team in the DoSu interface, but not seen in the Github UI
> who actually approved/corrected it.
>
> This has very interesting side effects:
>
> * AI-assisted knowledge transfer between maintainers and triage team -
> since the generated response will use past knowledge base, the maintainers
> / triage team members (if they treat it seriously and will actually spend a
> bit time on reading and understanding answers) - will actually learn from
> other's past answers this way. Especially if - as it is in Astro - we will
> include references to sources.
> * When the answer is posted by bot, and there is a follow-up question, the
> nice thing is that it's not tied to a particular maintainer - usually when
> now someone answers a question, almost by definition all follow-up
> questions are "bound" to that person who first answered it - with generic
> "a maintainer approved" badge - this binding is not there.
> * When maintainers correct an AI generated answer, this is a **perfect**
> learning data to improve the AI system - because you see the "bad" and
> "good" answer that is of a high quality (you know it was bad and most
> likely it's good after corrections) - which is a gold for AI learning
> process
> * If it works, it will actually raise contributor's trust to AI generated
> answers over time (and they will likely get better as well).
>
> We are **not** implementing it yet though - those are potential features we
> discussed with the Dosu team to implement (and happy to hear feedback on it
> as well), but one of the common reservations we have is that **just**
> posting bot answers is not going to happen - we need to do way better than
> that.
>
> J.
>
>
> On Thu, Jun 27, 2024 at 6:43 AM Amogh Desai <am...@gmail.com>
> wrote:
>
> > Agreed with Aritra here.
> >
> > I personally would love to have the "auto" labelling portion, which would
> > ensure that
> > issues get immediate "right" labels, without relying too much on the
> > triaging team but
> > I personally am not too comfortable with a bot answering my questions for
> > two reasons:
> >
> > 1. Maintainer/Contributor responses are tailored personally to a question
> > and the level of the issue reporter
> > 2. Bot responses sometimes are disregarded by readers, just because its a
> > "bot"
> >
> > Thanks & Regards,
> > Amogh Desai
> >
> >
> > On Thu, Jun 27, 2024 at 8:53 AM Aritra Basu <ar...@gmail.com>
> > wrote:
> >
> > > I'm a bit averse to having it answer questions (the similar issues
> part)
> > as
> > > I believe that's a slippery slope to where it could start being used to
> > > provide actual answers. I personally would be more comfortable with
> > > answering still being restricted to maintainers/contributors and maybe
> > > restricting the bot to just tagging similar issues while still
> requiring
> > a
> > > human to provide advice based on that similarity.
> > >
> > > Just my personal 2 cents. I'm not too strongly opposed to using it.
> > > --
> > > Regards,
> > > Aritra Basu
> > >
> > > On Thu, Jun 27, 2024, 2:41 AM Julian LaNeve
> <julian@astronomer.io.invalid
> > >
> > > wrote:
> > >
> > > > Not sure if I get an official vote here but we've been working with
> > Devin
> > > > and the Dosu team for Cosmos (
> > > > https://github.com/astronomer/astronomer-cosmos ) and it's been
> > working
> > > > great. Excited to see this in Airflow itself!
> > > >
> > > > Plus they use Airflow to power their data platform behind the scenes
> so
> > > > they have a vested interest in this one :)
> > > >
> > > > --
> > > >
> > > > *Julian LaNeve*
> > > > CTO
> > > >
> > > > Email: julian@astronomer.io
> > > > ( julian@astronomer.io ) Mobile: 330 509 5792
> > > >
> > > > On Wed, Jun 26, 2024 at 4:58 PM, Vikram Koka <
> > > vikram@astronomer.io.invalid
> > > > > wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > > +1
> > > > >
> > > > >
> > > > >
> > > > > Love it!
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck < vincbeck@ apache.
> > org (
> > > > > vincbeck@apache.org ) > wrote:
> > > > >
> > > > >
> > > > >>
> > > > >>
> > > > >> Fantastic idea!
> > > > >>
> > > > >>
> > > > >>
> > > > >> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> Hello everyone,
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Together with Elad, Kaxil, and the Dosu team [1], we’ve been
> > looking
> > > > into
> > > > >>> employing AI / Natural Language processing to help us triage
> issues
> > > for
> > > > >>> Apache Airflow. We do not want to go “all-in” into getting a
> > chatbot
> > > to
> > > > >>> respond to all our issues because we believe this is not how the
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> community
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> is being built. We looked at various ways we can start exploring
> > the
> > > > >>> capabilities of the new ML/AI/Natural Language processing
> > available.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> We worked with the Dosu team. They are approved by the Apache
> > > Software
> > > > >>> Foundation infrastructure as Github integration and few ASF
> > projects
> > > > >>> already use it (including our friends at Superset) - they have a
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> fantastic
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> offer to provide free service for open-source projects like
> > Airflow.
> > > > >>> Together we evaluated what we can start with and initially we
> have
> > a
> > > > >>> proposal to use auto-labeling of issues created in the Airflow
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> repository.
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> We have a number of rules that are established for the triage
> team
> > > [2]
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> but
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> those rules are mundane and difficult to follow, and generally a
> > lot
> > > of
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> our
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> issues are either not classified or badly classified, and
> currently
> > > we
> > > > >>> cannot rely on the classification.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> What we want to start with is to re-classify our issues and apply
> > the
> > > > >>> labels retro-actively for all past issues as well as start
> applying
> > > > them
> > > > >>> automatically for new issues.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> The risk of doing it is low, and it will allow us to explore
> > > > integration
> > > > >>> and follow up with more elaborated integration. We have some
> > options
> > > > such
> > > > >>> as getting automated proposals for answers for similar questions,
> > as
> > > > well
> > > > >>> as “chat-bot generated/maintainer approved” answers - but we
> > > definitely
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> do
> > > > >>
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> not want to have bots starting to answer automatically on PRs and
> > > > issues.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> We think that this will allow us to explore more ways how we can
> > make
> > > > >>> maintainers and triagers time more efficient - and help us while
> we
> > > are
> > > > >>> focusing also on Airflow 3 development soon.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> The Dosu founder - Devin, will send some more information soon
> and
> > is
> > > > >>> available for questions here and in the #triage-team channel on
> > > Slack.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Unless we hear some complaints, we will apply labelling changes
> in
> > a
> > > > few
> > > > >>> days, I think this stage is not really controversial, and we will
> > > run a
> > > > >>> LAZY CONSENSUS in a few days.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> J. E. K. (and the Dosu team).
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> [1] https:/ / dosu. dev/ ( https://dosu.dev/ )
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> [2]
> > > > >>>
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >> https:/ / github. com/ apache/ airflow/ blob/ main/
> > > > ISSUE_TRIAGE_PROCESS. rst#labels
> > > > >> (
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
> > > > >> )
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > To
> > > > >> unsubscribe, e-mail: dev-unsubscribe@ airflow. apache. org (
> > > > >> dev-unsubscribe@airflow.apache.org ) For additional commands,
> > e-mail:
> > > > dev-help@
> > > > >> airflow. apache. org ( dev-help@airflow.apache.org )
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > > >
> > >
> >
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Jarek Potiuk <ja...@potiuk.com>.
Julian - anyone has a voice here :) .. We are not yet voting on it - I
already see that lazy consensus will be enough - but even with voting,
non-committers are encouraged to vote anyway (their votes are non-binding,
but included in the results).
Aritra/Amogh - I very much share your perspective there. Pure "bot" answers
without human involvement are a bad way to build a community and it's
"no-go" for me.
The current idea we have, how we could still benefit from having AI/Bot
generated answers, is for maintainers and possibly triage teams to be able
to review the proposed answers and "approve" them before they are posted.
Such an answer might then be marked as "bot/automatically/AI generated" but
it will also get a "Maintainer approved" or "Triage team approved" badge,
This way anyone can see that the answer was reviewed by a maintainer/triage
team member. This way we can get way more efficiency in handling issues
(time for reviewing answers will be far less than writing it). Also
maintainers/triage team members will be able to correct the answers if they
are misleading / wrong. Who approved/corrected it will be visible for the
triage/maintainer team in the DoSu interface, but not seen in the Github UI
who actually approved/corrected it.
This has very interesting side effects:
* AI-assisted knowledge transfer between maintainers and triage team -
since the generated response will use past knowledge base, the maintainers
/ triage team members (if they treat it seriously and will actually spend a
bit time on reading and understanding answers) - will actually learn from
other's past answers this way. Especially if - as it is in Astro - we will
include references to sources.
* When the answer is posted by bot, and there is a follow-up question, the
nice thing is that it's not tied to a particular maintainer - usually when
now someone answers a question, almost by definition all follow-up
questions are "bound" to that person who first answered it - with generic
"a maintainer approved" badge - this binding is not there.
* When maintainers correct an AI generated answer, this is a **perfect**
learning data to improve the AI system - because you see the "bad" and
"good" answer that is of a high quality (you know it was bad and most
likely it's good after corrections) - which is a gold for AI learning
process
* If it works, it will actually raise contributor's trust to AI generated
answers over time (and they will likely get better as well).
We are **not** implementing it yet though - those are potential features we
discussed with the Dosu team to implement (and happy to hear feedback on it
as well), but one of the common reservations we have is that **just**
posting bot answers is not going to happen - we need to do way better than
that.
J.
On Thu, Jun 27, 2024 at 6:43 AM Amogh Desai <am...@gmail.com>
wrote:
> Agreed with Aritra here.
>
> I personally would love to have the "auto" labelling portion, which would
> ensure that
> issues get immediate "right" labels, without relying too much on the
> triaging team but
> I personally am not too comfortable with a bot answering my questions for
> two reasons:
>
> 1. Maintainer/Contributor responses are tailored personally to a question
> and the level of the issue reporter
> 2. Bot responses sometimes are disregarded by readers, just because its a
> "bot"
>
> Thanks & Regards,
> Amogh Desai
>
>
> On Thu, Jun 27, 2024 at 8:53 AM Aritra Basu <ar...@gmail.com>
> wrote:
>
> > I'm a bit averse to having it answer questions (the similar issues part)
> as
> > I believe that's a slippery slope to where it could start being used to
> > provide actual answers. I personally would be more comfortable with
> > answering still being restricted to maintainers/contributors and maybe
> > restricting the bot to just tagging similar issues while still requiring
> a
> > human to provide advice based on that similarity.
> >
> > Just my personal 2 cents. I'm not too strongly opposed to using it.
> > --
> > Regards,
> > Aritra Basu
> >
> > On Thu, Jun 27, 2024, 2:41 AM Julian LaNeve <julian@astronomer.io.invalid
> >
> > wrote:
> >
> > > Not sure if I get an official vote here but we've been working with
> Devin
> > > and the Dosu team for Cosmos (
> > > https://github.com/astronomer/astronomer-cosmos ) and it's been
> working
> > > great. Excited to see this in Airflow itself!
> > >
> > > Plus they use Airflow to power their data platform behind the scenes so
> > > they have a vested interest in this one :)
> > >
> > > --
> > >
> > > *Julian LaNeve*
> > > CTO
> > >
> > > Email: julian@astronomer.io
> > > ( julian@astronomer.io ) Mobile: 330 509 5792
> > >
> > > On Wed, Jun 26, 2024 at 4:58 PM, Vikram Koka <
> > vikram@astronomer.io.invalid
> > > > wrote:
> > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > >
> > > >
> > > > Love it!
> > > >
> > > >
> > > >
> > > > On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck < vincbeck@ apache.
> org (
> > > > vincbeck@apache.org ) > wrote:
> > > >
> > > >
> > > >>
> > > >>
> > > >> Fantastic idea!
> > > >>
> > > >>
> > > >>
> > > >> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> Hello everyone,
> > > >>>
> > > >>>
> > > >>>
> > > >>> Together with Elad, Kaxil, and the Dosu team [1], we’ve been
> looking
> > > into
> > > >>> employing AI / Natural Language processing to help us triage issues
> > for
> > > >>> Apache Airflow. We do not want to go “all-in” into getting a
> chatbot
> > to
> > > >>> respond to all our issues because we believe this is not how the
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> community
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> is being built. We looked at various ways we can start exploring
> the
> > > >>> capabilities of the new ML/AI/Natural Language processing
> available.
> > > >>>
> > > >>>
> > > >>>
> > > >>> We worked with the Dosu team. They are approved by the Apache
> > Software
> > > >>> Foundation infrastructure as Github integration and few ASF
> projects
> > > >>> already use it (including our friends at Superset) - they have a
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> fantastic
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> offer to provide free service for open-source projects like
> Airflow.
> > > >>> Together we evaluated what we can start with and initially we have
> a
> > > >>> proposal to use auto-labeling of issues created in the Airflow
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> repository.
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> We have a number of rules that are established for the triage team
> > [2]
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> but
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> those rules are mundane and difficult to follow, and generally a
> lot
> > of
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> our
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> issues are either not classified or badly classified, and currently
> > we
> > > >>> cannot rely on the classification.
> > > >>>
> > > >>>
> > > >>>
> > > >>> What we want to start with is to re-classify our issues and apply
> the
> > > >>> labels retro-actively for all past issues as well as start applying
> > > them
> > > >>> automatically for new issues.
> > > >>>
> > > >>>
> > > >>>
> > > >>> The risk of doing it is low, and it will allow us to explore
> > > integration
> > > >>> and follow up with more elaborated integration. We have some
> options
> > > such
> > > >>> as getting automated proposals for answers for similar questions,
> as
> > > well
> > > >>> as “chat-bot generated/maintainer approved” answers - but we
> > definitely
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> do
> > > >>
> > > >>
> > > >>>
> > > >>>
> > > >>> not want to have bots starting to answer automatically on PRs and
> > > issues.
> > > >>>
> > > >>>
> > > >>>
> > > >>> We think that this will allow us to explore more ways how we can
> make
> > > >>> maintainers and triagers time more efficient - and help us while we
> > are
> > > >>> focusing also on Airflow 3 development soon.
> > > >>>
> > > >>>
> > > >>>
> > > >>> The Dosu founder - Devin, will send some more information soon and
> is
> > > >>> available for questions here and in the #triage-team channel on
> > Slack.
> > > >>>
> > > >>>
> > > >>>
> > > >>> Unless we hear some complaints, we will apply labelling changes in
> a
> > > few
> > > >>> days, I think this stage is not really controversial, and we will
> > run a
> > > >>> LAZY CONSENSUS in a few days.
> > > >>>
> > > >>>
> > > >>>
> > > >>> J. E. K. (and the Dosu team).
> > > >>>
> > > >>>
> > > >>>
> > > >>> [1] https:/ / dosu. dev/ ( https://dosu.dev/ )
> > > >>>
> > > >>>
> > > >>>
> > > >>> [2]
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> https:/ / github. com/ apache/ airflow/ blob/ main/
> > > ISSUE_TRIAGE_PROCESS. rst#labels
> > > >> (
> > > >>
> > >
> >
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
> > > >> )
> > > >>
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > To
> > > >> unsubscribe, e-mail: dev-unsubscribe@ airflow. apache. org (
> > > >> dev-unsubscribe@airflow.apache.org ) For additional commands,
> e-mail:
> > > dev-help@
> > > >> airflow. apache. org ( dev-help@airflow.apache.org )
> > > >>
> > > >>
> > > >
> > > >
> > > >
> >
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Amogh Desai <am...@gmail.com>.
Agreed with Aritra here.
I personally would love to have the "auto" labelling portion, which would
ensure that
issues get immediate "right" labels, without relying too much on the
triaging team but
I personally am not too comfortable with a bot answering my questions for
two reasons:
1. Maintainer/Contributor responses are tailored personally to a question
and the level of the issue reporter
2. Bot responses sometimes are disregarded by readers, just because its a
"bot"
Thanks & Regards,
Amogh Desai
On Thu, Jun 27, 2024 at 8:53 AM Aritra Basu <ar...@gmail.com>
wrote:
> I'm a bit averse to having it answer questions (the similar issues part) as
> I believe that's a slippery slope to where it could start being used to
> provide actual answers. I personally would be more comfortable with
> answering still being restricted to maintainers/contributors and maybe
> restricting the bot to just tagging similar issues while still requiring a
> human to provide advice based on that similarity.
>
> Just my personal 2 cents. I'm not too strongly opposed to using it.
> --
> Regards,
> Aritra Basu
>
> On Thu, Jun 27, 2024, 2:41 AM Julian LaNeve <ju...@astronomer.io.invalid>
> wrote:
>
> > Not sure if I get an official vote here but we've been working with Devin
> > and the Dosu team for Cosmos (
> > https://github.com/astronomer/astronomer-cosmos ) and it's been working
> > great. Excited to see this in Airflow itself!
> >
> > Plus they use Airflow to power their data platform behind the scenes so
> > they have a vested interest in this one :)
> >
> > --
> >
> > *Julian LaNeve*
> > CTO
> >
> > Email: julian@astronomer.io
> > ( julian@astronomer.io ) Mobile: 330 509 5792
> >
> > On Wed, Jun 26, 2024 at 4:58 PM, Vikram Koka <
> vikram@astronomer.io.invalid
> > > wrote:
> >
> > >
> > >
> > >
> > > +1
> > >
> > >
> > >
> > > Love it!
> > >
> > >
> > >
> > > On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck < vincbeck@ apache. org (
> > > vincbeck@apache.org ) > wrote:
> > >
> > >
> > >>
> > >>
> > >> Fantastic idea!
> > >>
> > >>
> > >>
> > >> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> > >>
> > >>
> > >>>
> > >>>
> > >>> Hello everyone,
> > >>>
> > >>>
> > >>>
> > >>> Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking
> > into
> > >>> employing AI / Natural Language processing to help us triage issues
> for
> > >>> Apache Airflow. We do not want to go “all-in” into getting a chatbot
> to
> > >>> respond to all our issues because we believe this is not how the
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> community
> > >>
> > >>
> > >>>
> > >>>
> > >>> is being built. We looked at various ways we can start exploring the
> > >>> capabilities of the new ML/AI/Natural Language processing available.
> > >>>
> > >>>
> > >>>
> > >>> We worked with the Dosu team. They are approved by the Apache
> Software
> > >>> Foundation infrastructure as Github integration and few ASF projects
> > >>> already use it (including our friends at Superset) - they have a
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> fantastic
> > >>
> > >>
> > >>>
> > >>>
> > >>> offer to provide free service for open-source projects like Airflow.
> > >>> Together we evaluated what we can start with and initially we have a
> > >>> proposal to use auto-labeling of issues created in the Airflow
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> repository.
> > >>
> > >>
> > >>>
> > >>>
> > >>> We have a number of rules that are established for the triage team
> [2]
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> but
> > >>
> > >>
> > >>>
> > >>>
> > >>> those rules are mundane and difficult to follow, and generally a lot
> of
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> our
> > >>
> > >>
> > >>>
> > >>>
> > >>> issues are either not classified or badly classified, and currently
> we
> > >>> cannot rely on the classification.
> > >>>
> > >>>
> > >>>
> > >>> What we want to start with is to re-classify our issues and apply the
> > >>> labels retro-actively for all past issues as well as start applying
> > them
> > >>> automatically for new issues.
> > >>>
> > >>>
> > >>>
> > >>> The risk of doing it is low, and it will allow us to explore
> > integration
> > >>> and follow up with more elaborated integration. We have some options
> > such
> > >>> as getting automated proposals for answers for similar questions, as
> > well
> > >>> as “chat-bot generated/maintainer approved” answers - but we
> definitely
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> do
> > >>
> > >>
> > >>>
> > >>>
> > >>> not want to have bots starting to answer automatically on PRs and
> > issues.
> > >>>
> > >>>
> > >>>
> > >>> We think that this will allow us to explore more ways how we can make
> > >>> maintainers and triagers time more efficient - and help us while we
> are
> > >>> focusing also on Airflow 3 development soon.
> > >>>
> > >>>
> > >>>
> > >>> The Dosu founder - Devin, will send some more information soon and is
> > >>> available for questions here and in the #triage-team channel on
> Slack.
> > >>>
> > >>>
> > >>>
> > >>> Unless we hear some complaints, we will apply labelling changes in a
> > few
> > >>> days, I think this stage is not really controversial, and we will
> run a
> > >>> LAZY CONSENSUS in a few days.
> > >>>
> > >>>
> > >>>
> > >>> J. E. K. (and the Dosu team).
> > >>>
> > >>>
> > >>>
> > >>> [1] https:/ / dosu. dev/ ( https://dosu.dev/ )
> > >>>
> > >>>
> > >>>
> > >>> [2]
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> https:/ / github. com/ apache/ airflow/ blob/ main/
> > ISSUE_TRIAGE_PROCESS. rst#labels
> > >> (
> > >>
> >
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
> > >> )
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> To
> > >> unsubscribe, e-mail: dev-unsubscribe@ airflow. apache. org (
> > >> dev-unsubscribe@airflow.apache.org ) For additional commands, e-mail:
> > dev-help@
> > >> airflow. apache. org ( dev-help@airflow.apache.org )
> > >>
> > >>
> > >
> > >
> > >
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Aritra Basu <ar...@gmail.com>.
I'm a bit averse to having it answer questions (the similar issues part) as
I believe that's a slippery slope to where it could start being used to
provide actual answers. I personally would be more comfortable with
answering still being restricted to maintainers/contributors and maybe
restricting the bot to just tagging similar issues while still requiring a
human to provide advice based on that similarity.
Just my personal 2 cents. I'm not too strongly opposed to using it.
--
Regards,
Aritra Basu
On Thu, Jun 27, 2024, 2:41 AM Julian LaNeve <ju...@astronomer.io.invalid>
wrote:
> Not sure if I get an official vote here but we've been working with Devin
> and the Dosu team for Cosmos (
> https://github.com/astronomer/astronomer-cosmos ) and it's been working
> great. Excited to see this in Airflow itself!
>
> Plus they use Airflow to power their data platform behind the scenes so
> they have a vested interest in this one :)
>
> --
>
> *Julian LaNeve*
> CTO
>
> Email: julian@astronomer.io
> ( julian@astronomer.io ) Mobile: 330 509 5792
>
> On Wed, Jun 26, 2024 at 4:58 PM, Vikram Koka < vikram@astronomer.io.invalid
> > wrote:
>
> >
> >
> >
> > +1
> >
> >
> >
> > Love it!
> >
> >
> >
> > On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck < vincbeck@ apache. org (
> > vincbeck@apache.org ) > wrote:
> >
> >
> >>
> >>
> >> Fantastic idea!
> >>
> >>
> >>
> >> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> >>
> >>
> >>>
> >>>
> >>> Hello everyone,
> >>>
> >>>
> >>>
> >>> Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking
> into
> >>> employing AI / Natural Language processing to help us triage issues for
> >>> Apache Airflow. We do not want to go “all-in” into getting a chatbot to
> >>> respond to all our issues because we believe this is not how the
> >>>
> >>>
> >>
> >>
> >>
> >> community
> >>
> >>
> >>>
> >>>
> >>> is being built. We looked at various ways we can start exploring the
> >>> capabilities of the new ML/AI/Natural Language processing available.
> >>>
> >>>
> >>>
> >>> We worked with the Dosu team. They are approved by the Apache Software
> >>> Foundation infrastructure as Github integration and few ASF projects
> >>> already use it (including our friends at Superset) - they have a
> >>>
> >>>
> >>
> >>
> >>
> >> fantastic
> >>
> >>
> >>>
> >>>
> >>> offer to provide free service for open-source projects like Airflow.
> >>> Together we evaluated what we can start with and initially we have a
> >>> proposal to use auto-labeling of issues created in the Airflow
> >>>
> >>>
> >>
> >>
> >>
> >> repository.
> >>
> >>
> >>>
> >>>
> >>> We have a number of rules that are established for the triage team [2]
> >>>
> >>>
> >>
> >>
> >>
> >> but
> >>
> >>
> >>>
> >>>
> >>> those rules are mundane and difficult to follow, and generally a lot of
> >>>
> >>>
> >>
> >>
> >>
> >> our
> >>
> >>
> >>>
> >>>
> >>> issues are either not classified or badly classified, and currently we
> >>> cannot rely on the classification.
> >>>
> >>>
> >>>
> >>> What we want to start with is to re-classify our issues and apply the
> >>> labels retro-actively for all past issues as well as start applying
> them
> >>> automatically for new issues.
> >>>
> >>>
> >>>
> >>> The risk of doing it is low, and it will allow us to explore
> integration
> >>> and follow up with more elaborated integration. We have some options
> such
> >>> as getting automated proposals for answers for similar questions, as
> well
> >>> as “chat-bot generated/maintainer approved” answers - but we definitely
> >>>
> >>>
> >>
> >>
> >>
> >> do
> >>
> >>
> >>>
> >>>
> >>> not want to have bots starting to answer automatically on PRs and
> issues.
> >>>
> >>>
> >>>
> >>> We think that this will allow us to explore more ways how we can make
> >>> maintainers and triagers time more efficient - and help us while we are
> >>> focusing also on Airflow 3 development soon.
> >>>
> >>>
> >>>
> >>> The Dosu founder - Devin, will send some more information soon and is
> >>> available for questions here and in the #triage-team channel on Slack.
> >>>
> >>>
> >>>
> >>> Unless we hear some complaints, we will apply labelling changes in a
> few
> >>> days, I think this stage is not really controversial, and we will run a
> >>> LAZY CONSENSUS in a few days.
> >>>
> >>>
> >>>
> >>> J. E. K. (and the Dosu team).
> >>>
> >>>
> >>>
> >>> [1] https:/ / dosu. dev/ ( https://dosu.dev/ )
> >>>
> >>>
> >>>
> >>> [2]
> >>>
> >>>
> >>
> >>
> >>
> >> https:/ / github. com/ apache/ airflow/ blob/ main/
> ISSUE_TRIAGE_PROCESS. rst#labels
> >> (
> >>
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
> >> )
> >>
> >>
> >>
> >> --------------------------------------------------------------------- To
> >> unsubscribe, e-mail: dev-unsubscribe@ airflow. apache. org (
> >> dev-unsubscribe@airflow.apache.org ) For additional commands, e-mail:
> dev-help@
> >> airflow. apache. org ( dev-help@airflow.apache.org )
> >>
> >>
> >
> >
> >
Re: Using AI / Dosu to help us with triaging issues
Posted by Julian LaNeve <ju...@astronomer.io.INVALID>.
Not sure if I get an official vote here but we've been working with Devin and the Dosu team for Cosmos ( https://github.com/astronomer/astronomer-cosmos ) and it's been working great. Excited to see this in Airflow itself!
Plus they use Airflow to power their data platform behind the scenes so they have a vested interest in this one :)
--
*Julian LaNeve*
CTO
Email: julian@astronomer.io
( julian@astronomer.io ) Mobile: 330 509 5792
On Wed, Jun 26, 2024 at 4:58 PM, Vikram Koka < vikram@astronomer.io.invalid > wrote:
>
>
>
> +1
>
>
>
> Love it!
>
>
>
> On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck < vincbeck@ apache. org (
> vincbeck@apache.org ) > wrote:
>
>
>>
>>
>> Fantastic idea!
>>
>>
>>
>> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
>>
>>
>>>
>>>
>>> Hello everyone,
>>>
>>>
>>>
>>> Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking into
>>> employing AI / Natural Language processing to help us triage issues for
>>> Apache Airflow. We do not want to go “all-in” into getting a chatbot to
>>> respond to all our issues because we believe this is not how the
>>>
>>>
>>
>>
>>
>> community
>>
>>
>>>
>>>
>>> is being built. We looked at various ways we can start exploring the
>>> capabilities of the new ML/AI/Natural Language processing available.
>>>
>>>
>>>
>>> We worked with the Dosu team. They are approved by the Apache Software
>>> Foundation infrastructure as Github integration and few ASF projects
>>> already use it (including our friends at Superset) - they have a
>>>
>>>
>>
>>
>>
>> fantastic
>>
>>
>>>
>>>
>>> offer to provide free service for open-source projects like Airflow.
>>> Together we evaluated what we can start with and initially we have a
>>> proposal to use auto-labeling of issues created in the Airflow
>>>
>>>
>>
>>
>>
>> repository.
>>
>>
>>>
>>>
>>> We have a number of rules that are established for the triage team [2]
>>>
>>>
>>
>>
>>
>> but
>>
>>
>>>
>>>
>>> those rules are mundane and difficult to follow, and generally a lot of
>>>
>>>
>>
>>
>>
>> our
>>
>>
>>>
>>>
>>> issues are either not classified or badly classified, and currently we
>>> cannot rely on the classification.
>>>
>>>
>>>
>>> What we want to start with is to re-classify our issues and apply the
>>> labels retro-actively for all past issues as well as start applying them
>>> automatically for new issues.
>>>
>>>
>>>
>>> The risk of doing it is low, and it will allow us to explore integration
>>> and follow up with more elaborated integration. We have some options such
>>> as getting automated proposals for answers for similar questions, as well
>>> as “chat-bot generated/maintainer approved” answers - but we definitely
>>>
>>>
>>
>>
>>
>> do
>>
>>
>>>
>>>
>>> not want to have bots starting to answer automatically on PRs and issues.
>>>
>>>
>>>
>>> We think that this will allow us to explore more ways how we can make
>>> maintainers and triagers time more efficient - and help us while we are
>>> focusing also on Airflow 3 development soon.
>>>
>>>
>>>
>>> The Dosu founder - Devin, will send some more information soon and is
>>> available for questions here and in the #triage-team channel on Slack.
>>>
>>>
>>>
>>> Unless we hear some complaints, we will apply labelling changes in a few
>>> days, I think this stage is not really controversial, and we will run a
>>> LAZY CONSENSUS in a few days.
>>>
>>>
>>>
>>> J. E. K. (and the Dosu team).
>>>
>>>
>>>
>>> [1] https:/ / dosu. dev/ ( https://dosu.dev/ )
>>>
>>>
>>>
>>> [2]
>>>
>>>
>>
>>
>>
>> https:/ / github. com/ apache/ airflow/ blob/ main/ ISSUE_TRIAGE_PROCESS. rst#labels
>> (
>> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
>> )
>>
>>
>>
>> --------------------------------------------------------------------- To
>> unsubscribe, e-mail: dev-unsubscribe@ airflow. apache. org (
>> dev-unsubscribe@airflow.apache.org ) For additional commands, e-mail: dev-help@
>> airflow. apache. org ( dev-help@airflow.apache.org )
>>
>>
>
>
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Vikram Koka <vi...@astronomer.io.INVALID>.
+1
Love it!
On Wed, Jun 26, 2024 at 1:23 PM Vincent Beck <vi...@apache.org> wrote:
> Fantastic idea!
>
> On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> > Hello everyone,
> >
> > Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking into
> > employing AI / Natural Language processing to help us triage issues for
> > Apache Airflow. We do not want to go “all-in” into getting a chatbot to
> > respond to all our issues because we believe this is not how the
> community
> > is being built. We looked at various ways we can start exploring the
> > capabilities of the new ML/AI/Natural Language processing available.
> >
> > We worked with the Dosu team. They are approved by the Apache Software
> > Foundation infrastructure as Github integration and few ASF projects
> > already use it (including our friends at Superset) - they have a
> fantastic
> > offer to provide free service for open-source projects like Airflow.
> > Together we evaluated what we can start with and initially we have a
> > proposal to use auto-labeling of issues created in the Airflow
> repository.
> >
> > We have a number of rules that are established for the triage team [2]
> but
> > those rules are mundane and difficult to follow, and generally a lot of
> our
> > issues are either not classified or badly classified, and currently we
> > cannot rely on the classification.
> >
> > What we want to start with is to re-classify our issues and apply the
> > labels retro-actively for all past issues as well as start applying them
> > automatically for new issues.
> >
> > The risk of doing it is low, and it will allow us to explore integration
> > and follow up with more elaborated integration. We have some options such
> > as getting automated proposals for answers for similar questions, as well
> > as “chat-bot generated/maintainer approved” answers - but we definitely
> do
> > not want to have bots starting to answer automatically on PRs and issues.
> >
> > We think that this will allow us to explore more ways how we can make
> > maintainers and triagers time more efficient - and help us while we are
> > focusing also on Airflow 3 development soon.
> >
> > The Dosu founder - Devin, will send some more information soon and is
> > available for questions here and in the #triage-team channel on Slack.
> >
> >
> > Unless we hear some complaints, we will apply labelling changes in a few
> > days, I think this stage is not really controversial, and we will run a
> > LAZY CONSENSUS in a few days.
> >
> >
> > J. E. K. (and the Dosu team).
> >
> >
> > [1] https://dosu.dev/
> >
> > [2]
> >
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: Using AI / Dosu to help us with triaging issues
Posted by Vincent Beck <vi...@apache.org>.
Fantastic idea!
On 2024/06/26 20:12:43 Jarek Potiuk wrote:
> Hello everyone,
>
> Together with Elad, Kaxil, and the Dosu team [1], we’ve been looking into
> employing AI / Natural Language processing to help us triage issues for
> Apache Airflow. We do not want to go “all-in” into getting a chatbot to
> respond to all our issues because we believe this is not how the community
> is being built. We looked at various ways we can start exploring the
> capabilities of the new ML/AI/Natural Language processing available.
>
> We worked with the Dosu team. They are approved by the Apache Software
> Foundation infrastructure as Github integration and few ASF projects
> already use it (including our friends at Superset) - they have a fantastic
> offer to provide free service for open-source projects like Airflow.
> Together we evaluated what we can start with and initially we have a
> proposal to use auto-labeling of issues created in the Airflow repository.
>
> We have a number of rules that are established for the triage team [2] but
> those rules are mundane and difficult to follow, and generally a lot of our
> issues are either not classified or badly classified, and currently we
> cannot rely on the classification.
>
> What we want to start with is to re-classify our issues and apply the
> labels retro-actively for all past issues as well as start applying them
> automatically for new issues.
>
> The risk of doing it is low, and it will allow us to explore integration
> and follow up with more elaborated integration. We have some options such
> as getting automated proposals for answers for similar questions, as well
> as “chat-bot generated/maintainer approved” answers - but we definitely do
> not want to have bots starting to answer automatically on PRs and issues.
>
> We think that this will allow us to explore more ways how we can make
> maintainers and triagers time more efficient - and help us while we are
> focusing also on Airflow 3 development soon.
>
> The Dosu founder - Devin, will send some more information soon and is
> available for questions here and in the #triage-team channel on Slack.
>
>
> Unless we hear some complaints, we will apply labelling changes in a few
> days, I think this stage is not really controversial, and we will run a
> LAZY CONSENSUS in a few days.
>
>
> J. E. K. (and the Dosu team).
>
>
> [1] https://dosu.dev/
>
> [2]
> https://github.com/apache/airflow/blob/main/ISSUE_TRIAGE_PROCESS.rst#labels
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org