You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Freddy Demiane <fd...@google.com.INVALID> on 2024/06/12 09:53:53 UTC
Google Provider Package System Tests Dashboard
Hi Team,
At Google Cloud Composer, we developed a public dashboard that shows the
results of the System Tests of the head revision of the Google Provider
Package against the head revision of Apache Airflow. This will help detect
regressions caused by modifying an operator or a system test. At the
moment, the system tests are executed every 6 hours. Here is the link for
the dashboard:
https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
We hope this eases the development process!
Best,
Freddy
Re: Google Provider Package System Tests Dashboard
Posted by "Ferruzzi, Dennis" <fe...@amazon.com.INVALID>.
It's great to see another big provider adding a system test dashboard! As Jarek mentioned, that's a nice majority of the system tests covered now.
- ferruzzi
________________________________
From: Kaxil Naik <ka...@gmail.com>
Sent: Friday, June 14, 2024 3:14 AM
To: dev@airflow.apache.org
Subject: RE: [EXT] Google Provider Package System Tests Dashboard
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque.
Awesome stuff Freddy
On Fri, 14 Jun 2024 at 08:54, Michał Modras <mi...@google.com.invalid>
wrote:
> Great work Freddy!
>
> On Thu, Jun 13, 2024 at 4:08 AM Wei Lee <we...@gmail.com> wrote:
>
> > This is great!
> >
> > Best,
> > Wei
> >
> > > On Jun 12, 2024, at 9:47 PM, Bishundeo, Rajeshwar
> > <rb...@amazon.com.INVALID> wrote:
> > >
> > > Fantastic job from the Google team. Love it!!
> > >
> > > -- Rajesh
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 2024-06-12, 9:20 AM, "Pankaj Koti" <pankaj.koti@astronomer.io.INVA
> > <ma...@astronomer.io.INVA> <mailto:
> > pankaj.koti@astronomer.io.INVA>LID> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si
> vous
> > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas
> > certain que le contenu ne présente aucun risque.
> > >
> > >
> > >
> > >
> > >
> > >
> > > yes, indeed a great one!
> > >
> > >
> > >
> > >
> > > Best regards,
> > >
> > >
> > > *Pankaj Koti*
> > > Senior Software Engineer (Airflow OSS Engineering team)
> > > Location: Pune, Maharashtra, India
> > > Timezone: Indian Standard Time (IST)
> > >
> > >
> > >
> > >
> > > On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vincbeck@apache.org
> > <ma...@apache.org> <ma...@apache.org>> wrote:
> > >
> > >
> > >> Really nice!!
> > >>
> > >> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> > >>>> I believe we should include a reference to it in the Google provider
> > >>> documentation.
> > >>>
> > >>> Already there: https://github.com/apache/airflow/pull/40102 <
> > https://github.com/apache/airflow/pull/40102> in the
> > >> README
> > >>> documentation (which is where it should be as it is mostly
> > >>> maintainer/contributor docs rather than user's docs.
> > >>>
> > >>> J.
> > >>>
> > >>>
> > >>> On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <
> ags.pankaj1996@gmail.com
> > <ma...@gmail.com> <ma...@gmail.com>>
> > >>> wrote:
> > >>>
> > >>>> Nice! Thanks for sharing. I believe we should include a reference to
> > >> it in
> > >>>> the Google provider documentation.
> > >>>>
> > >>>> On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
> > >> <fdemiane@google.com.inva <ma...@google.com.inva> <mailto:
> > fdemiane@google.com.inva>lid
> > >>>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi Team,
> > >>>>>
> > >>>>> At Google Cloud Composer, we developed a public dashboard that
> shows
> > >> the
> > >>>>> results of the System Tests of the head revision of the Google
> > >> Provider
> > >>>>> Package against the head revision of Apache Airflow. This will help
> > >>>> detect
> > >>>>> regressions caused by modifying an operator or a system test. At
> the
> > >>>>> moment, the system tests are executed every 6 hours. Here is the
> > >> link for
> > >>>>> the dashboard:
> > >>>>>
> > >>
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html
> > <https://storage.googleapis.com/providers-dashboard-html/dashboard.html>
> .
> > >>>>> We hope this eases the development process!
> > >>>>>
> > >>>>> Best,
> > >>>>> Freddy
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> > dev-unsubscribe@airflow.apache.org> <mailto:
> > dev-unsubscribe@airflow.apache.org>
> > >> For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> > dev-help@airflow.apache.org> <ma...@airflow.apache.org>
> > >>
> > >>
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> > dev-unsubscribe@airflow.apache.org>
> > > For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> > dev-help@airflow.apache.org>
> >
>
Re: Google Provider Package System Tests Dashboard
Posted by Kaxil Naik <ka...@gmail.com>.
Awesome stuff Freddy
On Fri, 14 Jun 2024 at 08:54, Michał Modras <mi...@google.com.invalid>
wrote:
> Great work Freddy!
>
> On Thu, Jun 13, 2024 at 4:08 AM Wei Lee <we...@gmail.com> wrote:
>
> > This is great!
> >
> > Best,
> > Wei
> >
> > > On Jun 12, 2024, at 9:47 PM, Bishundeo, Rajeshwar
> > <rb...@amazon.com.INVALID> wrote:
> > >
> > > Fantastic job from the Google team. Love it!!
> > >
> > > -- Rajesh
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 2024-06-12, 9:20 AM, "Pankaj Koti" <pankaj.koti@astronomer.io.INVA
> > <ma...@astronomer.io.INVA> <mailto:
> > pankaj.koti@astronomer.io.INVA>LID> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si
> vous
> > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas
> > certain que le contenu ne présente aucun risque.
> > >
> > >
> > >
> > >
> > >
> > >
> > > yes, indeed a great one!
> > >
> > >
> > >
> > >
> > > Best regards,
> > >
> > >
> > > *Pankaj Koti*
> > > Senior Software Engineer (Airflow OSS Engineering team)
> > > Location: Pune, Maharashtra, India
> > > Timezone: Indian Standard Time (IST)
> > >
> > >
> > >
> > >
> > > On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vincbeck@apache.org
> > <ma...@apache.org> <ma...@apache.org>> wrote:
> > >
> > >
> > >> Really nice!!
> > >>
> > >> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> > >>>> I believe we should include a reference to it in the Google provider
> > >>> documentation.
> > >>>
> > >>> Already there: https://github.com/apache/airflow/pull/40102 <
> > https://github.com/apache/airflow/pull/40102> in the
> > >> README
> > >>> documentation (which is where it should be as it is mostly
> > >>> maintainer/contributor docs rather than user's docs.
> > >>>
> > >>> J.
> > >>>
> > >>>
> > >>> On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <
> ags.pankaj1996@gmail.com
> > <ma...@gmail.com> <ma...@gmail.com>>
> > >>> wrote:
> > >>>
> > >>>> Nice! Thanks for sharing. I believe we should include a reference to
> > >> it in
> > >>>> the Google provider documentation.
> > >>>>
> > >>>> On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
> > >> <fdemiane@google.com.inva <ma...@google.com.inva> <mailto:
> > fdemiane@google.com.inva>lid
> > >>>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi Team,
> > >>>>>
> > >>>>> At Google Cloud Composer, we developed a public dashboard that
> shows
> > >> the
> > >>>>> results of the System Tests of the head revision of the Google
> > >> Provider
> > >>>>> Package against the head revision of Apache Airflow. This will help
> > >>>> detect
> > >>>>> regressions caused by modifying an operator or a system test. At
> the
> > >>>>> moment, the system tests are executed every 6 hours. Here is the
> > >> link for
> > >>>>> the dashboard:
> > >>>>>
> > >>
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html
> > <https://storage.googleapis.com/providers-dashboard-html/dashboard.html>
> .
> > >>>>> We hope this eases the development process!
> > >>>>>
> > >>>>> Best,
> > >>>>> Freddy
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> > dev-unsubscribe@airflow.apache.org> <mailto:
> > dev-unsubscribe@airflow.apache.org>
> > >> For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> > dev-help@airflow.apache.org> <ma...@airflow.apache.org>
> > >>
> > >>
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> > dev-unsubscribe@airflow.apache.org>
> > > For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> > dev-help@airflow.apache.org>
> >
>
Re: Google Provider Package System Tests Dashboard
Posted by Michał Modras <mi...@google.com.INVALID>.
Great work Freddy!
On Thu, Jun 13, 2024 at 4:08 AM Wei Lee <we...@gmail.com> wrote:
> This is great!
>
> Best,
> Wei
>
> > On Jun 12, 2024, at 9:47 PM, Bishundeo, Rajeshwar
> <rb...@amazon.com.INVALID> wrote:
> >
> > Fantastic job from the Google team. Love it!!
> >
> > -- Rajesh
> >
> >
> >
> >
> >
> >
> > On 2024-06-12, 9:20 AM, "Pankaj Koti" <pankaj.koti@astronomer.io.INVA
> <ma...@astronomer.io.INVA> <mailto:
> pankaj.koti@astronomer.io.INVA>LID> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> >
> >
> >
> > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous
> ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas
> certain que le contenu ne présente aucun risque.
> >
> >
> >
> >
> >
> >
> > yes, indeed a great one!
> >
> >
> >
> >
> > Best regards,
> >
> >
> > *Pankaj Koti*
> > Senior Software Engineer (Airflow OSS Engineering team)
> > Location: Pune, Maharashtra, India
> > Timezone: Indian Standard Time (IST)
> >
> >
> >
> >
> > On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vincbeck@apache.org
> <ma...@apache.org> <ma...@apache.org>> wrote:
> >
> >
> >> Really nice!!
> >>
> >> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> >>>> I believe we should include a reference to it in the Google provider
> >>> documentation.
> >>>
> >>> Already there: https://github.com/apache/airflow/pull/40102 <
> https://github.com/apache/airflow/pull/40102> in the
> >> README
> >>> documentation (which is where it should be as it is mostly
> >>> maintainer/contributor docs rather than user's docs.
> >>>
> >>> J.
> >>>
> >>>
> >>> On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ags.pankaj1996@gmail.com
> <ma...@gmail.com> <ma...@gmail.com>>
> >>> wrote:
> >>>
> >>>> Nice! Thanks for sharing. I believe we should include a reference to
> >> it in
> >>>> the Google provider documentation.
> >>>>
> >>>> On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
> >> <fdemiane@google.com.inva <ma...@google.com.inva> <mailto:
> fdemiane@google.com.inva>lid
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Hi Team,
> >>>>>
> >>>>> At Google Cloud Composer, we developed a public dashboard that shows
> >> the
> >>>>> results of the System Tests of the head revision of the Google
> >> Provider
> >>>>> Package against the head revision of Apache Airflow. This will help
> >>>> detect
> >>>>> regressions caused by modifying an operator or a system test. At the
> >>>>> moment, the system tests are executed every 6 hours. Here is the
> >> link for
> >>>>> the dashboard:
> >>>>>
> >> https://storage.googleapis.com/providers-dashboard-html/dashboard.html
> <https://storage.googleapis.com/providers-dashboard-html/dashboard.html> .
> >>>>> We hope this eases the development process!
> >>>>>
> >>>>> Best,
> >>>>> Freddy
> >>>>>
> >>>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> dev-unsubscribe@airflow.apache.org> <mailto:
> dev-unsubscribe@airflow.apache.org>
> >> For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> dev-help@airflow.apache.org> <ma...@airflow.apache.org>
> >>
> >>
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <mailto:
> dev-unsubscribe@airflow.apache.org>
> > For additional commands, e-mail: dev-help@airflow.apache.org <mailto:
> dev-help@airflow.apache.org>
>
Re: Google Provider Package System Tests Dashboard
Posted by Wei Lee <we...@gmail.com>.
This is great!
Best,
Wei
> On Jun 12, 2024, at 9:47 PM, Bishundeo, Rajeshwar <rb...@amazon.com.INVALID> wrote:
>
> Fantastic job from the Google team. Love it!!
>
> -- Rajesh
>
>
>
>
>
>
> On 2024-06-12, 9:20 AM, "Pankaj Koti" <pankaj.koti@astronomer.io.INVA <ma...@astronomer.io.INVA> <ma...@astronomer.io.INVA>LID> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque.
>
>
>
>
>
>
> yes, indeed a great one!
>
>
>
>
> Best regards,
>
>
> *Pankaj Koti*
> Senior Software Engineer (Airflow OSS Engineering team)
> Location: Pune, Maharashtra, India
> Timezone: Indian Standard Time (IST)
>
>
>
>
> On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vincbeck@apache.org <ma...@apache.org> <ma...@apache.org>> wrote:
>
>
>> Really nice!!
>>
>> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
>>>> I believe we should include a reference to it in the Google provider
>>> documentation.
>>>
>>> Already there: https://github.com/apache/airflow/pull/40102 <https://github.com/apache/airflow/pull/40102> in the
>> README
>>> documentation (which is where it should be as it is mostly
>>> maintainer/contributor docs rather than user's docs.
>>>
>>> J.
>>>
>>>
>>> On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ags.pankaj1996@gmail.com <ma...@gmail.com> <ma...@gmail.com>>
>>> wrote:
>>>
>>>> Nice! Thanks for sharing. I believe we should include a reference to
>> it in
>>>> the Google provider documentation.
>>>>
>>>> On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
>> <fdemiane@google.com.inva <ma...@google.com.inva> <ma...@google.com.inva>lid
>>>>>
>>>> wrote:
>>>>
>>>>> Hi Team,
>>>>>
>>>>> At Google Cloud Composer, we developed a public dashboard that shows
>> the
>>>>> results of the System Tests of the head revision of the Google
>> Provider
>>>>> Package against the head revision of Apache Airflow. This will help
>>>> detect
>>>>> regressions caused by modifying an operator or a system test. At the
>>>>> moment, the system tests are executed every 6 hours. Here is the
>> link for
>>>>> the dashboard:
>>>>>
>> https://storage.googleapis.com/providers-dashboard-html/dashboard.html <https://storage.googleapis.com/providers-dashboard-html/dashboard.html> .
>>>>> We hope this eases the development process!
>>>>>
>>>>> Best,
>>>>> Freddy
>>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <ma...@airflow.apache.org> <ma...@airflow.apache.org>
>> For additional commands, e-mail: dev-help@airflow.apache.org <ma...@airflow.apache.org> <ma...@airflow.apache.org>
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <ma...@airflow.apache.org>
> For additional commands, e-mail: dev-help@airflow.apache.org <ma...@airflow.apache.org>
Re: Google Provider Package System Tests Dashboard
Posted by "Bishundeo, Rajeshwar" <rb...@amazon.com.INVALID>.
Fantastic job from the Google team. Love it!!
-- Rajesh
On 2024-06-12, 9:20 AM, "Pankaj Koti" <pankaj.koti@astronomer.io.INVA <ma...@astronomer.io.INVA>LID> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque.
yes, indeed a great one!
Best regards,
*Pankaj Koti*
Senior Software Engineer (Airflow OSS Engineering team)
Location: Pune, Maharashtra, India
Timezone: Indian Standard Time (IST)
On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vincbeck@apache.org <ma...@apache.org>> wrote:
> Really nice!!
>
> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> > > I believe we should include a reference to it in the Google provider
> > documentation.
> >
> > Already there: https://github.com/apache/airflow/pull/40102 <https://github.com/apache/airflow/pull/40102> in the
> README
> > documentation (which is where it should be as it is mostly
> > maintainer/contributor docs rather than user's docs.
> >
> > J.
> >
> >
> > On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ags.pankaj1996@gmail.com <ma...@gmail.com>>
> > wrote:
> >
> > > Nice! Thanks for sharing. I believe we should include a reference to
> it in
> > > the Google provider documentation.
> > >
> > > On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
> <fdemiane@google.com.inva <ma...@google.com.inva>lid
> > > >
> > > wrote:
> > >
> > > > Hi Team,
> > > >
> > > > At Google Cloud Composer, we developed a public dashboard that shows
> the
> > > > results of the System Tests of the head revision of the Google
> Provider
> > > > Package against the head revision of Apache Airflow. This will help
> > > detect
> > > > regressions caused by modifying an operator or a system test. At the
> > > > moment, the system tests are executed every 6 hours. Here is the
> link for
> > > > the dashboard:
> > > >
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html <https://storage.googleapis.com/providers-dashboard-html/dashboard.html> .
> > > > We hope this eases the development process!
> > > >
> > > > Best,
> > > > Freddy
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org <ma...@airflow.apache.org>
> For additional commands, e-mail: dev-help@airflow.apache.org <ma...@airflow.apache.org>
>
>
Re: Google Provider Package System Tests Dashboard
Posted by Pankaj Koti <pa...@astronomer.io.INVALID>.
yes, indeed a great one!
Best regards,
*Pankaj Koti*
Senior Software Engineer (Airflow OSS Engineering team)
Location: Pune, Maharashtra, India
Timezone: Indian Standard Time (IST)
On Wed, Jun 12, 2024 at 6:42 PM Vincent Beck <vi...@apache.org> wrote:
> Really nice!!
>
> On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> > > I believe we should include a reference to it in the Google provider
> > documentation.
> >
> > Already there: https://github.com/apache/airflow/pull/40102 in the
> README
> > documentation (which is where it should be as it is mostly
> > maintainer/contributor docs rather than user's docs.
> >
> > J.
> >
> >
> > On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ag...@gmail.com>
> > wrote:
> >
> > > Nice! Thanks for sharing. I believe we should include a reference to
> it in
> > > the Google provider documentation.
> > >
> > > On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane
> <fdemiane@google.com.invalid
> > > >
> > > wrote:
> > >
> > > > Hi Team,
> > > >
> > > > At Google Cloud Composer, we developed a public dashboard that shows
> the
> > > > results of the System Tests of the head revision of the Google
> Provider
> > > > Package against the head revision of Apache Airflow. This will help
> > > detect
> > > > regressions caused by modifying an operator or a system test. At the
> > > > moment, the system tests are executed every 6 hours. Here is the
> link for
> > > > the dashboard:
> > > >
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
> > > > We hope this eases the development process!
> > > >
> > > > Best,
> > > > Freddy
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: Google Provider Package System Tests Dashboard
Posted by Vincent Beck <vi...@apache.org>.
Really nice!!
On 2024/06/12 11:38:13 Jarek Potiuk wrote:
> > I believe we should include a reference to it in the Google provider
> documentation.
>
> Already there: https://github.com/apache/airflow/pull/40102 in the README
> documentation (which is where it should be as it is mostly
> maintainer/contributor docs rather than user's docs.
>
> J.
>
>
> On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ag...@gmail.com>
> wrote:
>
> > Nice! Thanks for sharing. I believe we should include a reference to it in
> > the Google provider documentation.
> >
> > On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane <fdemiane@google.com.invalid
> > >
> > wrote:
> >
> > > Hi Team,
> > >
> > > At Google Cloud Composer, we developed a public dashboard that shows the
> > > results of the System Tests of the head revision of the Google Provider
> > > Package against the head revision of Apache Airflow. This will help
> > detect
> > > regressions caused by modifying an operator or a system test. At the
> > > moment, the system tests are executed every 6 hours. Here is the link for
> > > the dashboard:
> > > https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
> > > We hope this eases the development process!
> > >
> > > Best,
> > > Freddy
> > >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: Google Provider Package System Tests Dashboard
Posted by Jarek Potiuk <ja...@potiuk.com>.
> I believe we should include a reference to it in the Google provider
documentation.
Already there: https://github.com/apache/airflow/pull/40102 in the README
documentation (which is where it should be as it is mostly
maintainer/contributor docs rather than user's docs.
J.
On Wed, Jun 12, 2024 at 1:34 PM Pankaj Singh <ag...@gmail.com>
wrote:
> Nice! Thanks for sharing. I believe we should include a reference to it in
> the Google provider documentation.
>
> On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane <fdemiane@google.com.invalid
> >
> wrote:
>
> > Hi Team,
> >
> > At Google Cloud Composer, we developed a public dashboard that shows the
> > results of the System Tests of the head revision of the Google Provider
> > Package against the head revision of Apache Airflow. This will help
> detect
> > regressions caused by modifying an operator or a system test. At the
> > moment, the system tests are executed every 6 hours. Here is the link for
> > the dashboard:
> > https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
> > We hope this eases the development process!
> >
> > Best,
> > Freddy
> >
>
Re: Google Provider Package System Tests Dashboard
Posted by Pankaj Singh <ag...@gmail.com>.
Nice! Thanks for sharing. I believe we should include a reference to it in
the Google provider documentation.
On Wed, Jun 12, 2024 at 3:24 PM Freddy Demiane <fd...@google.com.invalid>
wrote:
> Hi Team,
>
> At Google Cloud Composer, we developed a public dashboard that shows the
> results of the System Tests of the head revision of the Google Provider
> Package against the head revision of Apache Airflow. This will help detect
> regressions caused by modifying an operator or a system test. At the
> moment, the system tests are executed every 6 hours. Here is the link for
> the dashboard:
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
> We hope this eases the development process!
>
> Best,
> Freddy
>
Re: Google Provider Package System Tests Dashboard
Posted by Jarek Potiuk <ja...@potiuk.com>.
Wooohohoo! This is fantastic!
We also have it now added to our "ecosystem" page:
https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards
for easier finding.
One comment - something that not everyone might be aware of.
This one generally completes *AIP-4*:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324355
that I created in *September 2018(!)*: "AIP-4 Automation of System Tests
[Deps: AIP-47]". It had never been voted formally because we did not know
if this would be possible, but it's been followed by AIP-47 and a number of
other smaller or bigger updates in our code, policies (Mixed Governance
model) and smaller or bigger improvements in our development framework.
The AIP-4 is one of the oldest AIPs. It required a lot of effort and
coordination and engagement of the big stakeholders - Amazon, Google,
Astronomer. But finally all three of our stakeholders now manage and
operate the System test dashboards that give us much more certainty that
our important integrations are working fine with external services. I have
no hard data to back it up but with those three dashboards we have, we are
going to prevent regressions for way more than 70% of the usage of all the
"service" community providers (including those who use managed and on
premise Airflow)
One of the first comment of Ash in the AIP was:
"The main blocker to being able to run integration tests against a live
GCP in any kind of automated fashion is cost: we'd have to get Google (or
someone else) to sponsor us with GCP credits."
As it turned out, over the years we found a solution where we have a joint
effort between the stakeholders and PMC and this main blocker is solved
quite differently. I really hope it will help us to keep the quality of our
important service integrations, and that others will follow with their own
dashboards. And yes - that means that I would love the Azure team to
finally show up and do this as the first step. They are pretty much
completely absent in the community, but it would be great if they actually
care and are willing to be part of the community).
Astronomer, Google and Amazon clearly paved the way for them.
J.
On Wed, Jun 12, 2024 at 11:54 AM Freddy Demiane <fd...@google.com.invalid>
wrote:
> Hi Team,
>
> At Google Cloud Composer, we developed a public dashboard that shows the
> results of the System Tests of the head revision of the Google Provider
> Package against the head revision of Apache Airflow. This will help detect
> regressions caused by modifying an operator or a system test. At the
> moment, the system tests are executed every 6 hours. Here is the link for
> the dashboard:
> https://storage.googleapis.com/providers-dashboard-html/dashboard.html .
> We hope this eases the development process!
>
> Best,
> Freddy
>