You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@airflow.apache.org by Jarek Potiuk <Ja...@polidea.com> on 2020/05/26 20:43:06 UTC

[VOTE] Airflow Backport Providers 2020.5.20rc3

Hey all,

I have cut Airflow Backport Providers 2020.5.20rc3. This email is calling a
vote on the release,
which will last for 72 hours - which means that it will end on Fri May 29
22:42:38 CEST 2020.

Consider this my (binding) +1.

The changes vs. rc2 are explained at the end of the email.

Airflow Backport Providers 2020.5.20rc3 are available at:
https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/

*apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a source
release that comes
 with INSTALL instructions.

*apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz* are
the binary
 Python "sdist" release.

Public keys are available at:
https://dist.apache.org/repos/dist/release/airflow/KEYS

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason


Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Please note that the version number excludes the 'rcX' string, so it's now
simply 2020.5.20. This will allow us to rename the artifact without
modifying
the artifact checksums when we actually release it.

Each of the packages contains a detailed changelog. Here is the list of
links to
the released packages and changelogs:

https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/

J.


Appendix: The changes vs. RC2:

* During testing we found a few (20 out of 537) inconsistencies in the
names of the classes vs. the agreed AIP-21 standard. We fixed them all and
also introduced automated tests that will fail PR in case someone adds an
inconsistently named class.

* Several packages failed to import cleanly in 1.10. That was a great idea
from one of the PMCs (Bas Hanrenslak) to automatically import all the
classes from all the providers and see if they are importable in 1.10. We
identified a few operators in a few providers (papermill, odbc, hive, and a
few others) that would not import properly on 1.10. This is all now fixed
(except papermill - see below). The good thing is that we also added
automated tests for this: so we will be testing the importability of
backported packages with every PR. This means that the backport packages
will continue to be importable in the future and we check it with every
single PR!

Of course, importability does not guarantee that the operators will work
with !00% certainty - this can only be assured by automated system tests -
this is something we will be working on the coming months.

* Papermill operator has proven to be problematic. It uses the Airflow 2.0
lineage functionality that is not compatible with 1.10. That's why we
decided to remove papermill from the released backport packages and remove
the previous RCs for it.

* The most important reason why we needed RC3. We noticed that while the
README package instructions were all fine, the CHANGELOG and INSTALL in
source packages contained changelog/Install instructions for Airflow 2.0.
We fixed that and the INSTALL and CHANGELOG in the packages contain
instructions/changelog for the backport packages only, not for the whole
Airflow.

Cheers,

Jarek

-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

HAH.... Again canceling... seems like a routine now :).

Seems that enough people (everyone who spoke) care for consistency and
since (as discussed in
https://lists.apache.org/thread.html/r456565d372dcd7199d5ca34881d43911bf2c6ce623e515252a3aec73%40%3Cdev.airflow.apache.org%3E
)
we found an inconsistency in the names of transfer operators, the RC3 for
backport operators is cancelled.

It's likely we will have another RC on Tuesday, It seems that we are moving
away from the calver date 2020.5.20 so maybe we should consider abandoning
2020.5.20 and releasing a new RC1 candidate with a new date - 2020.06.05rc1
to be precise? Hopefully future releases will be much much faster :)

WDYT?

J.




On Fri, May 29, 2020 at 1:53 PM Kaxil Naik <ka...@gmail.com> wrote:

> Thanks Jarek :)
>
> On Fri, May 29, 2020 at 12:46 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > Absolutely. This discussion made me think we want to work on RC4 :).  I
> > think that's better to clarify it now since this is something that stay
> > with us for quite some time. I will send a separate vote shortly.
> >
> > J.
> >
> > On Fri, May 29, 2020 at 1:11 PM Kaxil Naik <ka...@gmail.com> wrote:
> >
> > > We might want to create a separate DISCUSS / VOTING Thread for
> > > https://apache-airflow.slack.com/archives/CCPRP7943/p1590742848402600
> > >
> > > i.e. Naming convention for the AToB Operators or Transfer Operators
> > >
> > > I think we should wait for the results of that before we release the
> > > Backport Packages i.e. create RC4 :(
> > >
> > > What do you think?
> > >
> > > Regards,
> > > Kaxil
> > >
> > >
> > >
> > > On Fri, May 29, 2020 at 11:34 AM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > > wrote:
> > >
> > > > Fixed :). Shift + Refresh
> > > >
> > > > On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <
> > Jarek.Potiuk@polidea.com>
> > > > wrote:
> > > >
> > > > > Sha of sha .... Wow.. That's something  like master level of
> security
> > > :).
> > > > >
> > > > > On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com>
> > > wrote:
> > > > >
> > > > >> I think we have some extra files at
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > > > >> (maybe
> > > > >> erroneously generated by the Bash scripts)
> > > > >>
> > > > >> Examples:
> > > > >>
> > > > >>    -
> > > > >>
> > > > >>
> > > >
> > >
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > > > >>    <
> > > > >>
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > > > >> >
> > > > >>    -
> > > > >>
> > > > >>
> > > >
> > >
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > > >>    <
> > > > >>
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > > >> >
> > > > >>    -
> > > > >>
> > > > >>
> > > >
> > >
> >
> apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > > >>    <
> > > > >>
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > > >> >
> > > > >>
> > > > >>
> > > > >> Can we remove those first, please, they are not a blocker to the
> > > release
> > > > >> but makes it difficult to view the actual packages?
> > > > >>
> > > > >> Regards,
> > > > >> Kaxil
> > > > >>
> > > > >> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <
> > > Jarek.Potiuk@polidea.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Looking forward to some PMC votes :) But happy to hear more
> > feedback
> > > > >> from
> > > > >> > others!
> > > > >> >
> > > > >> > Just a kind reminder (not sure if visible to everyone but let me
> > > try):
> > > > >> >
> > > > >> > [image: majestic_rabbit.jpg]
> > > > >> >
> > > > >> >  Fri May 29 22:42:38 CEST 2020.
> > > > >> >
> > > > >> > J.
> > > > >> >
> > > > >> >
> > > > >> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <
> > > > Jarek.Potiuk@polidea.com
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Indeed What an oversight :). Thanks for +1 - even non-binding
> is
> > > > super
> > > > >> >> important :) Anyone else trying the backport packages is great
> -
> > I
> > > > >> think
> > > > >> >> it's super important for the Airflow community and great step
> to
> > > ease
> > > > >> >> migration to 2.0 :D
> > > > >> >>
> > > > >> >> J.
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> > > > >> >> <ba...@godatadriven.com.invalid> wrote:
> > > > >> >>
> > > > >> >>> Thanks for the compliment but I’m no PMC so here’s my
> > non-binding
> > > +1
> > > > >> :-)
> > > > >> >>>
> > > > >> >>> Bas
> > > > >> >>>
> > > > >> >>> On 26 May 2020, at 22:43, Jarek Potiuk <
> > Jarek.Potiuk@polidea.com
> > > > >> <mailto:
> > > > >> >>> Jarek.Potiuk@polidea.com>> wrote:
> > > > >> >>>
> > > > >> >>> Hey all,
> > > > >> >>>
> > > > >> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email
> > is
> > > > >> >>> calling a
> > > > >> >>> vote on the release,
> > > > >> >>> which will last for 72 hours - which means that it will end on
> > Fri
> > > > >> May 29
> > > > >> >>> 22:42:38 CEST 2020.
> > > > >> >>>
> > > > >> >>> Consider this my (binding) +1.
> > > > >> >>>
> > > > >> >>> The changes vs. rc2 are explained at the end of the email.
> > > > >> >>>
> > > > >> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz*
> > is
> > > a
> > > > >> >>> source
> > > > >> >>> release that comes
> > > > >> >>> with INSTALL instructions.
> > > > >> >>>
> > > > >> >>>
> > > >
> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
> > > > >> >>> are
> > > > >> >>> the binary
> > > > >> >>> Python "sdist" release.
> > > > >> >>>
> > > > >> >>> Public keys are available at:
> > > > >> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> > > > >> >>>
> > > > >> >>> Please vote accordingly:
> > > > >> >>>
> > > > >> >>> [ ] +1 approve
> > > > >> >>> [ ] +0 no opinion
> > > > >> >>> [ ] -1 disapprove with the reason
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> Only votes from PMC members are binding, but members of the
> > > > community
> > > > >> are
> > > > >> >>> encouraged to test the release and vote with "(non-binding)".
> > > > >> >>>
> > > > >> >>> Please note that the version number excludes the 'rcX' string,
> > so
> > > > it's
> > > > >> >>> now
> > > > >> >>> simply 2020.5.20. This will allow us to rename the artifact
> > > without
> > > > >> >>> modifying
> > > > >> >>> the artifact checksums when we actually release it.
> > > > >> >>>
> > > > >> >>> Each of the packages contains a detailed changelog. Here is
> the
> > > list
> > > > >> of
> > > > >> >>> links to
> > > > >> >>> the released packages and changelogs:
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
> > > > >> >>>
> > > > >> >>> J.
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> Appendix: The changes vs. RC2:
> > > > >> >>>
> > > > >> >>> * During testing we found a few (20 out of 537)
> inconsistencies
> > in
> > > > the
> > > > >> >>> names of the classes vs. the agreed AIP-21 standard. We fixed
> > them
> > > > all
> > > > >> >>> and
> > > > >> >>> also introduced automated tests that will fail PR in case
> > someone
> > > > >> adds an
> > > > >> >>> inconsistently named class.
> > > > >> >>>
> > > > >> >>> * Several packages failed to import cleanly in 1.10. That was
> a
> > > > great
> > > > >> >>> idea
> > > > >> >>> from one of the PMCs (Bas Hanrenslak) to automatically import
> > all
> > > > the
> > > > >> >>> classes from all the providers and see if they are importable
> in
> > > > >> 1.10. We
> > > > >> >>> identified a few operators in a few providers (papermill,
> odbc,
> > > > hive,
> > > > >> >>> and a
> > > > >> >>> few others) that would not import properly on 1.10. This is
> all
> > > now
> > > > >> fixed
> > > > >> >>> (except papermill - see below). The good thing is that we also
> > > added
> > > > >> >>> automated tests for this: so we will be testing the
> > importability
> > > of
> > > > >> >>> backported packages with every PR. This means that the
> backport
> > > > >> packages
> > > > >> >>> will continue to be importable in the future and we check it
> > with
> > > > >> every
> > > > >> >>> single PR!
> > > > >> >>>
> > > > >> >>> Of course, importability does not guarantee that the operators
> > > will
> > > > >> work
> > > > >> >>> with !00% certainty - this can only be assured by automated
> > system
> > > > >> tests
> > > > >> >>> -
> > > > >> >>> this is something we will be working on the coming months.
> > > > >> >>>
> > > > >> >>> * Papermill operator has proven to be problematic. It uses the
> > > > Airflow
> > > > >> >>> 2.0
> > > > >> >>> lineage functionality that is not compatible with 1.10. That's
> > why
> > > > we
> > > > >> >>> decided to remove papermill from the released backport
> packages
> > > and
> > > > >> >>> remove
> > > > >> >>> the previous RCs for it.
> > > > >> >>>
> > > > >> >>> * The most important reason why we needed RC3. We noticed that
> > > while
> > > > >> the
> > > > >> >>> README package instructions were all fine, the CHANGELOG and
> > > INSTALL
> > > > >> in
> > > > >> >>> source packages contained changelog/Install instructions for
> > > Airflow
> > > > >> 2.0.
> > > > >> >>> We fixed that and the INSTALL and CHANGELOG in the packages
> > > contain
> > > > >> >>> instructions/changelog for the backport packages only, not for
> > the
> > > > >> whole
> > > > >> >>> Airflow.
> > > > >> >>>
> > > > >> >>> Cheers,
> > > > >> >>>
> > > > >> >>> Jarek
> > > > >> >>>
> > > > >> >>> --
> > > > >> >>>
> > > > >> >>> Jarek Potiuk
> > > > >> >>> Polidea <https://www.polidea.com/> | Principal Software
> > Engineer
> > > > >> >>>
> > > > >> >>> M: +48 660 796 129 <+48660796129>
> > > > >> >>> [image: Polidea] <https://www.polidea.com/>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>
> > > > >> >> --
> > > > >> >>
> > > > >> >> Jarek Potiuk
> > > > >> >> Polidea <https://www.polidea.com/> | Principal Software
> Engineer
> > > > >> >>
> > > > >> >> M: +48 660 796 129 <+48660796129>
> > > > >> >> [image: Polidea] <https://www.polidea.com/>
> > > > >> >>
> > > > >> >>
> > > > >> >
> > > > >> > --
> > > > >> >
> > > > >> > Jarek Potiuk
> > > > >> > Polidea <https://www.polidea.com/> | Principal Software
> Engineer
> > > > >> >
> > > > >> > M: +48 660 796 129 <+48660796129>
> > > > >> > [image: Polidea] <https://www.polidea.com/>
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Jarek Potiuk
> > > > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > > >
> > > > > M: +48 660 796 129 <+48660796129>
> > > > > [image: Polidea] <https://www.polidea.com/>
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Jarek Potiuk
> > > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > >
> > > > M: +48 660 796 129 <+48660796129>
> > > > [image: Polidea] <https://www.polidea.com/>
> > > >
> > >
> >
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Kaxil Naik <ka...@gmail.com>.

Thanks Jarek :)

On Fri, May 29, 2020 at 12:46 PM Jarek Potiuk <Ja...@polidea.com>
wrote:

> Absolutely. This discussion made me think we want to work on RC4 :).  I
> think that's better to clarify it now since this is something that stay
> with us for quite some time. I will send a separate vote shortly.
>
> J.
>
> On Fri, May 29, 2020 at 1:11 PM Kaxil Naik <ka...@gmail.com> wrote:
>
> > We might want to create a separate DISCUSS / VOTING Thread for
> > https://apache-airflow.slack.com/archives/CCPRP7943/p1590742848402600
> >
> > i.e. Naming convention for the AToB Operators or Transfer Operators
> >
> > I think we should wait for the results of that before we release the
> > Backport Packages i.e. create RC4 :(
> >
> > What do you think?
> >
> > Regards,
> > Kaxil
> >
> >
> >
> > On Fri, May 29, 2020 at 11:34 AM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> > > Fixed :). Shift + Refresh
> > >
> > > On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > > wrote:
> > >
> > > > Sha of sha .... Wow.. That's something  like master level of security
> > :).
> > > >
> > > > On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com>
> > wrote:
> > > >
> > > >> I think we have some extra files at
> > > >>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > > >> (maybe
> > > >> erroneously generated by the Bash scripts)
> > > >>
> > > >> Examples:
> > > >>
> > > >>    -
> > > >>
> > > >>
> > >
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > > >>    <
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > > >> >
> > > >>    -
> > > >>
> > > >>
> > >
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > >>    <
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > >> >
> > > >>    -
> > > >>
> > > >>
> > >
> >
> apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > >>    <
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > > >> >
> > > >>
> > > >>
> > > >> Can we remove those first, please, they are not a blocker to the
> > release
> > > >> but makes it difficult to view the actual packages?
> > > >>
> > > >> Regards,
> > > >> Kaxil
> > > >>
> > > >> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <
> > Jarek.Potiuk@polidea.com>
> > > >> wrote:
> > > >>
> > > >> > Looking forward to some PMC votes :) But happy to hear more
> feedback
> > > >> from
> > > >> > others!
> > > >> >
> > > >> > Just a kind reminder (not sure if visible to everyone but let me
> > try):
> > > >> >
> > > >> > [image: majestic_rabbit.jpg]
> > > >> >
> > > >> >  Fri May 29 22:42:38 CEST 2020.
> > > >> >
> > > >> > J.
> > > >> >
> > > >> >
> > > >> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <
> > > Jarek.Potiuk@polidea.com
> > > >> >
> > > >> > wrote:
> > > >> >
> > > >> >> Indeed What an oversight :). Thanks for +1 - even non-binding is
> > > super
> > > >> >> important :) Anyone else trying the backport packages is great -
> I
> > > >> think
> > > >> >> it's super important for the Airflow community and great step to
> > ease
> > > >> >> migration to 2.0 :D
> > > >> >>
> > > >> >> J.
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> > > >> >> <ba...@godatadriven.com.invalid> wrote:
> > > >> >>
> > > >> >>> Thanks for the compliment but I’m no PMC so here’s my
> non-binding
> > +1
> > > >> :-)
> > > >> >>>
> > > >> >>> Bas
> > > >> >>>
> > > >> >>> On 26 May 2020, at 22:43, Jarek Potiuk <
> Jarek.Potiuk@polidea.com
> > > >> <mailto:
> > > >> >>> Jarek.Potiuk@polidea.com>> wrote:
> > > >> >>>
> > > >> >>> Hey all,
> > > >> >>>
> > > >> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email
> is
> > > >> >>> calling a
> > > >> >>> vote on the release,
> > > >> >>> which will last for 72 hours - which means that it will end on
> Fri
> > > >> May 29
> > > >> >>> 22:42:38 CEST 2020.
> > > >> >>>
> > > >> >>> Consider this my (binding) +1.
> > > >> >>>
> > > >> >>> The changes vs. rc2 are explained at the end of the email.
> > > >> >>>
> > > >> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > > >> >>>
> > > >> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz*
> is
> > a
> > > >> >>> source
> > > >> >>> release that comes
> > > >> >>> with INSTALL instructions.
> > > >> >>>
> > > >> >>>
> > > *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
> > > >> >>> are
> > > >> >>> the binary
> > > >> >>> Python "sdist" release.
> > > >> >>>
> > > >> >>> Public keys are available at:
> > > >> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> > > >> >>>
> > > >> >>> Please vote accordingly:
> > > >> >>>
> > > >> >>> [ ] +1 approve
> > > >> >>> [ ] +0 no opinion
> > > >> >>> [ ] -1 disapprove with the reason
> > > >> >>>
> > > >> >>>
> > > >> >>> Only votes from PMC members are binding, but members of the
> > > community
> > > >> are
> > > >> >>> encouraged to test the release and vote with "(non-binding)".
> > > >> >>>
> > > >> >>> Please note that the version number excludes the 'rcX' string,
> so
> > > it's
> > > >> >>> now
> > > >> >>> simply 2020.5.20. This will allow us to rename the artifact
> > without
> > > >> >>> modifying
> > > >> >>> the artifact checksums when we actually release it.
> > > >> >>>
> > > >> >>> Each of the packages contains a detailed changelog. Here is the
> > list
> > > >> of
> > > >> >>> links to
> > > >> >>> the released packages and changelogs:
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
> > > >> >>>
> > > >> >>> J.
> > > >> >>>
> > > >> >>>
> > > >> >>> Appendix: The changes vs. RC2:
> > > >> >>>
> > > >> >>> * During testing we found a few (20 out of 537) inconsistencies
> in
> > > the
> > > >> >>> names of the classes vs. the agreed AIP-21 standard. We fixed
> them
> > > all
> > > >> >>> and
> > > >> >>> also introduced automated tests that will fail PR in case
> someone
> > > >> adds an
> > > >> >>> inconsistently named class.
> > > >> >>>
> > > >> >>> * Several packages failed to import cleanly in 1.10. That was a
> > > great
> > > >> >>> idea
> > > >> >>> from one of the PMCs (Bas Hanrenslak) to automatically import
> all
> > > the
> > > >> >>> classes from all the providers and see if they are importable in
> > > >> 1.10. We
> > > >> >>> identified a few operators in a few providers (papermill, odbc,
> > > hive,
> > > >> >>> and a
> > > >> >>> few others) that would not import properly on 1.10. This is all
> > now
> > > >> fixed
> > > >> >>> (except papermill - see below). The good thing is that we also
> > added
> > > >> >>> automated tests for this: so we will be testing the
> importability
> > of
> > > >> >>> backported packages with every PR. This means that the backport
> > > >> packages
> > > >> >>> will continue to be importable in the future and we check it
> with
> > > >> every
> > > >> >>> single PR!
> > > >> >>>
> > > >> >>> Of course, importability does not guarantee that the operators
> > will
> > > >> work
> > > >> >>> with !00% certainty - this can only be assured by automated
> system
> > > >> tests
> > > >> >>> -
> > > >> >>> this is something we will be working on the coming months.
> > > >> >>>
> > > >> >>> * Papermill operator has proven to be problematic. It uses the
> > > Airflow
> > > >> >>> 2.0
> > > >> >>> lineage functionality that is not compatible with 1.10. That's
> why
> > > we
> > > >> >>> decided to remove papermill from the released backport packages
> > and
> > > >> >>> remove
> > > >> >>> the previous RCs for it.
> > > >> >>>
> > > >> >>> * The most important reason why we needed RC3. We noticed that
> > while
> > > >> the
> > > >> >>> README package instructions were all fine, the CHANGELOG and
> > INSTALL
> > > >> in
> > > >> >>> source packages contained changelog/Install instructions for
> > Airflow
> > > >> 2.0.
> > > >> >>> We fixed that and the INSTALL and CHANGELOG in the packages
> > contain
> > > >> >>> instructions/changelog for the backport packages only, not for
> the
> > > >> whole
> > > >> >>> Airflow.
> > > >> >>>
> > > >> >>> Cheers,
> > > >> >>>
> > > >> >>> Jarek
> > > >> >>>
> > > >> >>> --
> > > >> >>>
> > > >> >>> Jarek Potiuk
> > > >> >>> Polidea <https://www.polidea.com/> | Principal Software
> Engineer
> > > >> >>>
> > > >> >>> M: +48 660 796 129 <+48660796129>
> > > >> >>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>
> > > >> >>>
> > > >> >>
> > > >> >> --
> > > >> >>
> > > >> >> Jarek Potiuk
> > > >> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > >> >>
> > > >> >> M: +48 660 796 129 <+48660796129>
> > > >> >> [image: Polidea] <https://www.polidea.com/>
> > > >> >>
> > > >> >>
> > > >> >
> > > >> > --
> > > >> >
> > > >> > Jarek Potiuk
> > > >> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > >> >
> > > >> > M: +48 660 796 129 <+48660796129>
> > > >> > [image: Polidea] <https://www.polidea.com/>
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > >
> > > > Jarek Potiuk
> > > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > >
> > > > M: +48 660 796 129 <+48660796129>
> > > > [image: Polidea] <https://www.polidea.com/>
> > > >
> > > >
> > >
> > > --
> > >
> > > Jarek Potiuk
> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >
> > > M: +48 660 796 129 <+48660796129>
> > > [image: Polidea] <https://www.polidea.com/>
> > >
> >
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

Absolutely. This discussion made me think we want to work on RC4 :).  I
think that's better to clarify it now since this is something that stay
with us for quite some time. I will send a separate vote shortly.

J.

On Fri, May 29, 2020 at 1:11 PM Kaxil Naik <ka...@gmail.com> wrote:

> We might want to create a separate DISCUSS / VOTING Thread for
> https://apache-airflow.slack.com/archives/CCPRP7943/p1590742848402600
>
> i.e. Naming convention for the AToB Operators or Transfer Operators
>
> I think we should wait for the results of that before we release the
> Backport Packages i.e. create RC4 :(
>
> What do you think?
>
> Regards,
> Kaxil
>
>
>
> On Fri, May 29, 2020 at 11:34 AM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > Fixed :). Shift + Refresh
> >
> > On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> > > Sha of sha .... Wow.. That's something  like master level of security
> :).
> > >
> > > On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com>
> wrote:
> > >
> > >> I think we have some extra files at
> > >>
> > >>
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > >> (maybe
> > >> erroneously generated by the Bash scripts)
> > >>
> > >> Examples:
> > >>
> > >>    -
> > >>
> > >>
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > >>    <
> > >>
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> > >> >
> > >>    -
> > >>
> > >>
> >
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > >>    <
> > >>
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > >> >
> > >>    -
> > >>
> > >>
> >
> apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > >>    <
> > >>
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> > >> >
> > >>
> > >>
> > >> Can we remove those first, please, they are not a blocker to the
> release
> > >> but makes it difficult to view the actual packages?
> > >>
> > >> Regards,
> > >> Kaxil
> > >>
> > >> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > >> wrote:
> > >>
> > >> > Looking forward to some PMC votes :) But happy to hear more feedback
> > >> from
> > >> > others!
> > >> >
> > >> > Just a kind reminder (not sure if visible to everyone but let me
> try):
> > >> >
> > >> > [image: majestic_rabbit.jpg]
> > >> >
> > >> >  Fri May 29 22:42:38 CEST 2020.
> > >> >
> > >> > J.
> > >> >
> > >> >
> > >> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <
> > Jarek.Potiuk@polidea.com
> > >> >
> > >> > wrote:
> > >> >
> > >> >> Indeed What an oversight :). Thanks for +1 - even non-binding is
> > super
> > >> >> important :) Anyone else trying the backport packages is great - I
> > >> think
> > >> >> it's super important for the Airflow community and great step to
> ease
> > >> >> migration to 2.0 :D
> > >> >>
> > >> >> J.
> > >> >>
> > >> >>
> > >> >>
> > >> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> > >> >> <ba...@godatadriven.com.invalid> wrote:
> > >> >>
> > >> >>> Thanks for the compliment but I’m no PMC so here’s my non-binding
> +1
> > >> :-)
> > >> >>>
> > >> >>> Bas
> > >> >>>
> > >> >>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com
> > >> <mailto:
> > >> >>> Jarek.Potiuk@polidea.com>> wrote:
> > >> >>>
> > >> >>> Hey all,
> > >> >>>
> > >> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is
> > >> >>> calling a
> > >> >>> vote on the release,
> > >> >>> which will last for 72 hours - which means that it will end on Fri
> > >> May 29
> > >> >>> 22:42:38 CEST 2020.
> > >> >>>
> > >> >>> Consider this my (binding) +1.
> > >> >>>
> > >> >>> The changes vs. rc2 are explained at the end of the email.
> > >> >>>
> > >> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
> > >> >>>
> > >> >>>
> > >>
> >
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> > >> >>>
> > >> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is
> a
> > >> >>> source
> > >> >>> release that comes
> > >> >>> with INSTALL instructions.
> > >> >>>
> > >> >>>
> > *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
> > >> >>> are
> > >> >>> the binary
> > >> >>> Python "sdist" release.
> > >> >>>
> > >> >>> Public keys are available at:
> > >> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> > >> >>>
> > >> >>> Please vote accordingly:
> > >> >>>
> > >> >>> [ ] +1 approve
> > >> >>> [ ] +0 no opinion
> > >> >>> [ ] -1 disapprove with the reason
> > >> >>>
> > >> >>>
> > >> >>> Only votes from PMC members are binding, but members of the
> > community
> > >> are
> > >> >>> encouraged to test the release and vote with "(non-binding)".
> > >> >>>
> > >> >>> Please note that the version number excludes the 'rcX' string, so
> > it's
> > >> >>> now
> > >> >>> simply 2020.5.20. This will allow us to rename the artifact
> without
> > >> >>> modifying
> > >> >>> the artifact checksums when we actually release it.
> > >> >>>
> > >> >>> Each of the packages contains a detailed changelog. Here is the
> list
> > >> of
> > >> >>> links to
> > >> >>> the released packages and changelogs:
> > >> >>>
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
> > >> >>>
> > >> >>>
> > >>
> >
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
> > >> >>>
> > >> >>> J.
> > >> >>>
> > >> >>>
> > >> >>> Appendix: The changes vs. RC2:
> > >> >>>
> > >> >>> * During testing we found a few (20 out of 537) inconsistencies in
> > the
> > >> >>> names of the classes vs. the agreed AIP-21 standard. We fixed them
> > all
> > >> >>> and
> > >> >>> also introduced automated tests that will fail PR in case someone
> > >> adds an
> > >> >>> inconsistently named class.
> > >> >>>
> > >> >>> * Several packages failed to import cleanly in 1.10. That was a
> > great
> > >> >>> idea
> > >> >>> from one of the PMCs (Bas Hanrenslak) to automatically import all
> > the
> > >> >>> classes from all the providers and see if they are importable in
> > >> 1.10. We
> > >> >>> identified a few operators in a few providers (papermill, odbc,
> > hive,
> > >> >>> and a
> > >> >>> few others) that would not import properly on 1.10. This is all
> now
> > >> fixed
> > >> >>> (except papermill - see below). The good thing is that we also
> added
> > >> >>> automated tests for this: so we will be testing the importability
> of
> > >> >>> backported packages with every PR. This means that the backport
> > >> packages
> > >> >>> will continue to be importable in the future and we check it with
> > >> every
> > >> >>> single PR!
> > >> >>>
> > >> >>> Of course, importability does not guarantee that the operators
> will
> > >> work
> > >> >>> with !00% certainty - this can only be assured by automated system
> > >> tests
> > >> >>> -
> > >> >>> this is something we will be working on the coming months.
> > >> >>>
> > >> >>> * Papermill operator has proven to be problematic. It uses the
> > Airflow
> > >> >>> 2.0
> > >> >>> lineage functionality that is not compatible with 1.10. That's why
> > we
> > >> >>> decided to remove papermill from the released backport packages
> and
> > >> >>> remove
> > >> >>> the previous RCs for it.
> > >> >>>
> > >> >>> * The most important reason why we needed RC3. We noticed that
> while
> > >> the
> > >> >>> README package instructions were all fine, the CHANGELOG and
> INSTALL
> > >> in
> > >> >>> source packages contained changelog/Install instructions for
> Airflow
> > >> 2.0.
> > >> >>> We fixed that and the INSTALL and CHANGELOG in the packages
> contain
> > >> >>> instructions/changelog for the backport packages only, not for the
> > >> whole
> > >> >>> Airflow.
> > >> >>>
> > >> >>> Cheers,
> > >> >>>
> > >> >>> Jarek
> > >> >>>
> > >> >>> --
> > >> >>>
> > >> >>> Jarek Potiuk
> > >> >>> Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >> >>>
> > >> >>> M: +48 660 796 129 <+48660796129>
> > >> >>> [image: Polidea] <https://www.polidea.com/>
> > >> >>>
> > >> >>>
> > >> >>
> > >> >> --
> > >> >>
> > >> >> Jarek Potiuk
> > >> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >> >>
> > >> >> M: +48 660 796 129 <+48660796129>
> > >> >> [image: Polidea] <https://www.polidea.com/>
> > >> >>
> > >> >>
> > >> >
> > >> > --
> > >> >
> > >> > Jarek Potiuk
> > >> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >> >
> > >> > M: +48 660 796 129 <+48660796129>
> > >> > [image: Polidea] <https://www.polidea.com/>
> > >> >
> > >> >
> > >>
> > >
> > >
> > > --
> > >
> > > Jarek Potiuk
> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >
> > > M: +48 660 796 129 <+48660796129>
> > > [image: Polidea] <https://www.polidea.com/>
> > >
> > >
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Kaxil Naik <ka...@gmail.com>.

We might want to create a separate DISCUSS / VOTING Thread for
https://apache-airflow.slack.com/archives/CCPRP7943/p1590742848402600

i.e. Naming convention for the AToB Operators or Transfer Operators

I think we should wait for the results of that before we release the
Backport Packages i.e. create RC4 :(

What do you think?

Regards,
Kaxil



On Fri, May 29, 2020 at 11:34 AM Jarek Potiuk <Ja...@polidea.com>
wrote:

> Fixed :). Shift + Refresh
>
> On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > Sha of sha .... Wow.. That's something  like master level of security :).
> >
> > On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com> wrote:
> >
> >> I think we have some extra files at
> >>
> >>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> >> (maybe
> >> erroneously generated by the Bash scripts)
> >>
> >> Examples:
> >>
> >>    -
> >>
> >>
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> >>    <
> >>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> >> >
> >>    -
> >>
> >>
> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >>    <
> >>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >> >
> >>    -
> >>
> >>
> apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >>    <
> >>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >> >
> >>
> >>
> >> Can we remove those first, please, they are not a blocker to the release
> >> but makes it difficult to view the actual packages?
> >>
> >> Regards,
> >> Kaxil
> >>
> >> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <Ja...@polidea.com>
> >> wrote:
> >>
> >> > Looking forward to some PMC votes :) But happy to hear more feedback
> >> from
> >> > others!
> >> >
> >> > Just a kind reminder (not sure if visible to everyone but let me try):
> >> >
> >> > [image: majestic_rabbit.jpg]
> >> >
> >> >  Fri May 29 22:42:38 CEST 2020.
> >> >
> >> > J.
> >> >
> >> >
> >> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <
> Jarek.Potiuk@polidea.com
> >> >
> >> > wrote:
> >> >
> >> >> Indeed What an oversight :). Thanks for +1 - even non-binding is
> super
> >> >> important :) Anyone else trying the backport packages is great - I
> >> think
> >> >> it's super important for the Airflow community and great step to ease
> >> >> migration to 2.0 :D
> >> >>
> >> >> J.
> >> >>
> >> >>
> >> >>
> >> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> >> >> <ba...@godatadriven.com.invalid> wrote:
> >> >>
> >> >>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1
> >> :-)
> >> >>>
> >> >>> Bas
> >> >>>
> >> >>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com
> >> <mailto:
> >> >>> Jarek.Potiuk@polidea.com>> wrote:
> >> >>>
> >> >>> Hey all,
> >> >>>
> >> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is
> >> >>> calling a
> >> >>> vote on the release,
> >> >>> which will last for 72 hours - which means that it will end on Fri
> >> May 29
> >> >>> 22:42:38 CEST 2020.
> >> >>>
> >> >>> Consider this my (binding) +1.
> >> >>>
> >> >>> The changes vs. rc2 are explained at the end of the email.
> >> >>>
> >> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
> >> >>>
> >> >>>
> >>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> >> >>>
> >> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a
> >> >>> source
> >> >>> release that comes
> >> >>> with INSTALL instructions.
> >> >>>
> >> >>>
> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
> >> >>> are
> >> >>> the binary
> >> >>> Python "sdist" release.
> >> >>>
> >> >>> Public keys are available at:
> >> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> >> >>>
> >> >>> Please vote accordingly:
> >> >>>
> >> >>> [ ] +1 approve
> >> >>> [ ] +0 no opinion
> >> >>> [ ] -1 disapprove with the reason
> >> >>>
> >> >>>
> >> >>> Only votes from PMC members are binding, but members of the
> community
> >> are
> >> >>> encouraged to test the release and vote with "(non-binding)".
> >> >>>
> >> >>> Please note that the version number excludes the 'rcX' string, so
> it's
> >> >>> now
> >> >>> simply 2020.5.20. This will allow us to rename the artifact without
> >> >>> modifying
> >> >>> the artifact checksums when we actually release it.
> >> >>>
> >> >>> Each of the packages contains a detailed changelog. Here is the list
> >> of
> >> >>> links to
> >> >>> the released packages and changelogs:
> >> >>>
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
> >> >>>
> >> >>>
> >>
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
> >> >>>
> >> >>> J.
> >> >>>
> >> >>>
> >> >>> Appendix: The changes vs. RC2:
> >> >>>
> >> >>> * During testing we found a few (20 out of 537) inconsistencies in
> the
> >> >>> names of the classes vs. the agreed AIP-21 standard. We fixed them
> all
> >> >>> and
> >> >>> also introduced automated tests that will fail PR in case someone
> >> adds an
> >> >>> inconsistently named class.
> >> >>>
> >> >>> * Several packages failed to import cleanly in 1.10. That was a
> great
> >> >>> idea
> >> >>> from one of the PMCs (Bas Hanrenslak) to automatically import all
> the
> >> >>> classes from all the providers and see if they are importable in
> >> 1.10. We
> >> >>> identified a few operators in a few providers (papermill, odbc,
> hive,
> >> >>> and a
> >> >>> few others) that would not import properly on 1.10. This is all now
> >> fixed
> >> >>> (except papermill - see below). The good thing is that we also added
> >> >>> automated tests for this: so we will be testing the importability of
> >> >>> backported packages with every PR. This means that the backport
> >> packages
> >> >>> will continue to be importable in the future and we check it with
> >> every
> >> >>> single PR!
> >> >>>
> >> >>> Of course, importability does not guarantee that the operators will
> >> work
> >> >>> with !00% certainty - this can only be assured by automated system
> >> tests
> >> >>> -
> >> >>> this is something we will be working on the coming months.
> >> >>>
> >> >>> * Papermill operator has proven to be problematic. It uses the
> Airflow
> >> >>> 2.0
> >> >>> lineage functionality that is not compatible with 1.10. That's why
> we
> >> >>> decided to remove papermill from the released backport packages and
> >> >>> remove
> >> >>> the previous RCs for it.
> >> >>>
> >> >>> * The most important reason why we needed RC3. We noticed that while
> >> the
> >> >>> README package instructions were all fine, the CHANGELOG and INSTALL
> >> in
> >> >>> source packages contained changelog/Install instructions for Airflow
> >> 2.0.
> >> >>> We fixed that and the INSTALL and CHANGELOG in the packages contain
> >> >>> instructions/changelog for the backport packages only, not for the
> >> whole
> >> >>> Airflow.
> >> >>>
> >> >>> Cheers,
> >> >>>
> >> >>> Jarek
> >> >>>
> >> >>> --
> >> >>>
> >> >>> Jarek Potiuk
> >> >>> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >> >>>
> >> >>> M: +48 660 796 129 <+48660796129>
> >> >>> [image: Polidea] <https://www.polidea.com/>
> >> >>>
> >> >>>
> >> >>
> >> >> --
> >> >>
> >> >> Jarek Potiuk
> >> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >> >>
> >> >> M: +48 660 796 129 <+48660796129>
> >> >> [image: Polidea] <https://www.polidea.com/>
> >> >>
> >> >>
> >> >
> >> > --
> >> >
> >> > Jarek Potiuk
> >> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >> >
> >> > M: +48 660 796 129 <+48660796129>
> >> > [image: Polidea] <https://www.polidea.com/>
> >> >
> >> >
> >>
> >
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
> >
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

Fixed :). Shift + Refresh

On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <Ja...@polidea.com>
wrote:

> Sha of sha .... Wow.. That's something  like master level of security :).
>
> On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com> wrote:
>
>> I think we have some extra files at
>>
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
>> (maybe
>> erroneously generated by the Bash scripts)
>>
>> Examples:
>>
>>    -
>>
>>  apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
>>    <
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
>> >
>>    -
>>
>>  apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>>    <
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>> >
>>    -
>>
>>  apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>>    <
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>> >
>>
>>
>> Can we remove those first, please, they are not a blocker to the release
>> but makes it difficult to view the actual packages?
>>
>> Regards,
>> Kaxil
>>
>> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>> > Looking forward to some PMC votes :) But happy to hear more feedback
>> from
>> > others!
>> >
>> > Just a kind reminder (not sure if visible to everyone but let me try):
>> >
>> > [image: majestic_rabbit.jpg]
>> >
>> >  Fri May 29 22:42:38 CEST 2020.
>> >
>> > J.
>> >
>> >
>> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <Jarek.Potiuk@polidea.com
>> >
>> > wrote:
>> >
>> >> Indeed What an oversight :). Thanks for +1 - even non-binding is super
>> >> important :) Anyone else trying the backport packages is great - I
>> think
>> >> it's super important for the Airflow community and great step to ease
>> >> migration to 2.0 :D
>> >>
>> >> J.
>> >>
>> >>
>> >>
>> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
>> >> <ba...@godatadriven.com.invalid> wrote:
>> >>
>> >>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1
>> :-)
>> >>>
>> >>> Bas
>> >>>
>> >>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com
>> <mailto:
>> >>> Jarek.Potiuk@polidea.com>> wrote:
>> >>>
>> >>> Hey all,
>> >>>
>> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is
>> >>> calling a
>> >>> vote on the release,
>> >>> which will last for 72 hours - which means that it will end on Fri
>> May 29
>> >>> 22:42:38 CEST 2020.
>> >>>
>> >>> Consider this my (binding) +1.
>> >>>
>> >>> The changes vs. rc2 are explained at the end of the email.
>> >>>
>> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
>> >>>
>> >>>
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
>> >>>
>> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a
>> >>> source
>> >>> release that comes
>> >>> with INSTALL instructions.
>> >>>
>> >>> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
>> >>> are
>> >>> the binary
>> >>> Python "sdist" release.
>> >>>
>> >>> Public keys are available at:
>> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
>> >>>
>> >>> Please vote accordingly:
>> >>>
>> >>> [ ] +1 approve
>> >>> [ ] +0 no opinion
>> >>> [ ] -1 disapprove with the reason
>> >>>
>> >>>
>> >>> Only votes from PMC members are binding, but members of the community
>> are
>> >>> encouraged to test the release and vote with "(non-binding)".
>> >>>
>> >>> Please note that the version number excludes the 'rcX' string, so it's
>> >>> now
>> >>> simply 2020.5.20. This will allow us to rename the artifact without
>> >>> modifying
>> >>> the artifact checksums when we actually release it.
>> >>>
>> >>> Each of the packages contains a detailed changelog. Here is the list
>> of
>> >>> links to
>> >>> the released packages and changelogs:
>> >>>
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
>> >>>
>> >>>
>> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
>> >>>
>> >>> J.
>> >>>
>> >>>
>> >>> Appendix: The changes vs. RC2:
>> >>>
>> >>> * During testing we found a few (20 out of 537) inconsistencies in the
>> >>> names of the classes vs. the agreed AIP-21 standard. We fixed them all
>> >>> and
>> >>> also introduced automated tests that will fail PR in case someone
>> adds an
>> >>> inconsistently named class.
>> >>>
>> >>> * Several packages failed to import cleanly in 1.10. That was a great
>> >>> idea
>> >>> from one of the PMCs (Bas Hanrenslak) to automatically import all the
>> >>> classes from all the providers and see if they are importable in
>> 1.10. We
>> >>> identified a few operators in a few providers (papermill, odbc, hive,
>> >>> and a
>> >>> few others) that would not import properly on 1.10. This is all now
>> fixed
>> >>> (except papermill - see below). The good thing is that we also added
>> >>> automated tests for this: so we will be testing the importability of
>> >>> backported packages with every PR. This means that the backport
>> packages
>> >>> will continue to be importable in the future and we check it with
>> every
>> >>> single PR!
>> >>>
>> >>> Of course, importability does not guarantee that the operators will
>> work
>> >>> with !00% certainty - this can only be assured by automated system
>> tests
>> >>> -
>> >>> this is something we will be working on the coming months.
>> >>>
>> >>> * Papermill operator has proven to be problematic. It uses the Airflow
>> >>> 2.0
>> >>> lineage functionality that is not compatible with 1.10. That's why we
>> >>> decided to remove papermill from the released backport packages and
>> >>> remove
>> >>> the previous RCs for it.
>> >>>
>> >>> * The most important reason why we needed RC3. We noticed that while
>> the
>> >>> README package instructions were all fine, the CHANGELOG and INSTALL
>> in
>> >>> source packages contained changelog/Install instructions for Airflow
>> 2.0.
>> >>> We fixed that and the INSTALL and CHANGELOG in the packages contain
>> >>> instructions/changelog for the backport packages only, not for the
>> whole
>> >>> Airflow.
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Jarek
>> >>>
>> >>> --
>> >>>
>> >>> Jarek Potiuk
>> >>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>> >>>
>> >>> M: +48 660 796 129 <+48660796129>
>> >>> [image: Polidea] <https://www.polidea.com/>
>> >>>
>> >>>
>> >>
>> >> --
>> >>
>> >> Jarek Potiuk
>> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
>> >>
>> >> M: +48 660 796 129 <+48660796129>
>> >> [image: Polidea] <https://www.polidea.com/>
>> >>
>> >>
>> >
>> > --
>> >
>> > Jarek Potiuk
>> > Polidea <https://www.polidea.com/> | Principal Software Engineer
>> >
>> > M: +48 660 796 129 <+48660796129>
>> > [image: Polidea] <https://www.polidea.com/>
>> >
>> >
>>
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>

-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

Sha of sha .... Wow.. That's something  like master level of security :).

On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <ka...@gmail.com> wrote:

> I think we have some extra files at
>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> (maybe
> erroneously generated by the Bash scripts)
>
> Examples:
>
>    -
>
>  apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
>    <
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
> >
>    -
>
>  apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>    <
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >
>    -
>
>  apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
>    <
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
> >
>
>
> Can we remove those first, please, they are not a blocker to the release
> but makes it difficult to view the actual packages?
>
> Regards,
> Kaxil
>
> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > Looking forward to some PMC votes :) But happy to hear more feedback from
> > others!
> >
> > Just a kind reminder (not sure if visible to everyone but let me try):
> >
> > [image: majestic_rabbit.jpg]
> >
> >  Fri May 29 22:42:38 CEST 2020.
> >
> > J.
> >
> >
> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> >> Indeed What an oversight :). Thanks for +1 - even non-binding is super
> >> important :) Anyone else trying the backport packages is great - I think
> >> it's super important for the Airflow community and great step to ease
> >> migration to 2.0 :D
> >>
> >> J.
> >>
> >>
> >>
> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> >> <ba...@godatadriven.com.invalid> wrote:
> >>
> >>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1
> :-)
> >>>
> >>> Bas
> >>>
> >>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com
> <mailto:
> >>> Jarek.Potiuk@polidea.com>> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is
> >>> calling a
> >>> vote on the release,
> >>> which will last for 72 hours - which means that it will end on Fri May
> 29
> >>> 22:42:38 CEST 2020.
> >>>
> >>> Consider this my (binding) +1.
> >>>
> >>> The changes vs. rc2 are explained at the end of the email.
> >>>
> >>> Airflow Backport Providers 2020.5.20rc3 are available at:
> >>>
> >>>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
> >>>
> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a
> >>> source
> >>> release that comes
> >>> with INSTALL instructions.
> >>>
> >>> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
> >>> are
> >>> the binary
> >>> Python "sdist" release.
> >>>
> >>> Public keys are available at:
> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS
> >>>
> >>> Please vote accordingly:
> >>>
> >>> [ ] +1 approve
> >>> [ ] +0 no opinion
> >>> [ ] -1 disapprove with the reason
> >>>
> >>>
> >>> Only votes from PMC members are binding, but members of the community
> are
> >>> encouraged to test the release and vote with "(non-binding)".
> >>>
> >>> Please note that the version number excludes the 'rcX' string, so it's
> >>> now
> >>> simply 2020.5.20. This will allow us to rename the artifact without
> >>> modifying
> >>> the artifact checksums when we actually release it.
> >>>
> >>> Each of the packages contains a detailed changelog. Here is the list of
> >>> links to
> >>> the released packages and changelogs:
> >>>
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
> >>>
> >>>
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
> >>>
> >>> J.
> >>>
> >>>
> >>> Appendix: The changes vs. RC2:
> >>>
> >>> * During testing we found a few (20 out of 537) inconsistencies in the
> >>> names of the classes vs. the agreed AIP-21 standard. We fixed them all
> >>> and
> >>> also introduced automated tests that will fail PR in case someone adds
> an
> >>> inconsistently named class.
> >>>
> >>> * Several packages failed to import cleanly in 1.10. That was a great
> >>> idea
> >>> from one of the PMCs (Bas Hanrenslak) to automatically import all the
> >>> classes from all the providers and see if they are importable in 1.10.
> We
> >>> identified a few operators in a few providers (papermill, odbc, hive,
> >>> and a
> >>> few others) that would not import properly on 1.10. This is all now
> fixed
> >>> (except papermill - see below). The good thing is that we also added
> >>> automated tests for this: so we will be testing the importability of
> >>> backported packages with every PR. This means that the backport
> packages
> >>> will continue to be importable in the future and we check it with every
> >>> single PR!
> >>>
> >>> Of course, importability does not guarantee that the operators will
> work
> >>> with !00% certainty - this can only be assured by automated system
> tests
> >>> -
> >>> this is something we will be working on the coming months.
> >>>
> >>> * Papermill operator has proven to be problematic. It uses the Airflow
> >>> 2.0
> >>> lineage functionality that is not compatible with 1.10. That's why we
> >>> decided to remove papermill from the released backport packages and
> >>> remove
> >>> the previous RCs for it.
> >>>
> >>> * The most important reason why we needed RC3. We noticed that while
> the
> >>> README package instructions were all fine, the CHANGELOG and INSTALL in
> >>> source packages contained changelog/Install instructions for Airflow
> 2.0.
> >>> We fixed that and the INSTALL and CHANGELOG in the packages contain
> >>> instructions/changelog for the backport packages only, not for the
> whole
> >>> Airflow.
> >>>
> >>> Cheers,
> >>>
> >>> Jarek
> >>>
> >>> --
> >>>
> >>> Jarek Potiuk
> >>> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>>
> >>> M: +48 660 796 129 <+48660796129>
> >>> [image: Polidea] <https://www.polidea.com/>
> >>>
> >>>
> >>
> >> --
> >>
> >> Jarek Potiuk
> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>
> >> M: +48 660 796 129 <+48660796129>
> >> [image: Polidea] <https://www.polidea.com/>
> >>
> >>
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
> >
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Kaxil Naik <ka...@gmail.com>.

I think we have some extra files at
https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
(maybe
erroneously generated by the Bash scripts)

Examples:

   -
   apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512
   <https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512>
   -
   apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
   <https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512>
   -
   apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512
   <https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512>


Can we remove those first, please, they are not a blocker to the release
but makes it difficult to view the actual packages?

Regards,
Kaxil

On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <Ja...@polidea.com>
wrote:

> Looking forward to some PMC votes :) But happy to hear more feedback from
> others!
>
> Just a kind reminder (not sure if visible to everyone but let me try):
>
> [image: majestic_rabbit.jpg]
>
>  Fri May 29 22:42:38 CEST 2020.
>
> J.
>
>
> On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Indeed What an oversight :). Thanks for +1 - even non-binding is super
>> important :) Anyone else trying the backport packages is great - I think
>> it's super important for the Airflow community and great step to ease
>> migration to 2.0 :D
>>
>> J.
>>
>>
>>
>> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
>> <ba...@godatadriven.com.invalid> wrote:
>>
>>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1 :-)
>>>
>>> Bas
>>>
>>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com<mailto:
>>> Jarek.Potiuk@polidea.com>> wrote:
>>>
>>> Hey all,
>>>
>>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is
>>> calling a
>>> vote on the release,
>>> which will last for 72 hours - which means that it will end on Fri May 29
>>> 22:42:38 CEST 2020.
>>>
>>> Consider this my (binding) +1.
>>>
>>> The changes vs. rc2 are explained at the end of the email.
>>>
>>> Airflow Backport Providers 2020.5.20rc3 are available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
>>>
>>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a
>>> source
>>> release that comes
>>> with INSTALL instructions.
>>>
>>> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz*
>>> are
>>> the binary
>>> Python "sdist" release.
>>>
>>> Public keys are available at:
>>> https://dist.apache.org/repos/dist/release/airflow/KEYS
>>>
>>> Please vote accordingly:
>>>
>>> [ ] +1 approve
>>> [ ] +0 no opinion
>>> [ ] -1 disapprove with the reason
>>>
>>>
>>> Only votes from PMC members are binding, but members of the community are
>>> encouraged to test the release and vote with "(non-binding)".
>>>
>>> Please note that the version number excludes the 'rcX' string, so it's
>>> now
>>> simply 2020.5.20. This will allow us to rename the artifact without
>>> modifying
>>> the artifact checksums when we actually release it.
>>>
>>> Each of the packages contains a detailed changelog. Here is the list of
>>> links to
>>> the released packages and changelogs:
>>>
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
>>>
>>> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
>>>
>>> J.
>>>
>>>
>>> Appendix: The changes vs. RC2:
>>>
>>> * During testing we found a few (20 out of 537) inconsistencies in the
>>> names of the classes vs. the agreed AIP-21 standard. We fixed them all
>>> and
>>> also introduced automated tests that will fail PR in case someone adds an
>>> inconsistently named class.
>>>
>>> * Several packages failed to import cleanly in 1.10. That was a great
>>> idea
>>> from one of the PMCs (Bas Hanrenslak) to automatically import all the
>>> classes from all the providers and see if they are importable in 1.10. We
>>> identified a few operators in a few providers (papermill, odbc, hive,
>>> and a
>>> few others) that would not import properly on 1.10. This is all now fixed
>>> (except papermill - see below). The good thing is that we also added
>>> automated tests for this: so we will be testing the importability of
>>> backported packages with every PR. This means that the backport packages
>>> will continue to be importable in the future and we check it with every
>>> single PR!
>>>
>>> Of course, importability does not guarantee that the operators will work
>>> with !00% certainty - this can only be assured by automated system tests
>>> -
>>> this is something we will be working on the coming months.
>>>
>>> * Papermill operator has proven to be problematic. It uses the Airflow
>>> 2.0
>>> lineage functionality that is not compatible with 1.10. That's why we
>>> decided to remove papermill from the released backport packages and
>>> remove
>>> the previous RCs for it.
>>>
>>> * The most important reason why we needed RC3. We noticed that while the
>>> README package instructions were all fine, the CHANGELOG and INSTALL in
>>> source packages contained changelog/Install instructions for Airflow 2.0.
>>> We fixed that and the INSTALL and CHANGELOG in the packages contain
>>> instructions/changelog for the backport packages only, not for the whole
>>> Airflow.
>>>
>>> Cheers,
>>>
>>> Jarek
>>>
>>> --
>>>
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>
>>> M: +48 660 796 129 <+48660796129>
>>> [image: Polidea] <https://www.polidea.com/>
>>>
>>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

Looking forward to some PMC votes :) But happy to hear more feedback from
others!

Just a kind reminder (not sure if visible to everyone but let me try):

[image: majestic_rabbit.jpg]

 Fri May 29 22:42:38 CEST 2020.

J.


On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <Ja...@polidea.com>
wrote:

> Indeed What an oversight :). Thanks for +1 - even non-binding is super
> important :) Anyone else trying the backport packages is great - I think
> it's super important for the Airflow community and great step to ease
> migration to 2.0 :D
>
> J.
>
>
>
> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
> <ba...@godatadriven.com.invalid> wrote:
>
>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1 :-)
>>
>> Bas
>>
>> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com<mailto:
>> Jarek.Potiuk@polidea.com>> wrote:
>>
>> Hey all,
>>
>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is calling
>> a
>> vote on the release,
>> which will last for 72 hours - which means that it will end on Fri May 29
>> 22:42:38 CEST 2020.
>>
>> Consider this my (binding) +1.
>>
>> The changes vs. rc2 are explained at the end of the email.
>>
>> Airflow Backport Providers 2020.5.20rc3 are available at:
>>
>> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
>>
>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a source
>> release that comes
>> with INSTALL instructions.
>>
>> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz* are
>> the binary
>> Python "sdist" release.
>>
>> Public keys are available at:
>> https://dist.apache.org/repos/dist/release/airflow/KEYS
>>
>> Please vote accordingly:
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove with the reason
>>
>>
>> Only votes from PMC members are binding, but members of the community are
>> encouraged to test the release and vote with "(non-binding)".
>>
>> Please note that the version number excludes the 'rcX' string, so it's now
>> simply 2020.5.20. This will allow us to rename the artifact without
>> modifying
>> the artifact checksums when we actually release it.
>>
>> Each of the packages contains a detailed changelog. Here is the list of
>> links to
>> the released packages and changelogs:
>>
>>
>> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
>>
>> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
>>
>> J.
>>
>>
>> Appendix: The changes vs. RC2:
>>
>> * During testing we found a few (20 out of 537) inconsistencies in the
>> names of the classes vs. the agreed AIP-21 standard. We fixed them all and
>> also introduced automated tests that will fail PR in case someone adds an
>> inconsistently named class.
>>
>> * Several packages failed to import cleanly in 1.10. That was a great idea
>> from one of the PMCs (Bas Hanrenslak) to automatically import all the
>> classes from all the providers and see if they are importable in 1.10. We
>> identified a few operators in a few providers (papermill, odbc, hive, and
>> a
>> few others) that would not import properly on 1.10. This is all now fixed
>> (except papermill - see below). The good thing is that we also added
>> automated tests for this: so we will be testing the importability of
>> backported packages with every PR. This means that the backport packages
>> will continue to be importable in the future and we check it with every
>> single PR!
>>
>> Of course, importability does not guarantee that the operators will work
>> with !00% certainty - this can only be assured by automated system tests -
>> this is something we will be working on the coming months.
>>
>> * Papermill operator has proven to be problematic. It uses the Airflow 2.0
>> lineage functionality that is not compatible with 1.10. That's why we
>> decided to remove papermill from the released backport packages and remove
>> the previous RCs for it.
>>
>> * The most important reason why we needed RC3. We noticed that while the
>> README package instructions were all fine, the CHANGELOG and INSTALL in
>> source packages contained changelog/Install instructions for Airflow 2.0.
>> We fixed that and the INSTALL and CHANGELOG in the packages contain
>> instructions/changelog for the backport packages only, not for the whole
>> Airflow.
>>
>> Cheers,
>>
>> Jarek
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>

-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Jarek Potiuk <Ja...@polidea.com>.

Indeed What an oversight :). Thanks for +1 - even non-binding is super
important :) Anyone else trying the backport packages is great - I think
it's super important for the Airflow community and great step to ease
migration to 2.0 :D

J.



On Tue, May 26, 2020 at 11:12 PM Bas Harenslak
<ba...@godatadriven.com.invalid> wrote:

> Thanks for the compliment but I’m no PMC so here’s my non-binding +1 :-)
>
> Bas
>
> On 26 May 2020, at 22:43, Jarek Potiuk <Jarek.Potiuk@polidea.com<mailto:
> Jarek.Potiuk@polidea.com>> wrote:
>
> Hey all,
>
> I have cut Airflow Backport Providers 2020.5.20rc3. This email is calling a
> vote on the release,
> which will last for 72 hours - which means that it will end on Fri May 29
> 22:42:38 CEST 2020.
>
> Consider this my (binding) +1.
>
> The changes vs. rc2 are explained at the end of the email.
>
> Airflow Backport Providers 2020.5.20rc3 are available at:
>
> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/
>
> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a source
> release that comes
> with INSTALL instructions.
>
> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz* are
> the binary
> Python "sdist" release.
>
> Public keys are available at:
> https://dist.apache.org/repos/dist/release/airflow/KEYS
>
> Please vote accordingly:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
>
> Only votes from PMC members are binding, but members of the community are
> encouraged to test the release and vote with "(non-binding)".
>
> Please note that the version number excludes the 'rcX' string, so it's now
> simply 2020.5.20. This will allow us to rename the artifact without
> modifying
> the artifact checksums when we actually release it.
>
> Each of the packages contains a detailed changelog. Here is the list of
> links to
> the released packages and changelogs:
>
>
> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
>
> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/
>
> J.
>
>
> Appendix: The changes vs. RC2:
>
> * During testing we found a few (20 out of 537) inconsistencies in the
> names of the classes vs. the agreed AIP-21 standard. We fixed them all and
> also introduced automated tests that will fail PR in case someone adds an
> inconsistently named class.
>
> * Several packages failed to import cleanly in 1.10. That was a great idea
> from one of the PMCs (Bas Hanrenslak) to automatically import all the
> classes from all the providers and see if they are importable in 1.10. We
> identified a few operators in a few providers (papermill, odbc, hive, and a
> few others) that would not import properly on 1.10. This is all now fixed
> (except papermill - see below). The good thing is that we also added
> automated tests for this: so we will be testing the importability of
> backported packages with every PR. This means that the backport packages
> will continue to be importable in the future and we check it with every
> single PR!
>
> Of course, importability does not guarantee that the operators will work
> with !00% certainty - this can only be assured by automated system tests -
> this is something we will be working on the coming months.
>
> * Papermill operator has proven to be problematic. It uses the Airflow 2.0
> lineage functionality that is not compatible with 1.10. That's why we
> decided to remove papermill from the released backport packages and remove
> the previous RCs for it.
>
> * The most important reason why we needed RC3. We noticed that while the
> README package instructions were all fine, the CHANGELOG and INSTALL in
> source packages contained changelog/Install instructions for Airflow 2.0.
> We fixed that and the INSTALL and CHANGELOG in the packages contain
> instructions/changelog for the backport packages only, not for the whole
> Airflow.
>
> Cheers,
>
> Jarek
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>

-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Re: [VOTE] Airflow Backport Providers 2020.5.20rc3

Posted by Bas Harenslak <ba...@godatadriven.com.INVALID>.

Thanks for the compliment but I’m no PMC so here’s my non-binding +1 :-)

Bas

On 26 May 2020, at 22:43, Jarek Potiuk <Ja...@polidea.com>> wrote:

Hey all,

I have cut Airflow Backport Providers 2020.5.20rc3. This email is calling a
vote on the release,
which will last for 72 hours - which means that it will end on Fri May 29
22:42:38 CEST 2020.

Consider this my (binding) +1.

The changes vs. rc2 are explained at the end of the email.

Airflow Backport Providers 2020.5.20rc3 are available at:
https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/

*apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a source
release that comes
with INSTALL instructions.

*apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz* are
the binary
Python "sdist" release.

Public keys are available at:
https://dist.apache.org/repos/dist/release/airflow/KEYS

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason


Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Please note that the version number excludes the 'rcX' string, so it's now
simply 2020.5.20. This will allow us to rename the artifact without
modifying
the artifact checksums when we actually release it.

Each of the packages contains a detailed changelog. Here is the list of
links to
the released packages and changelogs:

https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/
https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/

J.


Appendix: The changes vs. RC2:

* During testing we found a few (20 out of 537) inconsistencies in the
names of the classes vs. the agreed AIP-21 standard. We fixed them all and
also introduced automated tests that will fail PR in case someone adds an
inconsistently named class.

* Several packages failed to import cleanly in 1.10. That was a great idea
from one of the PMCs (Bas Hanrenslak) to automatically import all the
classes from all the providers and see if they are importable in 1.10. We
identified a few operators in a few providers (papermill, odbc, hive, and a
few others) that would not import properly on 1.10. This is all now fixed
(except papermill - see below). The good thing is that we also added
automated tests for this: so we will be testing the importability of
backported packages with every PR. This means that the backport packages
will continue to be importable in the future and we check it with every
single PR!

Of course, importability does not guarantee that the operators will work
with !00% certainty - this can only be assured by automated system tests -
this is something we will be working on the coming months.

* Papermill operator has proven to be problematic. It uses the Airflow 2.0
lineage functionality that is not compatible with 1.10. That's why we
decided to remove papermill from the released backport packages and remove
the previous RCs for it.

* The most important reason why we needed RC3. We noticed that while the
README package instructions were all fine, the CHANGELOG and INSTALL in
source packages contained changelog/Install instructions for Airflow 2.0.
We fixed that and the INSTALL and CHANGELOG in the packages contain
instructions/changelog for the backport packages only, not for the whole
Airflow.

Cheers,

Jarek

--

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>