You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/13 09:49:24 UTC
[GitHub] [airflow] owlphi opened a new issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
owlphi opened a new issue #12335:
URL: https://github.com/apache/airflow/issues/12335
**Apache Airflow version**: 2.0.0b1
### Issue The google cloud storage hook is not backward compatible with the version 1.12.
In the version 1.12 the method download has the following parameters: (bucket, object, filename) in this order. See [here](
https://airflow.apache.org/docs/stable/_api/airflow/contrib/hooks/gcs_hook/index.html#airflow.contrib.hooks.gcs_hook.GoogleCloudStorageHook.download)
In the version 2.0.0b1 the method download has the following parameters (object_name, bucket_name, filename) in this order. See [here](https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/hooks/gcs.py#L259-L260)
You can see that the order of the parameters have changed from version 1.12 to 2.0.0b1.
**The problem:**
If we use the method download with positional parameters like in this operator: https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267. It will fail
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek edited a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek edited a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726677159
Yes, it's a breaking change as described here:
https://github.com/apache/airflow/blob/master/UPDATING.md#airflowprovidersgooglecloudhooksgcsgcshook
Airflow 2.0 as a main version is allowed to have breaking changes (for example new cli commands) although we tried to limit breaking changes to minimium,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek closed issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek closed issue #12335:
URL: https://github.com/apache/airflow/issues/12335
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] owlphi edited a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
owlphi edited a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726698305
@turbaszek . Thank you for the quick reply. In that case this line of the code
https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267
is not working because the order of parameters is wrong. This is what I tried to point it out.
Should I create a new issue? or could we have this open with a better title of the issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726677159
Yes, it's a breaking change as described here:
https://github.com/apache/airflow/blob/master/UPDATING.md#airflowprovidersgooglecloudhooksgcsgcshook
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] owlphi edited a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
owlphi edited a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726698305
@turbaszek . Thank you for the quick reply. In that case this line of the code
https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267
is not working because the order of parameters is wrong. This is what I tried to point it out.
Should I create a new issue? or could we have this open?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] owlphi commented on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
owlphi commented on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726698305
@turbaszek . Thank you for the quick reply. In that case this line of the code
https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267
is not working because the order of parameters is wrong. This is what I have tried to point it out.
Should I create a new issue? or could we have this open?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Zedmor removed a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
Zedmor removed a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726877266
Addresses in https://github.com/apache/airflow/pull/12305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726664147
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] owlphi edited a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
owlphi edited a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726698305
@turbaszek . Thank you for the quick reply. In that case this line of the code
https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267
is not working because the order of parameters is wrong. This is what I tried to point it out.
Should I create a new issue? or could we have this open with a better issue's tittle?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] owlphi edited a comment on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
owlphi edited a comment on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726698305
@turbaszek . Thank you for the quick reply. In that case this line of the code
https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py#L267
is not working because the order of parameters is wrong. This is what I tried to point it out.
Should I create a new issue? or could we have this open with a better issues' tittle?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Zedmor commented on issue #12335: The signature for the gcs hooks is different in apache 1.12 compare with the beta version 2.0
Posted by GitBox <gi...@apache.org>.
Zedmor commented on issue #12335:
URL: https://github.com/apache/airflow/issues/12335#issuecomment-726877266
Addresses in https://github.com/apache/airflow/pull/12305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org