You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/04 16:22:01 UTC
[GitHub] [airflow] sarahamilton opened a new issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
sarahamilton opened a new issue #14075:
URL: https://github.com/apache/airflow/issues/14075
**Apache Airflow version**: 1.10.12
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**: Google Cloud Composer 1.13.3
- **Cloud provider or hardware configuration**:
- Google Cloud Composer
**What happened**:
The GoogleDisplayVideo360SDFtoGCSOperator is not able to download media. The operator calls the [download_media hook](https://github.com/apache/airflow/blob/10c026cb7a7189d9573f30f2f2242f0f76842a72/airflow/providers/google/marketing_platform/hooks/display_video.py#L237) and should pass in the name of the media resource to download. However, it is currently passing in the full resource object. This breaks the API call to download_media. The [API spec is here](https://developers.google.com/display-video/api/reference/rest/v1/media/download), for reference. Thus, the operator will fail every time.
An example error -- as you can see, it's requesting the full object in the path instead of just "sdfdownloadtasks/media/25447314'":
`ERROR - <HttpError 404 when requesting https://displayvideo.googleapis.com/download/%7B'name':%20'sdfdownloadtasks/operations/25447314',%20'metadata':%20%7B'@type':%20'type.googleapis.com/google.ads.displayvideo.v1.SdfDownloadTaskMetadata',%20'createTime':%20'2021-02-03T16:57:20.950Z',%20'endTime':%20'2021-02-03T16:57:52.898Z',%20'version':%20'SDF_VERSION_5_1'%7D,%20'done':%20True,%20'response':%20%7B'@type':%20'type.googleapis.com/google.ads.displayvideo.v1.SdfDownloadTask',%20'resourceName':%20'sdfdownloadtasks/media/25447314'%7D%7D?alt=media returned "Resource "{'name': 'sdfdownloadtasks/operations/25447314', 'metadata': {'@type': 'type.googleapis.com/google.ads.displayvideo.v1.SdfDownloadTaskMetadata', 'createTime': '2021-02-03T16:57:20.950Z', 'endTime': '2021-02-03T16:57:52.898Z', 'version': 'SDF_VERSION_5_1'}, 'done': True, 'response': {'@type': 'type.googleapis.com/google.ads.displayvideo.v1.SdfDownloadTask', 'resourceName': 'sdfdownloadtasks/media/25447314'}}" cannot be f
ound.">`
**What you expected to happen**: The GoogleDisplayVideo360SDFtoGCSOperator should only pass in the resourceName and correctly download the file.
**How to reproduce it**: Run any workflow requesting to download an SDF file.
**Anything else we need to know**: I have written a patch and will submit it shortly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-773440989
> Anything else we need to know: I have written a patch and will submit it shortly.
If you create pull-request, you don't have to create an issue first, but if you want, you can do it. Creating an issue will allow you to collect feedback or share plans with other people.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-1075586461
@pierrejeambrun since this is not working now you can break it any further :)
I'm not really sure if google offer free trial accounts? you can check it.
On the PR you can explain why the change is fixing the problem (reference to the API/underlying lib relevant source code to gain more confidence that the fix is right)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun commented on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
pierrejeambrun commented on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-1075579709
Hello @eladkal and @potiuk,
I think I can help on this one. (Looks like the closed PR was almost it).
I am not sure on how i'm going to test it thought as I need a `Google Display & Video 360` account and they do not seem to offer "free trial" stuff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #14075:
URL: https://github.com/apache/airflow/issues/14075
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-773440989
> Anything else we need to know: I have written a patch and will submit it shortly.
If you create pull-request, you don't have to create an issue first, but if you want, you can do it. Creating an issue will allow you to collect feedback or share plans with other people.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun edited a comment on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
pierrejeambrun edited a comment on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-1075579709
Hello @eladkal and @potiuk,
Thank you @eladkal for pointing me to this issue, I think I can help on this one. (Looks like the closed PR was almost it).
I am not sure how i'm going to test it thought as I need a `Google Display & Video 360` account and they do not seem to offer "free trial" stuff, any suggestions ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun edited a comment on issue #14075: GoogleDisplayVideo360SDFtoGCSOperator does not pass the correct resource_name to the download_media hook
Posted by GitBox <gi...@apache.org>.
pierrejeambrun edited a comment on issue #14075:
URL: https://github.com/apache/airflow/issues/14075#issuecomment-1075579709
Hello @eladkal and @potiuk,
Thank you @eladkal for pointing me to this issue, I think I can help on this one. (Looks like the closed PR was almost it).
I am not sure on how i'm going to test it thought as I need a `Google Display & Video 360` account and they do not seem to offer "free trial" stuff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org