You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/04 16:00:54 UTC
[GitHub] [airflow] sarahamilton opened a new pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
sarahamilton opened a new pull request #14074:
URL: https://github.com/apache/airflow/pull/14074
The GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not create an xcom that is necessary in the SDF workflow. The "name" xcom is needed for the next step in the process, GoogleDisplayVideo360SDFtoGCSOperator, to work correctly. This patch adds that xcom so it is available to the next step in the workflow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#issuecomment-778619487
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14074: Fix GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#issuecomment-848361526
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #14074: Fix GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #14074:
URL: https://github.com/apache/airflow/pull/14074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a change in pull request #14074: Fix GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
eladkal commented on a change in pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#discussion_r606855906
##########
File path: airflow/providers/google/marketing_platform/hooks/display_video.py
##########
@@ -228,7 +228,7 @@ def get_sdf_download_operation(self, operation_name: str):
result = (
self.get_conn_to_display_video() # pylint: disable=no-member
.sdfdownloadtasks()
- .operation()
+ .operations()
Review comment:
This change already covered in https://github.com/apache/airflow/pull/13703
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sarahamilton commented on a change in pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
sarahamilton commented on a change in pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#discussion_r581332634
##########
File path: airflow/providers/google/marketing_platform/hooks/display_video.py
##########
@@ -228,7 +228,7 @@ def get_sdf_download_operation(self, operation_name: str):
result = (
self.get_conn_to_display_video() # pylint: disable=no-member
.sdfdownloadtasks()
- .operation()
+ .operations()
Review comment:
Sorry for the late response! I just saw this. Yes, the operator was previously not working at all. I identified several different issues all causing the operator to not work and submitted in a few additional patches.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#discussion_r575667283
##########
File path: airflow/providers/google/marketing_platform/operators/display_video.py
##########
@@ -635,6 +635,10 @@ def execute(self, context: dict) -> Dict[str, Any]:
self.log.info("Creating operation for SDF download task...")
operation = hook.create_sdf_download_operation(body_request=self.body_request)
+ name = operation["name"]
+ self.xcom_push(context, key="name", value=name)
+ self.log.info("Created SDF operation with name: %s", name)
Review comment:
Nice! This will definitely simplify accessing the value - currently users still can access it doing via `{{ task_instance.xcom_pull("task_id")["name"] }}`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#issuecomment-787501381
Hey @sarahamilton -> can you please rebase this one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14074: Fixes Issue #14073 - GoogleDisplayVideo360CreateSDFDownloadTaskOperator does not push necessary xcom
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14074:
URL: https://github.com/apache/airflow/pull/14074#discussion_r575667128
##########
File path: airflow/providers/google/marketing_platform/hooks/display_video.py
##########
@@ -228,7 +228,7 @@ def get_sdf_download_operation(self, operation_name: str):
result = (
self.get_conn_to_display_video() # pylint: disable=no-member
.sdfdownloadtasks()
- .operation()
+ .operations()
Review comment:
Does this mean that the operator was not working previously? 👀
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org