[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

[GitBox <gi...@apache.org>]

turbaszek commented on a change in pull request #8954:
URL: https://github.com/apache/airflow/pull/8954#discussion_r429236013



##########
File path: airflow/providers/google/cloud/hooks/datafusion.py
##########
@@ -357,12 +418,45 @@ def list_pipelines(
             )
         return json.loads(response.data)
 
+    def _get_pipeline(
+        self,
+        pipeline_name: str,
+        faux_pipeline_id: str,
+        instance_url: str,
+        namespace: str = "default",
+    ) -> Optional[Dict]:
+        url = os.path.join(
+            instance_url,
+            "v3",
+            "namespaces",
+            namespace,
+            "apps",
+            pipeline_name,
+            "workflows",
+            "DataPipelineWorkflow",
+            "runs",
+        )
+        response = self._cdap_request(url=url, method="GET")
+        if response.status != 200:
+            raise AirflowException(
+                f"Retrieving a pipeline failed with code {response.status}"
+            )
+
+        pipelines_list = json.loads(response.data)
+        for pipe in pipelines_list:

Review comment:
       @jaketf do I correctly understand that state of program can be acceses under:
   ```
   https://xxx-dot-eun1.datafusion.googleusercontent.com/api/v3/namespaces/default/apps/airflow_test/workflows/DataPipelineWorkflow/runs/5d1a277b-17ae-4cab-b3a6-5d23a2f8f672
   ```
   at least that's how it seems to be done here
   https://github.com/GoogleCloudPlatform/terraform-provider-cdap/blob/e16a2c5188bf2299482d5c8261ca01f27cd19a44/cdap/resource_streaming_program_run.go#L244
   
   The problem is that I get 404, any ideas?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org