Re: [PR] Created new exception, that provides details of each step of a job ru… [airflow]

["geraj1010 (via GitHub)" <gi...@apache.org>]

geraj1010 commented on code in PR #48466:
URL: https://github.com/apache/airflow/pull/48466#discussion_r2070752128


##########
providers/dbt/cloud/src/airflow/providers/dbt/cloud/operators/dbt.py:
##########
@@ -213,7 +213,12 @@ def execute(self, context: Context):
                 ):
                     self.log.info("Job run %s has completed successfully.", self.run_id)
                 else:
-                    raise DbtCloudJobRunException(f"Job run {self.run_id} has failed or has been cancelled.")
+                    # Gather job run details (including run steps), so we can output the logs from each step
+                    run_details = self.hook.get_job_run(
+                        run_id=self.run_id, account_id=self.account_id, include_related=["run_steps"]
+                    ).json()["data"]
+
+                    raise DbtCloudJobRunDetailsException(self.run_id, run_details)

Review Comment:
   I'm not opposed to that. The only place I see the original exception `DbtCloudJobRunException` being used, is in the hook method `wait_for_job_run_status`. I think we would still want the ability to retain the same message for that method though, so I will need to refactor the new exception to allow for custom message. In addition, I think I might use the old name for the exception, since there would no longer be a need to distinguish between the two exceptions IMO.
   
   `wait_for_job_run_status` does have `account_id` in its signature, so I think I can utilize the new exception there, with the suggested refactoring above. 
   
   If there is potential for many new exceptions in the future, then I think it would make sense to have a separate module for them.
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org