[GitHub] [airflow] hussein-awala commented on a diff in pull request #32847: Update Error details for Generic Error Code

["hussein-awala (via GitHub)" <gi...@apache.org>]

hussein-awala commented on code in PR #32847:
URL: https://github.com/apache/airflow/pull/32847#discussion_r1284716038


##########
airflow/providers/google/cloud/hooks/dataflow.py:
##########
@@ -1014,8 +1014,9 @@ def start_sql_job(
         self.log.info("Output: %s", proc.stdout.decode())
         self.log.warning("Stderr: %s", proc.stderr.decode())
         self.log.info("Exit code %d", proc.returncode)
+        stderr_last_20_lines = "\n".join(proc.stderr.decode().strip().splitlines()[-20:])
         if proc.returncode != 0:
-            raise AirflowException(f"Process exit with non-zero exit code. Exit code: {proc.returncode}")
+            raise AirflowException(f"Process exit with non-zero exit code. Exit code: \ {proc.returncode} \n Error Details : {stderr_last_20_lines}")

Review Comment:
   I see that we already log the stderr lines in warning level, so maybe we could move this part to the condition and log it in error level if the return code is not equal to 0, and if it is 0 we log it in warning level.
   WDYT?
   something like:
   ```python
           if proc.returncode != 0:
               self.log.error("Stderr: %s", proc.stderr.decode())
               self.log.info("Exit code %d", proc.returncode)
               raise AirflowException(f"Process exit with non-zero exit code. Exit code: {proc.returncode}")
          else:
              self.log.warning("Stderr: %s", proc.stderr.decode())
              self.log.info("Exit code %d", proc.returncode)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org