[GitHub] [airflow] tooptoop4 opened a new issue #8453: Spark version 3 does not work

[GitBox <gi...@apache.org>]

tooptoop4 opened a new issue #8453: Spark version 3 does not work
URL: https://github.com/apache/airflow/issues/8453
 
 
   If anyone runs into this in the future I've found out where the issue is in the spark_submit_hook.py.
   Line 419 in spark submit hook
   
   match_exit_code = re.search(r'\s*Exit code: (\d+)', line)
   in spark 3.0 the line that prints the exit code is actually lower case E on "Exit code:" so this re.search will never find that value. To fix this you can simply switch the line to this
   match_exit_code = re.search(r'\s*Exit code: (\d+)', line, re.IGNORECASE)
   Which should also be backwards compatible.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services