You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/12 22:19:38 UTC

[GitHub] [airflow] SamWheating commented on issue #20832: Unable to specify Python version for AwsGlueJobOperator

SamWheating commented on issue #20832:
URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011502821


   I think that this is because the GlueHook is pretty opinionated and hardcodes the value of `Command` when running the `create_glue_job` command:
   
   https://github.com/apache/airflow/blob/2ab2ae8849bf6d80a700b1b74cef37eb187161ad/airflow/providers/amazon/aws/hooks/glue.py#L181-L225
   
   So when you provide `Command` as a `create_job_kwargs`, it ends up being supplied twice to that function (Although I suspect that this would be a typeError, not a keyError 🤔)
   
   Anyways, thoughts on just adding a `pythonVersion` argument to the `GlueJobOperator`?
   
   If y'all think that this is a satisfactory fix, feel free to assign this issue to me and I can put up a quick PR. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org