[GitHub] [airflow] uranusjr commented on pull request #15359: Operator to run a Python callable against an arbitrary Python interpreter

[GitBox <gi...@apache.org>]

uranusjr commented on pull request #15359:
URL: https://github.com/apache/airflow/pull/15359#issuecomment-819416512


   The main advantage to me is the ability to easily pass arguments to and return values from the operator. I suspect (no verifications whatsoever) this may provide a smoother upgrade path than `PythonVirtualenvOperator` for people migrating from Airflow 1 on Python 2.7 as well (they already have an environment for the Airflow 1 installation). `SubprocessHook` can only take command line arguments, and returning is more complicated as well (stdout parsing is always flimsy). The biggest advantage of Python-specific operators to me is they have a dedicated channel for data exchange, which even with the pickling restrictions is much more robust and useful.
   
   The current implementation is sharing most subprocessing stuff with `PythonVirtualenvOperator` that was already there previously. It’s probably worthwhile to do further refactoring to make them share the implementation of `SubprocessHook`, but that’s involve potentially changing the behaviour of `PythonVirtualenvOperator` as well, so I’m more inclined to do it separately.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org