[GitHub] [airflow] rounakdatta commented on a change in pull request #17349: #16037 Add support for passing templated requirements.txt in PythonVirtualenvOperator

[GitBox <gi...@apache.org>]

rounakdatta commented on a change in pull request #17349:
URL: https://github.com/apache/airflow/pull/17349#discussion_r680879448



##########
File path: airflow/operators/python.py
##########
@@ -333,20 +337,31 @@ def __init__(
             templates_exts=templates_exts,
             **kwargs,
         )
-        self.requirements = list(requirements or [])
+        if not isinstance(requirements, str):
+            self.requirements = list(requirements or [])
+        else:
+            self.requirements = requirements
         self.string_args = string_args or []
         self.python_version = python_version
         self.use_dill = use_dill
         self.system_site_packages = system_site_packages
-        if not self.system_site_packages and self.use_dill and 'dill' not in self.requirements:
-            self.requirements.append('dill')
         self.pickling_library = dill if self.use_dill else pickle
 
+    def pre_execute(self, context: Any):
+        if isinstance(self.requirements, list):
+            return
+
+        import pkg_resources
+        self.requirements = [str(req) for req in pkg_resources.parse_requirements(self.requirements)]

Review comment:
       Ack, makes sense, let me work on that change.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org