Re: [PR] sdk: Refactor `get_python_source` to strip comments (#50327) [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on PR #57782:
URL: https://github.com/apache/airflow/pull/57782#issuecomment-3496216224

   @ashb 
   
   ```
   from types import SimpleNamespace
   task = SimpleNamespace(kubernetes=lambda f: f)
   ```
   
   Yeah, my thought exactly that we could not remove the decorators but replace them with something empty. But I think in this case also indentation matter - because we would have to keep the functions at the nested indentation level in the generated script (those callbacks are converted to top-level functions now). I am not sure if that can be handled easily. No esy idea for that one or that it's worth the effort.
   
   I think for now simple removeal of the comments has drawbacks of course. It does not handle all cases -  for example it will not work with multi-line strings and likely some other constructs.
   
   ````
       def b_task():
           print(""
   hello
   """
           print("more hello")
   ````
   
   In order to properly handle those constructs, we would really have to parse the whole code with AST and know which lines are supposed to be indented and which not. 
   
   For example this case is not easy to handle without knowing that hello2 is part of a multi-line string -- you need to parse the whole python file to know.
   
   ````
       def b_task():
           print(""
   hello
   
          hello 2
   """
           print("more hello")
   ````
   
   
   Or a better would would be simply to forbid those cases and turn ParserError into a more meaningful one (only use plain functions, do not break indentation. That wouls be simple- we extract the whispace from the first line and reject any function that has line that do not begin with the same whitespace prefix.
   
   That would be **simple** and effective solution (and back-compatible - those functions now cause ParsingException). We can easily add this limitation and even document it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org