[GitHub] [buildstream] abderrahim commented on issue #1412: Potentially remove all those pesky .pyi files

[GitBox <gi...@apache.org>]

abderrahim commented on issue #1412:
URL: https://github.com/apache/buildstream/issues/1412#issuecomment-1005902890


   There are actually three "pure python" modes in cython:
   1. good old "pure python" code
   2. good old "pure python" + hot code paths rewritten in cython syntax in a separate .pxd file
   3. pure python with special cython decorators
   
   All three can be executed as pure python (3. needs to have cython installed), and can be compiled for performance gain (1. offers modest gains, while 2. and 3. can leverage the full potential of cython). 2. has the obvious downside of having to keep two implementations in sync for the hot paths.
   
   What I was investigating is 3. which should allow us to get rid of the cython syntax without losing the other advantages of cython. Currentyl the only missing feature I noticed is `cdef enum` (https://github.com/cython/cython/issues/4252). Then there is the huge bug that shows that nobody is currently using this in production (https://github.com/cython/cython/issues/3957).
   
   What I'm currently investigating is 4. mypyc. mypyc is an alternative to cython. It is used by two tools we depend on (mypy and black) so it's used in production (even though it is still considered alpha software). mypyc compiles (a growing subset of) real pure python with type annotations. It is supposed to get a modest speedup when compiling code without type annotations, and a good speedup when adding type annotations.
   
   The upside is that we'd end up with real pure python that can be executed without a compilation step during e.g. debug-edit cycle, and potentially compile the whole tool (with heavy type annotations on the hot paths) on release.
   
   Currently, trying to port `_node.pyx`, I'm struggling mainly with the fact that mypy takes None-ability very seriously. So if something can be `None`, it should be checked before being used.
   
   I think there is a buildstream benchmark somewhere. It would be nice to use it to test this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org