Re: [I] [Feature Request]: Upgrade or provide ETA on dill [beam]

["siddharthab (via GitHub)" <gi...@apache.org>]

siddharthab commented on issue #22893:
URL: https://github.com/apache/beam/issues/22893#issuecomment-1907299034

   The suggestion to force a version of dill may not work in many situations, and so a proper remediation will be very much appreciated.
   
   Many people are using pip-tools as their dependency manager these days, which [does not allow](https://github.com/jazzband/pip-tools/issues/215) a user to force install a specific version. Build systems like Bazel rules for Python also rely on pip-tools to build the dependency tree with resolved versions. I was unable to override the version constraints from my dependencies in my devops setup.
   
   Environment compatibility checks between submission and runtime environments are actually supposed to be part of distributed compute systems. Dask and newer versions of Beam SDK have them. Should we not rely on these checks and remove the narrow version constraints in the requirements? If a user wants to use a provided (instead of custom) Beam docker image, then maybe each image can come with suggested job submission environment setup scripts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org