You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2023/01/18 19:13:07 UTC

[GitHub] [iceberg] srilman opened a new issue, #6620: Python: More Flexible Dependency Requirements, especially for Optional Deps

srilman opened a new issue, #6620:
URL: https://github.com/apache/iceberg/issues/6620

   ### Feature Request / Improvement
   
   I started to use PyIceberg as a dependency for an application that uses a lot of data-science related libraries, but I started to run into issues due to many conflicting sub-dependency versions, particularly:
   
   - PyArrow (Optional)
   - S3FS (Optional)
   - Boto3 (Optional)
   - FsSpec (Required, but its version is tied with s3fs I believe)
   
    I was wondering, is it necessary for all of PyIceberg's dependencies to be fixed to one version, or can we have more flexible options. This would be especially helpful for optional dependencies.
   
   I haven't done a deep-dive into the code to see how these dependencies are used, but it looks to me that the latest version of PyArrow is not required.
   
   ### Query engine
   
   None


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] srilman closed issue #6620: Python: More Flexible Dependency Requirements, especially for Optional Deps

Posted by "srilman (via GitHub)" <gi...@apache.org>.
srilman closed issue #6620: Python: More Flexible Dependency Requirements, especially for Optional Deps
URL: https://github.com/apache/iceberg/issues/6620


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] srilman commented on issue #6620: Python: More Flexible Dependency Requirements, especially for Optional Deps

Posted by "srilman (via GitHub)" <gi...@apache.org>.
srilman commented on issue #6620:
URL: https://github.com/apache/iceberg/issues/6620#issuecomment-1404304169

   @fokko Any thoughts?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] Fokko commented on issue #6620: Python: More Flexible Dependency Requirements, especially for Optional Deps

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #6620:
URL: https://github.com/apache/iceberg/issues/6620#issuecomment-1414042971

   Thanks for pining for me @srilman 
   
   Most libraries can vary the version, and some are fixed. For example, `aiobotocore` is linked to `boto3`. For Arrow, I think you're okay using an older version.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org