You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/04/04 23:38:00 UTC
[jira] [Work logged] (BEAM-14235) parquetio module does not parse PEP-440 compliant Pyarrow version
[ https://issues.apache.org/jira/browse/BEAM-14235?focusedWorklogId=752564&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-752564 ]
ASF GitHub Bot logged work on BEAM-14235:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Apr/22 23:37
Start Date: 04/Apr/22 23:37
Worklog Time Spent: 10m
Work Description: asf-ci commented on PR #17275:
URL: https://github.com/apache/beam/pull/17275#issuecomment-1088123359
Can one of the admins verify this patch?
Issue Time Tracking
-------------------
Worklog Id: (was: 752564)
Time Spent: 40m (was: 0.5h)
> parquetio module does not parse PEP-440 compliant Pyarrow version
> -----------------------------------------------------------------
>
> Key: BEAM-14235
> URL: https://issues.apache.org/jira/browse/BEAM-14235
> Project: Beam
> Issue Type: Bug
> Components: io-py-parquet
> Affects Versions: 2.27.0
> Reporter: Arwin S Tio
> Priority: P3
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In version > 2.27, introduced by this PR: [https://github.com/apache/beam/pull/13302/files#diff-33b0b6b112036df96f341aa83b88efba9215ec14dfabc9db9e9ffe66a23154a2R55]
> The parquetio module parses the pyarrow version like this:
> {code:java}
> ARROW_MAJOR_VERSION, _, _ = map(int, pa.__version__.split('.')) {code}
> (see [https://github.com/apache/beam/blob/v2.27.0/sdks/python/apache_beam/io/parquetio.py#L55)]
>
> This does not support all PEP-440 compliant versions: [https://peps.python.org/pep-0440/]
>
> For example, if pyarrow were to have a version like this: *1.0.0+abc.7,* then this module would fail:
> {code:java}
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/runpy.py", line 183, in _run_module_as_main
> mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
> File "/usr/local/lib/python3.7/runpy.py", line 109, in _get_module_details
> __import__(pkg_name)
> File "/usr/local/lib/python3.7/site-packages/apache_beam/__init__.py", line 93, in <module>
> from apache_beam import io
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/__init__.py", line 28, in <module>
> from apache_beam.io.parquetio import *
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/parquetio.py", line 53, in <module>
> ARROW_MAJOR_VERSION, _, _ = map(int, pa.__version__.split('.'))
> ValueError: invalid literal for int() with base 10: '0+abc.7'{code}
>
> In practice, this would fail when somebody forks pyarrow, like yours truly.
>
> We can fix this by using *pkg_resourses.parse_version* which is PEP-440 compliant starting setuptools 6.0.
>
> If maintainers agree with this change I would be wiling to submit a PR.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)