You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2023/06/28 10:12:24 UTC
[GitHub] [arrow] raulcd opened a new issue, #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
raulcd opened a new issue, #36349:
URL: https://github.com/apache/arrow/issues/36349
### Describe the bug, including details regarding any error messages, version, and platform.
During the last 5 days the `test_python_datetime_with_pytz_timezone` on [test-conda-python-3.11-hypothesis](https://github.com/ursacomputing/crossbow/actions/runs/5395935281/jobs/9798979749) has been failing with the following error:
```
> raise UnknownTimeZoneError(zone)
E pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
E Falsifying example: test_python_datetime_with_pytz_timezone(
E self=<pyarrow.tests.test_pandas.TestConvertDateTimeLikeTypes object at 0x7f9626db7710>,
E tz=ZoneInfo('build/etc/localtime'),
E )
```
I have been **NOT** able to reproduce locally with:
`PYTHON=3.11 PYARROW_TEST_HYPOTHESIS="ON" PYTEST_ARGS="-m hypothesis" docker-compose run --rm conda-python-pandas`
### Component(s)
Continuous Integration, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36349:
URL: https://github.com/apache/arrow/issues/36349#issuecomment-1611725713
No, I looked at the traceback a while ago (it's already failing longer than 5 days), and honestly no idea how this can happen or where this comes from ..
The test is using the `hypothesis.extra.pytz.timezones()` and `hypothesis.strategies.timezones()` strategies. But somehow this strategy can produce a `tzinfo` that gives a "build/etc/localtime" string representation. I wonder if that's a bug upstream, as such a zone doesn't sound to make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd closed issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd closed issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
URL: https://github.com/apache/arrow/issues/36349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36349:
URL: https://github.com/apache/arrow/issues/36349#issuecomment-1611742422
So actually, just trying that out, this string works for `zoneinfo`, and this is one of the "available" zones:
```python
>>> import datetime
>>> import zoneinfo
>>> "build/etc/localtime" in zoneinfo.available_timezones()
True
>>> tz = zoneinfo.ZoneInfo("build/etc/localtime")
>>> tz
zoneinfo.ZoneInfo(key='build/etc/localtime')
>>> tz.key
'build/etc/localtime'
>>> tz.utcoffset(datetime.datetime.now())
datetime.timedelta(0)
>>> print(datetime.datetime.now().replace(tzinfo=tz))
2023-06-28 18:19:13.737991+00:00
```
So probably this timezone gets created by the zoneinfo hypothesis strategy, but we still always use pytz timezones for export (there is an issue about stopping to do this), and then pytz doesn't recognize is:
```python
>>> pa.lib.tzinfo_to_string(tz)
'build/etc/localtime'
>>> pa.lib.string_to_tzinfo("build/etc/localtime")
...
UnknownTimeZoneError: 'build/etc/localtime'
```
which comes from
```python
>>> pytz.timezone("build/etc/localtime")
>>>
UnknownTimeZoneError: 'build/etc/localtime'
```
---
I think the question is what we should do with such a `'build/etc/localtime'` string though. The easy thing would be to check for this string and then create a zoneinfo timezone instead of pytz (in `pa.lib.string_to_tzinfo`). But is that the correct thing to do? Should we even allow such a timezone in the arrow type? I think our spec says that we allow either fixed offset strings or a name from the Olsen database. `zoneinfo` should be using an Olsen tzdata database, but is `'build/etc/localtime'` part of that? And is that cross-platform? Can we somehow convert that in a proper name or fixed offset?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36349:
URL: https://github.com/apache/arrow/issues/36349#issuecomment-1611761506
So the "build/etc/localtime" is an actual file in my tzdata directory (base path from `zoneinfo.TZPATH`), and it's an actual TZif2 file (just like all other zones).
But for example `pytz` bundles its own copy of the timezone database, and in this directory there is no such "build" subdir.
And the `tzdata` package (https://pypi.org/project/tzdata) also doesn't include such a "build/etc/localtime" (because it can't know what the local time is of the machine it will get installed on).
Maybe the best path forward is to disallow this timezone? (although in general we don't really check if the timezone provided by the user is correct, i.e. is a fixed offset or valid Olsen zone name)
Or short term for this test, skip it in case the timezone is this 'build/etc/localtime'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #36349: [Python][CI] Hypothesis nightly test fails with pytz.exceptions.UnknownTimeZoneError: 'build/etc/localtime'
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #36349:
URL: https://github.com/apache/arrow/issues/36349#issuecomment-1611140380
@jorisvandenbossche any idea on that one? I am not sure why it tries to use `build/etc/localtime` as a timezone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org