You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "rohanjain101 (via GitHub)" <gi...@apache.org> on 2023/05/15 20:04:25 UTC
[GitHub] [arrow] rohanjain101 opened a new issue, #35600: Allow setting path to timezone db in pyarrow through python API
rohanjain101 opened a new issue, #35600:
URL: https://github.com/apache/arrow/issues/35600
### Describe the enhancement requested
The timezone DB is required to be at %USERPROFILE%\Downloads\tzdata in pyarrow on windows. It would be nice to have a python API to change the path where this db should be found, I believe this is currently only possible through the C++ implementation using arrow:initialize.
It would also be nice if the timezone db was not required if tzdata/pytz are installed.
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #35600: Allow setting path to timezone db in pyarrow through python API
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1549227248
> It would also be nice if the timezone db was not required if tzdata/pytz are installed.
If I remember correctly, the problem here is that this uses a different format than what the tz implementation we use (https://howardhinnant.github.io/date/tz.html) expects.
See eg https://github.com/apache/arrow/issues/28868#issuecomment-1378032344 and https://github.com/apache/arrow/issues/31472 for previous discussion about this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1580650747
@mattijn what does `import os; os.path.expanduser("~/Downloads")` return for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1549235149
And the R bindings already expose this, which essentially is a small wrapper around the setting a C++ option (`arrow::GlobalOptions.timezone_db_path`):
https://github.com/apache/arrow/blob/f6e447944f2a2ab108d5971daf351b7443bc96fb/r/src/config.cpp#L38-L47
So it should certainly be possible to add a similar function to pyarrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1580711158
So then the error message _could_ be expected? On the C++ side, it is using `wordexp` for the path expansion of `~\Downloads`, no idea how that exactly behaves on Windows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mattijn commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "mattijn (via GitHub)" <gi...@apache.org>.
mattijn commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1580624761
Thanks for all the work on arrow!
I'm on windows and when running the following:
```python
from datetime import datetime
import pyarrow.compute as pc
pc.strftime(datetime(2004, 8, 1))
```
I get the following error:
> ```cmd
> ArrowInvalid: Cannot locate timezone 'UTC': Timezone database not found at "D:\Users\Hoek\Downloads\tzdata"
> ```
Observe, that the error path refers to `D:`.
But in OP the following is mentioned:
> The timezone DB is required to be at %USERPROFILE%\Downloads\tzdata in pyarrow on windows.
But if I check:
```python
! echo %USERPROFILE%
```
> ```cmd
> C:\Users\Hoek
> ```
and
```python
from pathlib import Path
Path.home()
```
> ```cmd
> WindowsPath('C:/Users/Hoek')
> ```
It is both linking to `C:`. How you know it is on `D:`? `D:` is right though..
There is no `Downloads` folder for me on `C:\Users\Hoek` (therefor https://stackoverflow.com/a/74292266/2459096 errored)
I assume it is just early days and this user experience will improve soon.
Again, thanks for all the hard work!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Allow setting path to timezone db through python API [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche closed issue #35600: [Python] Allow setting path to timezone db through python API
URL: https://github.com/apache/arrow/issues/35600
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mattijn commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "mattijn (via GitHub)" <gi...@apache.org>.
mattijn commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1580718180
Yes. The error message is right indeed, C++ gets the right location using `wordexp`.
Using `%USERPROFILE%` or `os.path.expanduser("~/Downloads")` not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mattijn commented on issue #35600: [Python] Allow setting path to timezone db through python API
Posted by "mattijn (via GitHub)" <gi...@apache.org>.
mattijn commented on issue #35600:
URL: https://github.com/apache/arrow/issues/35600#issuecomment-1580689094
```cmd
'C:\\Users\\Hoek/Downloads'
```
Apparently my Downloads location is not on `C`. See Downloads dialog:
<img src="https://github.com/apache/arrow/assets/5186265/e111eb0d-15fb-4423-8479-cc98fa39ec8a" width="320" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org