You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "canassa (via GitHub)" <gi...@apache.org> on 2026/02/20 22:56:07 UTC
[PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
canassa opened a new pull request, #49353:
URL: https://github.com/apache/arrow/pull/49353
### Rationale for This Change
The vendored Howard Hinnant date library hardcodes `/usr/share/zoneinfo` as the timezone database path in `discover_tz_dir()`. It does not check the `TZDIR` environment variable, which is the POSIX standard mechanism for overriding this path.
This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where `zoneinfo` resides under a non standard path like `/nix/store/.../share/zoneinfo`.
The upstream library also lacks `TZDIR` support (HowardHinnant/date#858).
---
### What Changes Are Included in This PR?
* `cpp/src/arrow/vendored/datetime/tz.cpp`: Check the `TZDIR` environment variable in `discover_tz_dir()` before falling back to platform specific hardcoded paths. Uses `stat()` and `S_ISDIR()` for validation, matching the existing pattern in the function.
* `cpp/src/arrow/vendored/datetime/README.md`: Document the patch.
* `cpp/src/arrow/public_api_test.cc`: Add a non Windows test that sets `TZDIR` and verifies timezone resolution succeeds through Arrow's compute API.
* `python/pyarrow/conftest.py`: Respect `TZDIR` in the emscripten `timezone_data` test marker.
---
### Are These Changes Tested?
Yes. A new `Misc.TZDIREnvironmentVariable` test sets `TZDIR` to a valid `zoneinfo` directory and casts a UTC timestamp to `America/New_York`, verifying the code path works end to end.
---
### Are There Any User Facing Changes?
Arrow now respects the `TZDIR` environment variable on non Windows platforms, enabling timezone operations on systems without `/usr/share/zoneinfo`.
Fixes #49351
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "FreaxMATE (via GitHub)" <gi...@apache.org>.
FreaxMATE commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3997523277
> > Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?
>
> Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.
>
> I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.
I havee the same issue. For anyone trying this quick workaround for nixos:
```fish
sudo mkdir -p /usr/share
sudo ln -s /etc/zoneinfo /usr/share/zoneinfo
```
Thanks @canassa for the fix!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "canassa (via GitHub)" <gi...@apache.org>.
canassa commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937617676
> iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in `cpp/src/arrow/vendored/` if possible. Could you check if we can solve this without the change in `cpp/src/arrow/vendored/datetime/tz.cpp`?
Sure, I will take a look!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "canassa (via GitHub)" <gi...@apache.org>.
canassa commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937590220
> Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?
Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.
I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3998148568
Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!
We had recently moved to use `std::chrono` for accessing timezone data on Windows https://github.com/apache/arrow/pull/48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)
If you're still seeing this issue, it would be more productive to disable vendored library further and switch to `std::chrono` than to keep hardcoding `TZDIR`.
If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937608214
Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in `cpp/src/arrow/vendored/` if possible. Could you check if we can solve this without the change in `cpp/src/arrow/vendored/datetime/tz.cpp`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937570934
Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937539135
:warning: GitHub issue #49351 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3937673675
The linting issue can be fixed with ([source](https://arrow.apache.org/docs/developers/python/development.html#python-coding-style)):
```bash
pre-commit run --show-diff-on-failure --color=always --all-files cpp
```
The cpp paginator issues are unrelated (https://github.com/apache/arrow/issues/49347).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49351: [C++] Check TZDIR environment variable in vendored date library [arrow]
Posted by "canassa (via GitHub)" <gi...@apache.org>.
canassa commented on PR #49353:
URL: https://github.com/apache/arrow/pull/49353#issuecomment-3938463321
Thanks, I pushed the linting fixes.
I also investigated other approaches, but the vendored library change seem to be unavoidable. The `get_tz_dir()` is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org