You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "h-vetinari (via GitHub)" <gi...@apache.org> on 2026/01/16 02:47:25 UTC
[I] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
h-vetinari opened a new issue, #48874:
URL: https://github.com/apache/arrow/issues/48874
### Describe the enhancement requested
I've started looking into this because the tests on the arrow feedstock are failing with
```
%SRC_DIR%\cpp\src\arrow\compute\kernels\test_util_internal.cc(87): error: Failed
'_error_or_value26.status()' failed with Invalid: Cannot locate or parse timezone 'Australia/Broken_Hill': Timezone database not found at "C:\Users\VssAdministrator\Downloads\tzdata"
```
(these failures had been hidden by a bug in a batch file, apologies).
Conda-forge ships `tzdata` as a noarch package, and it also works on windows; for example, I've contributed basic support for using conda-forge's tzdata to orc: https://github.com/apache/orc/pull/1882
I've started looking at patching this into arrow as well, but the code is pretty involved. I can put up a draft PR with some very basic first steps that I have, if that helps.
It's a bit urgent for me to fix this, because I need to fix the CI on the feedstock, and - short of disabling the tzdb-related tests on windows - I can't currently.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3776651810
> I went back and found that the issue appeared (exactly) with arrow v22; v21 and below are fine.
Hmm, I see. I'm not sure what changed that made it appear, but you could try to go with the patch in https://github.com/apache/arrow/issues/48874#issuecomment-3772831757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3773804152
> I've started looking into this because the tests on the arrow feedstock are failing with
With which Arrow version, and when have they started failing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3783101240
> Generally, I'm not very keen on the whole "needs env var to run tests", because it ends up testing a very different thing from what's happening for user installs.
Ideally yes, in practice it's difficult to avoid when there is no standardized location for a given piece of data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3789787978
Gandiva is unmaintained, feel free to disable those tests.
Le 23 janvier 2026 07:29:06 GMT+01:00, h-vetinari ***@***.***> a écrit :
>h-vetinari left a comment (apache/arrow#48874)
>
>Quick update: I backported #48601 to v22 in https://github.com/conda-forge/arrow-cpp-feedstock/pull/1913; that gets us quite a bit further, but there are still some remaining failures:
>```
>98% tests passed, 2 tests failed out of 102
>[...]
>The following tests FAILED:
> 100 - gandiva-internals-test (Failed) gandiva-tests unittest
> 102 - gandiva-projector-test (Failed) gandiva-tests unittest
>```
>
><details>
>
>### `gandiva-internals-test`
>
>```
>[ RUN ] TestGdvFnStubs.TestToUtcTimezone
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1081): error: Expected equality of these values:
> 1330423200000
> ts2
> Which is: 0
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1087): error: Expected equality of these values:
> ts2
> Which is: 0
> -1800000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1094): error: Expected equality of these values:
> ts2
> Which is: 0
> 1520758800000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1100): error: Expected equality of these values:
> ts2
> Which is: 0
> 1520841600000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1104): error: Value of: context.get_error()
>Expected: is equal to "America/LA is an invalid time zone name."
> Actual: "Asia/Kolkata is an invalid time zone name."
>
>[ FAILED ] TestGdvFnStubs.TestToUtcTimezone (5 ms)
>[ RUN ] TestGdvFnStubs.TestFromUtcTimezone
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1118): error: Expected equality of these values:
> ts2
> Which is: 0
> 55800000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1124): error: Expected equality of these values:
> ts2
> Which is: 0
> 18000000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1130): error: Expected equality of these values:
> ts2
> Which is: 0
> 1520730000000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1136): error: Expected equality of these values:
> ts2
> Which is: 0
> 1520816400000
>
>%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1140): error: Value of: context.get_error()
>Expected: is equal to "India is an invalid time zone name."
> Actual: "Asia/Kolkata is an invalid time zone name."
>
>[ FAILED ] TestGdvFnStubs.TestFromUtcTimezone (0 ms)
>
>[...]
>
>[----------] Global test environment tear-down
>[==========] 143 tests from 26 test suites ran. (647 ms total)
>[ PASSED ] 141 tests.
>[ FAILED ] 2 tests, listed below:
>[ FAILED ] TestGdvFnStubs.TestToUtcTimezone
>[ FAILED ] TestGdvFnStubs.TestFromUtcTimezone
>
> 2 FAILED TESTS
>```
>
>### `gandiva-projector-test`
>
>```
>[ RUN ] DateTimeTestProjector.TestToUtcTimestamp
>%SRC_DIR%\cpp\src\gandiva\tests\date_time_test.cc(771): error: Value of: status.ok()
> Actual: false
>Expected: true
>
>unknown file: error: C++ exception with description "invalid vector subscript" thrown in the test body.
>
>[ FAILED ] DateTimeTestProjector.TestToUtcTimestamp (49 ms)
>[ RUN ] DateTimeTestProjector.TestFromUtcTimestamp
>%SRC_DIR%\cpp\src\gandiva\tests\date_time_test.cc(826): error: Value of: status.ok()
> Actual: false
>Expected: true
>
>unknown file: error: C++ exception with description "invalid vector subscript" thrown in the test body.
>
>[ FAILED ] DateTimeTestProjector.TestFromUtcTimestamp (46 ms)
>[----------] 12 tests from DateTimeTestProjector (1131 ms total)
>
>[...]
>
>[----------] Global test environment tear-down
>[==========] 211 tests from 16 test suites ran. (19339 ms total)
>[ PASSED ] 209 tests.
>[ FAILED ] 2 tests, listed below:
>[ FAILED ] DateTimeTestProjector.TestToUtcTimestamp
>[ FAILED ] DateTimeTestProjector.TestFromUtcTimestamp
>
> 2 FAILED TESTS
> YOU HAVE 2 DISABLED TESTS
>```
>
></details>
>
>Part of that could of course be due to the backport to v22 (which cherry-picked cleanly though, aside from some CI config), but I won't get around fixing this also on v22 one way or another. I was planning to fix CI on v22 first and then try v23. If helpful, I can also try out v23 (with the backport of #48601) as well.
>
>--
>Reply to this email directly or view it on GitHub:
>https://github.com/apache/arrow/issues/48874#issuecomment-3788531203
>You are receiving this because you were mentioned.
>
>Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3771572791
This is currently a blocker for anything on the feedstock, unfortunately. Well, we could continue to ignore the previously existing errors, but that's quite far from responsible maintainership. I'd consider it as a last resort maybe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3782854740
> The environment variable `ARROW_TIMEZONE_DATABASE` should be set to somewhere where the timezone database resides, is that not the case in your test script?
We've started running test for libarrow since v18 (though https://github.com/conda-forge/arrow-cpp-feedstock/pull/1058 was ~1.5yrs in the making), without setting `ARROW_TIMEZONE_DATABASE`, and that ran fine before v22 (still works fine on unix even for v22).
Generally, I'm not very keen on the whole "needs env var to run tests", because it ends up testing a very different thing from what ends up happening in user installs. Along the same lines, I removed
https://github.com/apache/arrow/blob/b29b786ed3d770a9d9765ee6b1ed221e887e9093/python/pyarrow/tests/conftest.py#L63-L67
see https://github.com/conda-forge/pyarrow-feedstock/pull/167 etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3777732569
(all these tests haven't really changed for years, btw)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3788531203
Quick update: I backported #48601 to v22 in https://github.com/conda-forge/arrow-cpp-feedstock/pull/1913; that gets us quite a bit further, but there are still some remaining failures:
```
98% tests passed, 2 tests failed out of 102
[...]
The following tests FAILED:
100 - gandiva-internals-test (Failed) gandiva-tests unittest
102 - gandiva-projector-test (Failed) gandiva-tests unittest
```
<details>
### `gandiva-internals-test`
```
[ RUN ] TestGdvFnStubs.TestToUtcTimezone
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1081): error: Expected equality of these values:
1330423200000
ts2
Which is: 0
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1087): error: Expected equality of these values:
ts2
Which is: 0
-1800000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1094): error: Expected equality of these values:
ts2
Which is: 0
1520758800000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1100): error: Expected equality of these values:
ts2
Which is: 0
1520841600000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1104): error: Value of: context.get_error()
Expected: is equal to "America/LA is an invalid time zone name."
Actual: "Asia/Kolkata is an invalid time zone name."
[ FAILED ] TestGdvFnStubs.TestToUtcTimezone (5 ms)
[ RUN ] TestGdvFnStubs.TestFromUtcTimezone
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1118): error: Expected equality of these values:
ts2
Which is: 0
55800000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1124): error: Expected equality of these values:
ts2
Which is: 0
18000000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1130): error: Expected equality of these values:
ts2
Which is: 0
1520730000000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1136): error: Expected equality of these values:
ts2
Which is: 0
1520816400000
%SRC_DIR%\cpp\src\gandiva\gdv_function_stubs_test.cc(1140): error: Value of: context.get_error()
Expected: is equal to "India is an invalid time zone name."
Actual: "Asia/Kolkata is an invalid time zone name."
[ FAILED ] TestGdvFnStubs.TestFromUtcTimezone (0 ms)
[...]
[----------] Global test environment tear-down
[==========] 143 tests from 26 test suites ran. (647 ms total)
[ PASSED ] 141 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] TestGdvFnStubs.TestToUtcTimezone
[ FAILED ] TestGdvFnStubs.TestFromUtcTimezone
2 FAILED TESTS
```
### `gandiva-projector-test`
```
[ RUN ] DateTimeTestProjector.TestToUtcTimestamp
%SRC_DIR%\cpp\src\gandiva\tests\date_time_test.cc(771): error: Value of: status.ok()
Actual: false
Expected: true
unknown file: error: C++ exception with description "invalid vector subscript" thrown in the test body.
[ FAILED ] DateTimeTestProjector.TestToUtcTimestamp (49 ms)
[ RUN ] DateTimeTestProjector.TestFromUtcTimestamp
%SRC_DIR%\cpp\src\gandiva\tests\date_time_test.cc(826): error: Value of: status.ok()
Actual: false
Expected: true
unknown file: error: C++ exception with description "invalid vector subscript" thrown in the test body.
[ FAILED ] DateTimeTestProjector.TestFromUtcTimestamp (46 ms)
[----------] 12 tests from DateTimeTestProjector (1131 ms total)
[...]
[----------] Global test environment tear-down
[==========] 211 tests from 16 test suites ran. (19339 ms total)
[ PASSED ] 209 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] DateTimeTestProjector.TestToUtcTimestamp
[ FAILED ] DateTimeTestProjector.TestFromUtcTimestamp
2 FAILED TESTS
YOU HAVE 2 DISABLED TESTS
```
</details>
Part of that could of course be due to the backport to v22 (which cherry-picked cleanly though, aside from some CI config), but I won't get around fixing this also on v22 one way or another. I was planning to fix CI on v22 first and then try v23. If helpful, I can also try out v23 (with the backport of #48601) as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772831757
Thanks a lot! Conda-forge has the ability to backport patches where necessary. Since `main` had barely any time to move on, and we need this change to fix our builds, I'll definitely consider a backport. That said, looking at the description of #48601, you're IMO going quite a bit further than would be necessary. Not that I'm against using C++20 std facilities per se (though they're still not completely implemented in libc++), but strictly speaking we only need a way to use conda-forge's tzdata.
When I started down this rabbit hole, I had some first trivial changes that looked as follows (before realizing that it's connected pretty deeply with the vendored stuff and probably not so easy to pipe through; from that POV, I can understand going with C++20 😅):
```diff
diff --git a/cpp/src/arrow/vendored/datetime/tz.cpp b/cpp/src/arrow/vendored/datetime/tz.cpp
index 2cf6c62a84..27a64701a4 100644
--- a/cpp/src/arrow/vendored/datetime/tz.cpp
+++ b/cpp/src/arrow/vendored/datetime/tz.cpp
@@ -476,16 +475,19 @@ CONSTDATA auto max_day = date::December/31;
CONSTCD14 const sys_seconds min_seconds = sys_days(min_year/min_day);
-#endif // USE_OS_TZDB
-
-#ifndef _WIN32
-
static
std::string
discover_tz_dir()
{
struct stat sb;
using namespace std;
+ // this is present if we're in an activated conda environment
+ const char* condaPrefix = getenv("CONDA_PREFIX");
+ if (condaPrefix) {
+ string condaDir(condaPrefix);
+ condaDir += "/share/zoneinfo";
+ return condaDir;
+ }
# if defined(ANDROID) || defined(__ANDROID__)
CONSTDATA auto tz_dir_default = "/apex/com.android.tzdata/etc/tz";
CONSTDATA auto tz_dir_fallback = "/system/usr/share/zoneinfo";
diff --git a/cpp/src/arrow/vendored/datetime/tz.h b/cpp/src/arrow/vendored/datetime/tz.h
index 61ab3df106..cd5fa70d29 100644
--- a/cpp/src/arrow/vendored/datetime/tz.h
+++ b/cpp/src/arrow/vendored/datetime/tz.h
@@ -82,12 +82,6 @@ static_assert(HAS_REMOTE_API == 0 ? AUTO_DOWNLOAD == 0 : true,
# define USE_SHELL_API 1
#endif
-#if USE_OS_TZDB
-# ifdef _WIN32
-# error "USE_OS_TZDB can not be used on Windows"
-# endif
-#endif
-
#ifndef HAS_DEDUCTION_GUIDES
# if __cplusplus >= 201703
# define HAS_DEDUCTION_GUIDES 1
diff --git a/cpp/src/arrow/vendored/datetime/visibility.h b/cpp/src/arrow/vendored/datetime/visibility.h
index 780c00d70b..cd182bdf70 100644
--- a/cpp/src/arrow/vendored/datetime/visibility.h
+++ b/cpp/src/arrow/vendored/datetime/visibility.h
@@ -17,9 +17,7 @@
#pragma once
-#ifndef _WIN32
-# define USE_OS_TZDB 1
-#endif
+#define USE_OS_TZDB 1
#if defined(ARROW_STATIC)
// intentially empty
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3776664538
I doubt that this patch is complete or even functional. The tzdb-handling in `cpp/src/arrow/vendored/datetime/tz.cpp` is all over the place, and I opened this issue before pouring hours down this rabbit hole. I can give it a try, but it'll be a long & messy process, and I don't know that code very well or at all...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772992360
If you don't use mingw and have GCC>=13 you should be able to avoid downloading tzdata on windows (I think). Sorry I can't spend time right now, but changes in https://github.com/apache/arrow/pull/48601 would probably be a good starting point. It might make sense for it to merge first and then backport the non-CI part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3783117771
> So perhaps you were downloading/installing the tzdb in AppData before?
It's not impossible that the windows images used to contain a tzdb in AppData due to something else (unrelated to our build process). For the testing we've done, it's only ever been something very close to the following:
```bat
:: cmake config etc.
cmake --build . --config Release
if %ERRORLEVEL% neq 0 exit 1
npm install -g azurite
set ARROW_TEST_DATA=%SRC_DIR%\testing\data
set PARQUET_TEST_DATA=%SRC_DIR%\cpp\submodules\parquet-testing\data
ctest --progress --output-on-failure
if %ERRORLEVEL% neq 0 exit 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3777731693
The environment variable `ARROW_TIMEZONE_DATABASE` should be set to somewhere where the timezone database resides, is that not the case in your test script?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3775061461
> With which Arrow version, and when have they started failing?
Unfortunately, the errors were hidden due to an issue with batchfiles (`%ERRORLEVEL%` doesn't work in an `if`-branch). However, the failures are still visible in the logs; I went back and found that the issue appeared (exactly) with arrow v22; v21 and below are fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3783092083
> We've started running test for libarrow since v18 (though [conda-forge/arrow-cpp-feedstock#1058](https://github.com/conda-forge/arrow-cpp-feedstock/pull/1058) was ~1.5yrs in the making) without ever setting `ARROW_TIMEZONE_DATABASE`, and that ran fine before v22 (and still works fine on unix even for v22).
Well, Unix uses the system tzdb, so that's expected. On Windows, the tzdb is expected either under `ARROW_TIMEZONE_DATABASE`, or in the user's AppData directory. So perhaps you were downloading/installing the tzdb in AppData before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "shashbha14 (via GitHub)" <gi...@apache.org>.
shashbha14 commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3758696603
I would like to work on this issue.Please assign this to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3777572010
Unfortunately not; everything tzdb-related fails
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772779409
We are working towards using stdlib provided tzdb on windows [here](https://github.com/apache/arrow/pull/48601). However we missed the 23.0.0 window and the change is too big to tack on now. I don't have much else to add.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772852020
@h-vetinari very much agreed! And a much more minimal patch can be cut out of the https://github.com/apache/arrow/pull/48601. Which compilers are you supporting?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772840702
There's one thing that a more minimal patch would be very useful for though: backporting, as we're still supporting v19-v22 (which are running into the same failures currently).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "shashbha14 (via GitHub)" <gi...@apache.org>.
shashbha14 commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3773607814
Hi, I've analyzed this issue and found it requires significant changes to the vendored datetime library. I see @rok is working on PR #48601 with a more comprehensive solution.
Should I:
a) Wait for #48601 to be merged and help with backports?
b) Try to implement a minimal patch based on @h-vetinari's suggestions?
Please advise on the best approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3772895180
> Which compilers are you supporting?
Our default compilers are currently GCC 14 on linux, Clang 19 on osx (will move to GCC 15 & Clang 21 in ~May/June) and VS 2022 on windows (where we have no control over the exact version, it's whatever comes along in https://github.com/actions/runner-images). Both clang and GCC are also usable on the other platforms if need be.
Note also that we do hook up [libc++](https://github.com/conda-forge/libcxx-feedstock/blob/main/recipe/patches/0005-patch-__libcpp_tzdb_directory-to-allow-use-on-osx.patch) and [libstdcxx](https://github.com/conda-forge/ctng-compilers-feedstock/blob/main/recipe/patches/0002-patch-zoneinfo_dir_override-to-point-to-our-tzdata.patch) to use our own tzdata. Even the partially implemented `std::chrono` in libc++ works according to some [smoke tests](https://github.com/conda-forge/libcxx-feedstock/blob/main/recipe/test_sources/tzdb/tzdb.cpp), with the exception of leap seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3768077334
@h-vetinari I suppose this is a blocker for Arrow 23.0.0 on conda right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3758796720
Done.
FYI: You can do it by adding "take" only comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3771631517
cc @rok in case you have some time to look into this issue and help with suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3773675165
I am not an expert here but the PR @rok is pointing out will be complex to backport to pre 23.0.0 conda feedstocks. That PR is also under current development and is not clear when will be finished. In my opinion we should try and work on a minimal patch for the feedstocks to work.
From my understanding those timezones where removed for the official tzdata but on some operating systems are still part of the tzdata-legacy packages. Is this the problem?
cc @pitrou who might have ideas about possible solutions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Enable use of USE_OS_TZDB with conda, also on windows [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48874:
URL: https://github.com/apache/arrow/issues/48874#issuecomment-3776721776
Is `Australia/Broken_Hill` the only timezone that's not found? If so, you could just disable the corresponding tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org