You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/08/26 20:38:44 UTC
[GitHub] [arrow] bkietz opened a new pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
bkietz opened a new pull request #11013:
URL: https://github.com/apache/arrow/pull/11013
Allows declaring blocks of code to be run at library load time.
For now this is a draft, I'm checking to see if CI accepts this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697539168
##########
File path: cpp/src/arrow/compute/exec/source_node.cc
##########
@@ -142,14 +143,10 @@ struct SourceNode : ExecNode {
AsyncGenerator<util::optional<ExecBatch>> generator_;
};
-} // namespace
-
-namespace internal {
+ARROW_INITIALIZER({
+ DCHECK_OK(default_exec_factory_registry()->AddFactory("source", SourceNode::Make));
Review comment:
that function only relies on a static local variable within the function body, which is initialized the [first time control passes through the decl](https://en.cppreference.com/w/cpp/language/storage_duration#Static_local_variables)- which means it will be fully initialized by the time `default_exec_factory_registry()` returns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#issuecomment-906820471
Revision: 4bb6d51654e41dd263a2cf209077799323fbac78
Submitted crossbow builds: [ursacomputing/crossbow @ actions-797](https://github.com/ursacomputing/crossbow/branches/all?query=actions-797)
|Task|Status|
|----|------|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-797-github-homebrew-r-autobrew)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r696976143
##########
File path: cpp/src/arrow/util/init.h
##########
@@ -0,0 +1,53 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "arrow/util/macros.h"
+
+namespace arrow {
+namespace internal {
+
+#define ARROW_INITIALIZER_NAME(counter) ARROW_CONCAT(arrow_initializer_, counter)
+
+/// Declares a block of code to be executed on load of the library.
+/// The init block can only fail by aborting, so use this with caution.
+///
+/// ARROW_INITIALIZER({
+/// DCHECK_OK(registry->Add(Thing::Make()));
+/// });
+#define ARROW_INITIALIZER(...) \
+ ARROW_INITIALIZER_IMPL(ARROW_INITIALIZER_NAME(__COUNTER__), __VA_ARGS__)
+
+#if !defined(_MSC_VER)
+
+// __attribute__((constructor)) is supported by GCC and Clang, and
+// declares that a function must be executed as part of library initialization.
+#define ARROW_INITIALIZER_IMPL(NAME, ...) \
+ __attribute__((constructor)) void NAME() __VA_ARGS__
+
+#else
+
+// MSVC has no equivalent of __attribute__((constructor)), so instead
+// specify an object whose constructor executes the required code.
+#define ARROW_INITIALIZER_IMPL(NAME, ...) \
+ __declspec(dllexport) class NAME { NAME() __VA_ARGS__ } NAME
Review comment:
```suggestion
__declspec(dllexport) struct NAME { NAME() __VA_ARGS__ } NAME
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#issuecomment-906726956
https://issues.apache.org/jira/browse/ARROW-13773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697541645
##########
File path: cpp/src/arrow/util/init.h
##########
@@ -0,0 +1,53 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "arrow/util/macros.h"
+
+namespace arrow {
+namespace internal {
+
+#define ARROW_INITIALIZER_NAME(counter) ARROW_CONCAT(arrow_initializer_, counter)
+
+/// Declares a block of code to be executed on load of the library.
+/// The init block can only fail by aborting, so use this with caution.
+///
+/// ARROW_INITIALIZER({
+/// DCHECK_OK(registry->Add(Thing::Make()));
+/// });
+#define ARROW_INITIALIZER(...) \
+ ARROW_INITIALIZER_IMPL(ARROW_INITIALIZER_NAME(__COUNTER__), __VA_ARGS__)
+
+#if !defined(_MSC_VER)
Review comment:
If you prefer, sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#issuecomment-907087344
Revision: 4bb6d51654e41dd263a2cf209077799323fbac78
Submitted crossbow builds: [ursacomputing/crossbow @ actions-798](https://github.com/ursacomputing/crossbow/branches/all?query=actions-798)
|Task|Status|
|----|------|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-conda-cpp)|
|test-conda-cpp-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-798-azure-test-conda-cpp-valgrind)|
|test-debian-10-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-debian-10-cpp)|
|test-fedora-33-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-fedora-33-cpp)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-798-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697315295
##########
File path: cpp/src/arrow/util/utf8.cc
##########
@@ -84,11 +85,9 @@ ARROW_EXPORT void CheckUTF8Initialized() {
} // namespace internal
-static std::once_flag utf8_initialized;
+void InitializeUTF8() {}
-void InitializeUTF8() {
- std::call_once(utf8_initialized, internal::InitializeLargeTable);
-}
+ARROW_INITIALIZER({ internal::InitializeLargeTable(); });
Review comment:
Sounds good.
As a sidenote, I wonder if the large table can be initialized at compile-time by using a `constexpr` initialization function. Perhaps C++11 is not potent enough?
```c++
constexpr const std::array<uint8_t> utf8_small_table = { ... };
static constexpr std::array<uint16_t, 9 * 256> InitializeLargeTable() = {
std::array<uint16_t, 9 * 256> utf8_large_table;
for (uint32_t state = 0; state < 9; ++state) {
for (uint32_t byte = 0; byte < 256; ++byte) {
uint32_t byte_class = utf8_small_table[byte];
uint8_t next_state = utf8_small_table[256 + state * 12 + byte_class] / 12;
DCHECK_LT(next_state, 9);
utf8_large_table[state * 256 + byte] = static_cast<uint16_t>(next_state * 256);
}
}
return utf8_large_table;
}
constexpr const auto utf8_large_table = InitializeLargeTable();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
bkietz commented on pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#issuecomment-906820230
@github-actions crossbow submit homebrew-r-autobrew
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697310986
##########
File path: cpp/src/arrow/util/init.h
##########
@@ -0,0 +1,53 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "arrow/util/macros.h"
+
+namespace arrow {
+namespace internal {
+
+#define ARROW_INITIALIZER_NAME(counter) ARROW_CONCAT(arrow_initializer_, counter)
+
+/// Declares a block of code to be executed on load of the library.
+/// The init block can only fail by aborting, so use this with caution.
+///
+/// ARROW_INITIALIZER({
+/// DCHECK_OK(registry->Add(Thing::Make()));
+/// });
+#define ARROW_INITIALIZER(...) \
+ ARROW_INITIALIZER_IMPL(ARROW_INITIALIZER_NAME(__COUNTER__), __VA_ARGS__)
+
+#if !defined(_MSC_VER)
Review comment:
Should we do the reverse and condition this block on `__GNUC__` or `__clang__`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#issuecomment-907086581
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697318152
##########
File path: cpp/src/arrow/util/init.h
##########
@@ -0,0 +1,53 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "arrow/util/macros.h"
+
+namespace arrow {
+namespace internal {
+
+#define ARROW_INITIALIZER_NAME(counter) ARROW_CONCAT(arrow_initializer_, counter)
+
+/// Declares a block of code to be executed on load of the library.
+/// The init block can only fail by aborting, so use this with caution.
+///
+/// ARROW_INITIALIZER({
+/// DCHECK_OK(registry->Add(Thing::Make()));
+/// });
+#define ARROW_INITIALIZER(...) \
+ ARROW_INITIALIZER_IMPL(ARROW_INITIALIZER_NAME(__COUNTER__), __VA_ARGS__)
+
+#if !defined(_MSC_VER)
Review comment:
Ah, but `_declspec(dllexport)` would only work on MSVC. Perhaps:
```c++
#if defined(__GNUC__) || defined(__clang__)
#define ARROW_INITIALIZER_IMPL(NAME, ...) \
__attribute__((constructor)) void NAME() __VA_ARGS__
#elif defined(_MSC_VER)
#define ARROW_INITIALIZER_IMPL(NAME, ...) \
__declspec(dllexport) struct NAME { NAME() __VA_ARGS__ } NAME
#else
volatile struct NAME { NAME() __VA_ARGS__ } NAME;
#endif
```
(not sure the `volatile` may help avoid elision?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697312740
##########
File path: cpp/src/arrow/compute/exec/source_node.cc
##########
@@ -142,14 +143,10 @@ struct SourceNode : ExecNode {
AsyncGenerator<util::optional<ExecBatch>> generator_;
};
-} // namespace
-
-namespace internal {
+ARROW_INITIALIZER({
+ DCHECK_OK(default_exec_factory_registry()->AddFactory("source", SourceNode::Make));
Review comment:
Are we sure that everything used by `default_exec_factory_registry()` is initialized at this point? Doing non-trivial initialization at library load time is always a bit tricky because of initialization order issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #11013: ARROW-13773: [C++] Cross platform initializer blocks
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #11013:
URL: https://github.com/apache/arrow/pull/11013#discussion_r697541321
##########
File path: cpp/src/arrow/util/utf8.cc
##########
@@ -84,11 +85,9 @@ ARROW_EXPORT void CheckUTF8Initialized() {
} // namespace internal
-static std::once_flag utf8_initialized;
+void InitializeUTF8() {}
-void InitializeUTF8() {
- std::call_once(utf8_initialized, internal::InitializeLargeTable);
-}
+ARROW_INITIALIZER({ internal::InitializeLargeTable(); });
Review comment:
I think it's possible. The most annoying part here IMO is that `std::array` is not constexpr constructible under c++11 so we'd need to use a raw array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org