You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by pe...@apache.org on 2021/06/25 04:56:09 UTC
[pulsar] 12/13: [C++] Fix Windows 32 bits compile and runtime
failures (#11082)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 68889ef0f5095cf407ae7d36bcc253e64b2dcfda
Author: Yunze Xu <xy...@163.com>
AuthorDate: Fri Jun 25 08:59:13 2021 +0800
[C++] Fix Windows 32 bits compile and runtime failures (#11082)
### Motivation
C++ source code cannot be compiled successfully for Windows 32-bit build. The compile error is:
> fatal error C1021: invalid preprocessor command 'warning'
It's because `#warning` preprocessor directive is not supported by MSVC compilers.
And even if the related code was remove and the build succeeded, a example producer program would crash. This bug is introduced from https://github.com/apache/pulsar/pull/6129. It's because hardware CRC32 implementation on Windows is only provided for 64-bit programs. However, the `__cpuid` check in `crc32_initialize()` still returns true for 32-bit build, so finally it went here:
```c++
uint32_t crc32c(uint32_t init, const void *buf, size_t len, const chunk_config *config) {
// SSE 4.2 extension for hw implementation are not present
abort();
}
```
The `abort()` call will terminate the process.
### Modifications
- Use `#pragma message()` to replace `#warning` if the compiler is MSVC.
- Fallback to software implementation of CRC32 checksum algorithm if the hardware implementation is not supported.
- Add the workflow to build Windows 32-bit C++ library.
- Fix the wrong document about how to build it for Windows 32-bit.
(cherry picked from commit c8fe1e89469228a59967b8716cada0fcf63315e1)
---
.github/workflows/ci-cpp-build-windows.yaml | 7 +++++++
pulsar-client-cpp/README.md | 8 +++-----
pulsar-client-cpp/lib/CMakeLists.txt | 2 +-
pulsar-client-cpp/lib/checksum/crc32c_sse42.cc | 12 +++++++++++-
4 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/ci-cpp-build-windows.yaml b/.github/workflows/ci-cpp-build-windows.yaml
index 6384968..c524fdc 100644
--- a/.github/workflows/ci-cpp-build-windows.yaml
+++ b/.github/workflows/ci-cpp-build-windows.yaml
@@ -52,6 +52,13 @@ jobs:
suffix: 'windows-win64'
generator: 'Visual Studio 16 2019'
arch: '-A x64'
+ - name: 'Windows x86'
+ os: windows-latest
+ triplet: x86-windows
+ vcpkg_dir: 'C:\vcpkg'
+ suffix: 'windows-win32'
+ generator: 'Visual Studio 16 2019'
+ arch: '-A Win32'
steps:
- name: checkout
diff --git a/pulsar-client-cpp/README.md b/pulsar-client-cpp/README.md
index f489c3d..61f6dff 100644
--- a/pulsar-client-cpp/README.md
+++ b/pulsar-client-cpp/README.md
@@ -189,13 +189,13 @@ ${PULSAR_PATH}/pulsar-client-cpp/perf/perfConsumer
It's highly recommended to use `vcpkg` for C++ package management on Windows. It's easy to install and well supported by Visual Studio (2015/2017/2019) and CMake. See [here](https://github.com/microsoft/vcpkg#quick-start-windows) for quick start.
-Take 64 bits Windows as an example, you only need to run
+Take Windows 64-bit library as an example, you only need to run
```bash
vcpkg install --feature-flags=manifests --triplet x64-windows
```
-> NOTE: The default triplet is `x86-windows`, see [here](https://github.com/microsoft/vcpkg/blob/master/docs/users/triplets.md) for more details.
+> NOTE: For Windows 32-bit library, change `x64-windows` to `x86-windows`, see [here](https://github.com/microsoft/vcpkg/blob/master/docs/users/triplets.md) for more details about the triplet concept in Vcpkg.
The all dependencies, which are specified by [vcpkg.json](vcpkg.json), will be installed in `vcpkg_installed/` subdirectory,
@@ -214,10 +214,8 @@ cmake --build ./build --config Release
Then all artifacts will be built into `build` subdirectory.
-> NOTE:
+> NOTE: For Windows 32-bit, you need to use `-A Win32` and `-DVCPKG_TRIPLET=x86-windows`.
>
-> 1. Change `Release` to `Debug` if you want to build a debug version library.
-> 2. For 32 bits Windows, you need to use `-A Win32` and `-DVCPKG_TRIPLET=x32-windows`.
#### Install dependencies manually
diff --git a/pulsar-client-cpp/lib/CMakeLists.txt b/pulsar-client-cpp/lib/CMakeLists.txt
index 2e540b2..96d7283 100644
--- a/pulsar-client-cpp/lib/CMakeLists.txt
+++ b/pulsar-client-cpp/lib/CMakeLists.txt
@@ -150,7 +150,7 @@ elseif(BUILD_STATIC_LIB)
# Install regular libpulsar.a
target_link_libraries(pulsarStatic ${COMMON_LIBS})
install(TARGETS pulsarStatic DESTINATION lib)
-endif(LINK_STATIC)
+endif()
if (BUILD_STATIC_LIB)
install(TARGETS pulsarStatic DESTINATION lib)
diff --git a/pulsar-client-cpp/lib/checksum/crc32c_sse42.cc b/pulsar-client-cpp/lib/checksum/crc32c_sse42.cc
index 89349a7..5126ec9 100644
--- a/pulsar-client-cpp/lib/checksum/crc32c_sse42.cc
+++ b/pulsar-client-cpp/lib/checksum/crc32c_sse42.cc
@@ -19,18 +19,27 @@
#if BOOST_VERSION >= 105500
#include <boost/predef.h>
#else
+#if _MSC_VER
+#pragma message("Boost version is < 1.55, disable CRC32C")
+#else
#warning "Boost version is < 1.55, disable CRC32C"
#endif
+#endif
#include <assert.h>
#include <stdlib.h>
+#include "lib/checksum/crc32c_sw.h"
#if BOOST_ARCH_X86_64
#include <nmmintrin.h> // SSE4.2
#include <wmmintrin.h> // PCLMUL
#else
+#ifdef _MSC_VER
+#pragma message("BOOST_ARCH_X86_64 is not defined, CRC32C will be disabled")
+#else
#warning "BOOST_ARCH_X86_64 is not defined, CRC32C will be disabled"
#endif
+#endif
#ifdef _MSC_VER
#include <intrin.h>
@@ -82,6 +91,7 @@ bool crc32c_initialize() {
DEBUG_PRINTF1("has_pclmulqdq = %d\n", has_pclmulqdq);
initialized = true;
}
+
return has_sse42;
}
@@ -251,7 +261,7 @@ uint32_t crc32c(uint32_t init, const void *buf, size_t len, const chunk_config *
uint32_t crc32c(uint32_t init, const void *buf, size_t len, const chunk_config *config) {
// SSE 4.2 extension for hw implementation are not present
- abort();
+ return crc32c_sw(init, buf, len); // fallback to the software implementation
}
#endif