You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "NEIL-smtg (via GitHub)" <gi...@apache.org> on 2024/12/02 09:14:21 UTC

[I] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

NEIL-smtg opened a new issue, #44896:
URL: https://github.com/apache/arrow/issues/44896

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   Recently, I encountered a few errors while building Arrow with the MSVC arm64 compiler:
   ```
   C:\gitP\apache\arrow\cpp\src\arrow\util\utf8_internal.h(242,29): error C2039: 'make_sized_batch_t': is not a member of 'xsimd' [C:\gitP\apache\arrow\cpp\build_arm64\src\arrow\arrow_util_static.vcxproj]
            (compiling source file '../../../src/arrow/util/utf8.cc')
                C:\gitP\apache\arrow\cpp\build_arm64\xsimd_ep\src\xsimd_ep-install\include\xsimd\memory\xsimd_aligned_allocator.hpp(29,11):
                see declaration of 'xsimd'
   C:\gitP\apache\arrow\cpp\src\arrow\util\utf8_internal.h(242,29): error C2061: syntax error: identifier 'make_sized_batch_t' [C:\gitP\apache\arrow\cpp\build_arm64\src\arrow\arrow_util_static.vcxproj]
            (compiling source file '../../../src/arrow/util/utf8.cc')
   ...
   ```
   
   This issue appears to occur only when using the arm64 compiler, as the code compiles without issues with arm64ec.
   
   Build log: [arr.txt](https://github.com/user-attachments/files/17975510/arr.txt)
   
   Repro steps:
   
   1. Open x64 Native tools command prompt for VS 2022
   2. set VSCMD_SKIP_SENDTELEMETRY=1 & "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat" -host_arch=amd64 -arch=arm64
   3. git clone https://github.com/apache/arrow.git
   4. mkdir apache\arrow\cpp\build_arm64 & cd apache\arrow\cpp\build_arm64
   5. set _CL_=/DUSE_SOFT_INTRINSICS %_CL_%
   6. cmake -G "Visual Studio 17 2022" -A arm64 -DCMAKE_SYSTEM_VERSION=10.0.22621.0  -DARROW_BUILD_TESTS=OFF .. 2>&1
   7. msbuild /m /p:Platform=arm64 /p:Configuration=Release arrow.sln /t:Rebuild 2>&1
   
   As a workaround, I have been predefining the following flags before building the project: `/D _M_AMD64  /DUSE_SOFT_INTRINSICS /D__INTRIN_H_`
   
   ### Component(s)
   
   C++


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2511533098

   Thanks for the report!
   
   This seems more like an xsimd issue than an arrow issue as it happens in the xsimd external project: `C:\gitP\apache\arrow\cpp\build_arm64\xsimd_ep\src\xsimd_ep-install\include\xsimd\memory\xsimd_aligned_allocator.hpp(29,11): see declaration of 'xsimd'`
   
   But good to hear that you have a workaround!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "NEIL-smtg (via GitHub)" <gi...@apache.org>.
NEIL-smtg commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2513451600

   Hi guys, I have tried creating a toolchain file and setting these 2 variables, but I'm still encountering the same issue. 
   ```
   set(CMAKE_SYSTEM_NAME Windows)
   set(CMAKE_SYSTEM_PROCESSOR arm64)
   ```
   
   Build log: [arr.txt](https://github.com/user-attachments/files/17986747/arr.txt)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2511534431

   Cc @pitrou 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2513597646

   We can't use MSVC for Windows/ARM64 yet...: https://arrow.apache.org/docs/developers/cpp/windows.html#building-on-windows-arm64-using-ninja-and-clang


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2512885444

   Right.
   You need to set some variables manually for cross-compiling.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2511733184

   But I think you simply are not setting up cross-compilation correctly:
   ```
   -- System processor: AMD64
   ```
   
   According to https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PROCESSOR.html :
   > When cross-compiling, a [CMAKE_TOOLCHAIN_FILE](https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html#variable:CMAKE_TOOLCHAIN_FILE) should set the CMAKE_SYSTEM_PROCESSOR variable to match target architecture that it specifies (via [CMAKE_<LANG>_COMPILER](https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER.html#variable:CMAKE_%3CLANG%3E_COMPILER) and perhaps [CMAKE_<LANG>_COMPILER_TARGET](https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_TARGET.html#variable:CMAKE_%3CLANG%3E_COMPILER_TARGET)).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "NEIL-smtg (via GitHub)" <gi...@apache.org>.
NEIL-smtg closed issue #44896: [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler
URL: https://github.com/apache/arrow/issues/44896


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "NEIL-smtg (via GitHub)" <gi...@apache.org>.
NEIL-smtg commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2513610936

   Ah I see, I'll go ahead to close this issue then.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2511726307

   This looks like a cross-compilation problem. SSE4.2 shouldn't be selected on a Arm64 target:
   ```
   --   ARROW_SIMD_LEVEL=SSE4_2 [default=DEFAULT|NONE|SSE4_2|AVX2|AVX512|NEON|SVE|SVE128|SVE256|SVE512]
   --       Compile-time SIMD optimization level
   ```
   
   cc @cyb70289 @kou 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] Compile error in arrow\cpp\src\arrow\util\utf8_internal.h with MSVC arm64 compiler [arrow]

Posted by "cyb70289 (via GitHub)" <gi...@apache.org>.
cyb70289 commented on issue #44896:
URL: https://github.com/apache/arrow/issues/44896#issuecomment-2513462924

   From below log, looks `-march=armv8-a` is appended to compiler flags, even for msvc.
   Guess we need a different option for msvc cross build, [/arch:armv8.0](https://learn.microsoft.com/en-us/cpp/build/reference/arch-arm64?view=msvc-170)?
   ```
   -- CMAKE_C_FLAGS: /DWIN32 /D_WINDOWS  /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4800 /wd4996 /wd4065 -march=armv8-a 
   -- CMAKE_CXX_FLAGS:  /DWIN32 /D_WINDOWS /GR /EHsc /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING  /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4800 /wd4996 /wd4065 -march=armv8-a 
   ```
   
   Related code at:
   https://github.com/apache/arrow/blob/main/cpp/cmake_modules/SetupCxxFlags.cmake#L125-L126
   https://github.com/apache/arrow/blob/main/cpp/cmake_modules/SetupCxxFlags.cmake#L543


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org