You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "David Li (Jira)" <ji...@apache.org> on 2022/07/08 19:26:00 UTC
[jira] [Resolved] (ARROW-17014) [Python][C++] Appveyor cython tests are failing
[ https://issues.apache.org/jira/browse/ARROW-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Li resolved ARROW-17014.
------------------------------
Resolution: Fixed
Issue resolved by pull request 13549
[https://github.com/apache/arrow/pull/13549]
> [Python][C++] Appveyor cython tests are failing
> -----------------------------------------------
>
> Key: ARROW-17014
> URL: https://issues.apache.org/jira/browse/ARROW-17014
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Python
> Reporter: Raúl Cumplido
> Assignee: Raúl Cumplido
> Priority: Critical
> Labels: pull-request-available
> Fix For: 9.0.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Two cython tests are failing on Appveyor builds ([https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/44114739/job/rresg43jos1wpbtv):]
> {code:java}
> _______________________________ test_cython_api _______________________________
> tmpdir = local('C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pytest-of-appveyor\\pytest-0\\test_cython_api0')
> @pytest.mark.cython
> def test_cython_api(tmpdir):
> """
> Basic test for the Cython API.
> """
> # Fail early if cython is not found
> import cython # noqa
>
> with tmpdir.as_cwd():
> # Set up temporary workspace
> pyx_file = 'pyarrow_cython_example.pyx'
> shutil.copyfile(os.path.join(here, pyx_file),
> os.path.join(str(tmpdir), pyx_file))
> # Create setup.py file
> setup_code = setup_template.format(pyx_file=pyx_file,
> compiler_opts=compiler_opts,
> test_ld_path=test_ld_path)
> with open('setup.py', 'w') as f:
> f.write(setup_code)
>
> # ARROW-2263: Make environment with this pyarrow/ package first on the
> # PYTHONPATH, for local dev environments
> subprocess_env = test_util.get_modified_env_with_pythonpath()
>
> # Compile extension module
> > subprocess.check_call([sys.executable, 'setup.py',
> 'build_ext', '--inplace'],
> env=subprocess_env)
> pyarrow\tests\test_cython.py:107:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> popenargs = (['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace'],)
> kwargs = {'env': {'7ZIP': '"C:\\Program Files\\7-Zip\\7z.exe"', 'ALLUSERSPROFILE': 'C:\\ProgramData', 'APPDATA': 'C:\\Users\\appveyor\\AppData\\Roaming', 'APPVEYOR': 'True', ...}}
> retcode = 1
> cmd = ['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace']
> def check_call(*popenargs, **kwargs):
> """Run command with arguments. Wait for command to complete. If
> the exit code was zero then return, otherwise raise
> CalledProcessError. The CalledProcessError object will have the
> return code in the returncode attribute.
>
> The arguments are the same as for the call function. Example:
>
> check_call(["ls", "-l"])
> """
> retcode = call(*popenargs, **kwargs)
> if retcode:
> cmd = kwargs.get("args")
> if cmd is None:
> cmd = popenargs[0]
> > raise CalledProcessError(retcode, cmd)
> E subprocess.CalledProcessError: Command '['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace']' returned non-zero exit status 1.
> C:\Miniconda37-x64\envs\arrow\lib\subprocess.py:364: CalledProcessError
> ---------------------------- Captured stdout call -----------------------------
> Compiling pyarrow_cython_example.pyx because it changed.
> [1/1] Cythonizing pyarrow_cython_example.pyx
> Extension module: <setuptools.extension.Extension('pyarrow_cython_example') at 0x2804221ae10> ['C:\\Miniconda37-x64\\envs\\arrow\\lib\\site-packages\\numpy\\core\\include', 'c:\\projects\\arrow\\python\\pyarrow\\include'] ['arrow', 'arrow_python'] ['c:\\projects\\arrow\\python\\pyarrow', 'C:/Miniconda37-x64/envs/arrow/Library/lib', 'C:\\Miniconda37-x64\\envs\\arrow\\Library\\lib']
> running build_ext
> building 'pyarrow_cython_example' extension
> creating build
> creating build\temp.win-amd64-cpython-38
> creating build\temp.win-amd64-cpython-38\Release
> "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Miniconda37-x64\envs\arrow\lib\site-packages\numpy\core\include -Ic:\projects\arrow\python\pyarrow\include -IC:\Miniconda37-x64\envs\arrow\include -IC:\Miniconda37-x64\envs\arrow\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /EHsc /Tppyarrow_cython_example.cpp /Fobuild\temp.win-amd64-cpython-38\Release\pyarrow_cython_example.obj
> pyarrow_cython_example.cpp
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1271): error C2338: You've instantiated std::aligned_storage<Len, Align> with an extended alignment (in other words, Align > alignof(max_align_t)). Before VS 2017 15.8, the member type would non-conformingly have an alignment of only alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the fix inherently changes layout and breaks binary compatibility (*only* for uses of aligned_storage with extended alignments). Please define either (1) _ENABLE_EXTENDED_ALIGNED_STORAGE to acknowledge that you understand this message and that you actually want a type with an extended alignment, or (2) _DISABLE_EXTENDED_ALIGNED_STORAGE to silence this message and get the old non-conformant behavior.
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1291): note: see reference to class template instantiation 'std::_Aligned<192,64,double,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1298): note: see reference to class template instantiation 'std::_Aligned<192,64,int,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1305): note: see reference to class template instantiation 'std::_Aligned<192,64,short,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1312): note: see reference to class template instantiation 'std::_Aligned<192,64,char,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1320): note: see reference to class template instantiation 'std::aligned_storage<192,64>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1352): note: see reference to alias template instantiation 'std::aligned_storage_t<192,64>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(132): note: see reference to class template instantiation 'std::aligned_union<0,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(140): note: see reference to class template instantiation 'arrow::util::detail::VariantStorage<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(245): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/compute/exec.h(313): note: see reference to class template instantiation 'arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/vendored/datetime/date.h(182): note: see reference to class template instantiation 'std::chrono::duration<int,std::ratio<86400,1>>' being compiled
> ---------------------------- Captured stderr call -----------------------------
> error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
> _____________________________ test_visit_strings ______________________________
> tmpdir = local('C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pytest-of-appveyor\\pytest-0\\test_visit_strings0')
> @pytest.mark.cython
> def test_visit_strings(tmpdir):
> with tmpdir.as_cwd():
> # Set up temporary workspace
> pyx_file = 'bound_function_visit_strings.pyx'
> shutil.copyfile(os.path.join(here, pyx_file),
> os.path.join(str(tmpdir), pyx_file))
> # Create setup.py file
> setup_code = setup_template.format(pyx_file=pyx_file,
> compiler_opts=compiler_opts,
> test_ld_path=test_ld_path)
> with open('setup.py', 'w') as f:
> f.write(setup_code)
>
> subprocess_env = test_util.get_modified_env_with_pythonpath()
>
> # Compile extension module
> > subprocess.check_call([sys.executable, 'setup.py',
> 'build_ext', '--inplace'],
> env=subprocess_env)
> pyarrow\tests\test_cython.py:162:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> popenargs = (['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace'],)
> kwargs = {'env': {'7ZIP': '"C:\\Program Files\\7-Zip\\7z.exe"', 'ALLUSERSPROFILE': 'C:\\ProgramData', 'APPDATA': 'C:\\Users\\appveyor\\AppData\\Roaming', 'APPVEYOR': 'True', ...}}
> retcode = 1
> cmd = ['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace']
> def check_call(*popenargs, **kwargs):
> """Run command with arguments. Wait for command to complete. If
> the exit code was zero then return, otherwise raise
> CalledProcessError. The CalledProcessError object will have the
> return code in the returncode attribute.
>
> The arguments are the same as for the call function. Example:
>
> check_call(["ls", "-l"])
> """
> retcode = call(*popenargs, **kwargs)
> if retcode:
> cmd = kwargs.get("args")
> if cmd is None:
> cmd = popenargs[0]
> > raise CalledProcessError(retcode, cmd)
> E subprocess.CalledProcessError: Command '['C:\\Miniconda37-x64\\envs\\arrow\\python.exe', 'setup.py', 'build_ext', '--inplace']' returned non-zero exit status 1.
> C:\Miniconda37-x64\envs\arrow\lib\subprocess.py:364: CalledProcessError
> ---------------------------- Captured stdout call -----------------------------
> Compiling bound_function_visit_strings.pyx because it changed.
> [1/1] Cythonizing bound_function_visit_strings.pyx
> Extension module: <setuptools.extension.Extension('bound_function_visit_strings') at 0x28ba9d0df00> ['C:\\Miniconda37-x64\\envs\\arrow\\lib\\site-packages\\numpy\\core\\include', 'c:\\projects\\arrow\\python\\pyarrow\\include'] ['arrow', 'arrow_python'] ['c:\\projects\\arrow\\python\\pyarrow', 'C:/Miniconda37-x64/envs/arrow/Library/lib', 'C:\\Miniconda37-x64\\envs\\arrow\\Library\\lib']
> running build_ext
> building 'bound_function_visit_strings' extension
> creating build
> creating build\temp.win-amd64-cpython-38
> creating build\temp.win-amd64-cpython-38\Release
> "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Miniconda37-x64\envs\arrow\lib\site-packages\numpy\core\include -Ic:\projects\arrow\python\pyarrow\include -IC:\Miniconda37-x64\envs\arrow\include -IC:\Miniconda37-x64\envs\arrow\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /EHsc /Tpbound_function_visit_strings.cpp /Fobuild\temp.win-amd64-cpython-38\Release\bound_function_visit_strings.obj
> bound_function_visit_strings.cpp
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1271): error C2338: You've instantiated std::aligned_storage<Len, Align> with an extended alignment (in other words, Align > alignof(max_align_t)). Before VS 2017 15.8, the member type would non-conformingly have an alignment of only alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the fix inherently changes layout and breaks binary compatibility (*only* for uses of aligned_storage with extended alignments). Please define either (1) _ENABLE_EXTENDED_ALIGNED_STORAGE to acknowledge that you understand this message and that you actually want a type with an extended alignment, or (2) _DISABLE_EXTENDED_ALIGNED_STORAGE to silence this message and get the old non-conformant behavior.
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1291): note: see reference to class template instantiation 'std::_Aligned<192,64,double,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1298): note: see reference to class template instantiation 'std::_Aligned<192,64,int,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1305): note: see reference to class template instantiation 'std::_Aligned<192,64,short,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1312): note: see reference to class template instantiation 'std::_Aligned<192,64,char,false>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1320): note: see reference to class template instantiation 'std::aligned_storage<192,64>' being compiled
> C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\type_traits(1352): note: see reference to alias template instantiation 'std::aligned_storage_t<192,64>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(132): note: see reference to class template instantiation 'std::aligned_union<0,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(140): note: see reference to class template instantiation 'arrow::util::detail::VariantStorage<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(157): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/util/variant.h(245): note: see reference to class template instantiation 'arrow::util::detail::VariantImpl<arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>,arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/compute/exec.h(313): note: see reference to class template instantiation 'arrow::util::Variant<arrow::ArraySpan,std::shared_ptr<arrow::ArrayData>>' being compiled
> c:\projects\arrow\python\pyarrow\include\arrow/vendored/datetime/date.h(182): note: see reference to class template instantiation 'std::chrono::duration<int,std::ratio<86400,1>>' being compiled
> ---------------------------- Captured stderr call -----------------------------
> error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023{code}
> It seems to have been failing on master since this was merged [https://github.com/apache/arrow/commit/6cc37cf2d1ba72c46b64fbc7ac499bd0d7296d20]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)