You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/05/27 03:38:50 UTC
[GitHub] [arrow] raulbocanegra opened a new pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
raulbocanegra opened a new pull request #7284:
URL: https://github.com/apache/arrow/pull/7284
Fix linking against Python in debug mode on Windows
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-644241777
@raulbocanegra ping
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-646182178
Hi!
First of all, sorry for the delay guys. I tried the `-DPYTHON_EXECUTABLE=python_d` but it then raise an error related to some numpy library.
I am really busy this week. Please, give me a few days more to give you some more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#discussion_r431438314
##########
File path: cpp/cmake_modules/DefineOptions.cmake
##########
@@ -120,6 +120,8 @@ if("${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
define_option(ARROW_GGDB_DEBUG "Pass -ggdb flag to debug builds" ON)
+ define_option(ARROW_USE_DEPRECATED_FIND_PYTHONLIBS "Force using deprecated FindPythonLibsNew script" OFF)
Review comment:
Could you remove this?
This isn't used for forcing using deprecated `FindPythonLibsNew`.
##########
File path: cpp/src/arrow/python/CMakeLists.txt
##########
@@ -60,7 +60,11 @@ endif()
set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
if(WIN32)
- list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+ if(ARROW_USE_DEPRECATED_FIND_PYTHONLIBS)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS $<$<CONFIG:Debug>:${PYTHON_DEBUG_LIBRARIES}> $<$<CONFIG:Release>:${PYTHON_LIBRARIES}>)
+ else()
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_OTHER_LIBS})
Review comment:
`${PYTHON_OTHER_LIBS}` should be always used.
##########
File path: cpp/cmake_modules/FindPythonLibsNew.cmake
##########
@@ -246,7 +248,7 @@ FUNCTION(PYTHON_ADD_MODULE _NAME )
SET_TARGET_PROPERTIES(${_NAME} PROPERTIES LINK_FLAGS
"-undefined dynamic_lookup")
ELSEIF(MSVC)
- target_link_libraries(${_NAME} ${PYTHON_LIBRARIES})
+ target_link_libraries(${_NAME} ${PYTHON_LIBRARIES} $<$<CONFIG:Debug>:${PYTHON_DEBUG_LIBRARIES}> $<$<CONFIG:Release>:${PYTHON_LIBRARIES}>)
Review comment:
`${PYTHON_LIBRARIES}` is duplicated in the `Release` configuration.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on a change in pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on a change in pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#discussion_r432762308
##########
File path: cpp/cmake_modules/DefineOptions.cmake
##########
@@ -120,6 +120,8 @@ if("${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
define_option(ARROW_GGDB_DEBUG "Pass -ggdb flag to debug builds" ON)
+ define_option(ARROW_USE_DEPRECATED_FIND_PYTHONLIBS "Force using deprecated FindPythonLibsNew script" OFF)
Review comment:
👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-634362206
https://issues.apache.org/jira/browse/ARROW-7409
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-639152015
>
>
> @raulbocanegra Could you confirm that [#7284 (review)](https://github.com/apache/arrow/pull/7284#pullrequestreview-419603540) without your changes can fix this problem?
Sorry, that change does not work because `PYTHON_IS_DEBUG` is not set to 1 unless `PYTHON_EXECUTABLE` is `python_d`, but `find_package(PythonInterp)` does not return `python_d`, but `python` release executable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#discussion_r433810778
##########
File path: cpp/src/arrow/python/CMakeLists.txt
##########
@@ -60,7 +60,14 @@ endif()
set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
if(WIN32)
- list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+ if(ARROW_USE_DEPRECATED_FIND_PYTHONLIBS)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS
+ $<$<CONFIG:Debug>:${PYTHON_DEBUG_LIBRARIES}>
+ $<$<CONFIG:Release>:${PYTHON_LIBRARIES}>
Review comment:
Can't you factor out the debug/release library switch?
##########
File path: cpp/src/arrow/python/CMakeLists.txt
##########
@@ -87,6 +94,10 @@ add_arrow_lib(arrow_python
EXTRA_INCLUDES
"${ARROW_PYTHON_INCLUDES}")
+if(WIN32 AND TARGET arrow_python_shared AND NOT ARROW_USE_DEPRECATED_FIND_PYTHONLIBS)
Review comment:
Why is this complication needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra edited a comment on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra edited a comment on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-646182178
Hi!
First of all, sorry for the delay guys. I tried the `-DPYTHON_EXECUTABLE=python_d` suggested by @kou but it then raised an error related to some numpy library.
I am really busy this week. Please, give me a few days more to give you some more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-693683266
Hi again,
The error related to numpy seems to be that I have a release version of numpy, but I am building a debug version of Arrow.
[Numpy docs](https://numpy.org/doc/stable/user/troubleshooting-importerror.html#debug-build-on-windows) recommends building using `RelWithDebInfo` or recompile the entire stack of python modules including numpy in Debug mode which it's a real pain in case you have a lot of modules.
To sum up, I will try to stick with `RelWithDebInfo` build type for Windows, so I feel that this PR could be closed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-639952334
Thanks for confirming it.
Could you confirm one more thing?
Does https://github.com/apache/arrow/pull/7284#pullrequestreview-419603540 without your changes and `cmake ... -DPYTHON_EXECUTABLE=python_d` fix this problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou closed pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou closed pull request #7284:
URL: https://github.com/apache/arrow/pull/7284
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on a change in pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on a change in pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#discussion_r435516397
##########
File path: cpp/src/arrow/python/CMakeLists.txt
##########
@@ -87,6 +94,10 @@ add_arrow_lib(arrow_python
EXTRA_INCLUDES
"${ARROW_PYTHON_INCLUDES}")
+if(WIN32 AND TARGET arrow_python_shared AND NOT ARROW_USE_DEPRECATED_FIND_PYTHONLIBS)
Review comment:
Well, the point here is that `PYTHON_LIBRARIES` was originally appended to `ARROW_PYTHON_SHARED_LINK_LIBS` as you can see on line 63 (red) above.
Because of that, we ended in a situation in which we have two different kinds of items in the `ARROW_PYTHON_SHARED_LINK_LIBS` list, the first item is the target `arrow_shared` and a second item in the form of `debug`, `optimized` keyword followed by the library path. It is described at the first and last items in the list [here](https://cmake.org/cmake/help/latest/command/target_link_libraries.html#id2).
I don't know if mixing different types of items in a list variable is supported, but the fact is that it fails, so I have to add that dependency on a second step here, after `add_arrow_lib`. We only need to do this on Windows, when we are not using the deprecated scripts, and of course, if the target is defined.
I know it is not the prettiest if branch you can find, but I couldn't find another way.
If the description of the problem above helps you finding a better solution I am happy to change it.
##########
File path: cpp/src/arrow/python/CMakeLists.txt
##########
@@ -60,7 +60,14 @@ endif()
set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
if(WIN32)
- list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+ if(ARROW_USE_DEPRECATED_FIND_PYTHONLIBS)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS
+ $<$<CONFIG:Debug>:${PYTHON_DEBUG_LIBRARIES}>
+ $<$<CONFIG:Release>:${PYTHON_LIBRARIES}>
Review comment:
Hi, `PYTHON_DEBUG_LIBRARIES` is set on the deprecated `FindPythonLibsNew` script. If I don't use it here, it won't link against `python_d` and the linking stage will fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-639132165
@raulbocanegra Could you confirm that https://github.com/apache/arrow/pull/7284#pullrequestreview-419603540 without your changes can fix this problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-646310278
No problem. Thanks for confirming.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulbocanegra commented on a change in pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
raulbocanegra commented on a change in pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#discussion_r432762353
##########
File path: cpp/cmake_modules/FindPythonLibsNew.cmake
##########
@@ -246,7 +248,7 @@ FUNCTION(PYTHON_ADD_MODULE _NAME )
SET_TARGET_PROPERTIES(${_NAME} PROPERTIES LINK_FLAGS
"-undefined dynamic_lookup")
ELSEIF(MSVC)
- target_link_libraries(${_NAME} ${PYTHON_LIBRARIES})
+ target_link_libraries(${_NAME} ${PYTHON_LIBRARIES} $<$<CONFIG:Debug>:${PYTHON_DEBUG_LIBRARIES}> $<$<CONFIG:Release>:${PYTHON_LIBRARIES}>)
Review comment:
:+1:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #7284: ARROW-7409: [C++][Python] Windows link error LNK1104: cannot open file 'python37_d.lib'
Posted by GitBox <gi...@apache.org>.
kou commented on pull request #7284:
URL: https://github.com/apache/arrow/pull/7284#issuecomment-693688982
OK. I close this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org