[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request #15551: Bypass cuda/cudnn checks if no driver.

[GitBox <gi...@apache.org>]

marcoabreu commented on a change in pull request #15551: Bypass cuda/cudnn checks if no driver.
URL: https://github.com/apache/incubator-mxnet/pull/15551#discussion_r303898968
 
 

 ##########
 File path: src/common/cuda_utils.cc
 ##########
 @@ -44,8 +44,15 @@ namespace cuda {
 // Dynamic init here will emit a warning if runtime and compile-time cuda lib versions mismatch.
 // Also if the user has recompiled their source to a version no longer tested by upstream CI.
 bool cuda_version_check_performed = []() {
-  // Don't bother with checks if there are no GPUs visible (e.g. with CUDA_VISIBLE_DEVICES="")
-  if (dmlc::GetEnv("MXNET_CUDA_VERSION_CHECKING", true) && Context::GetGPUCount() > 0) {
+  // MXNet might be built on a machine with a cuda toolkit, but no GPUs or GPU driver.
+  // To allow that machine to execute say: python -c 'import mxnet; print(mxnet.__version__)',
+  // we won't perform a check if there is no driver.  Any actual attempt to use the cuda API's
+  // will yield the desired message: CUDA driver version is insufficient for CUDA runtime version.
+  int cuda_driver_version = 0;
+  CUDA_CALL(cudaDriverGetVersion(&cuda_driver_version));
+  // Also, don't bother with checks if there are no GPUs visible (e.g. with CUDA_VISIBLE_DEVICES="")
+  if (dmlc::GetEnv("MXNET_CUDA_VERSION_CHECKING", true) && cuda_driver_version > 0
+                                                        && Context::GetGPUCount() > 0) {
 
 Review comment:
   Could you make ```Context::GetGPUCount ``` properly return 0 if ```cuda_driver_version == 0```  instead of adding this additional check?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services