[GitHub] thomelane opened a new issue #13853: Error while running the mxnet spark examples and test cases

[GitBox <gi...@apache.org>]

thomelane opened a new issue #13853: Error while running the mxnet spark examples and test cases
URL: https://github.com/apache/incubator-mxnet/issues/13853
 
 
   Opening issue on behalf of @adwivedi on the discussion forum (https://discuss.mxnet.io/t/error-while-running-the-mxnet-spark-examples-and-test-cases/2720).
   
   ### Quotes from the thread...
   
   I am trying to run the mxnet in distributed mode using spark as implemented here : https://github.com/apache/incubator-mxnet/tree/master/scala-package/spark
   
   but I am not able to run the examples and/or tests.
   
   The commented tests in the file : https://github.com/apache/incubator-mxnet/blob/master/scala-package/spark/src/test/scala/org/apache/mxnet/spark/MXNetGeneralSuite.scala keep running into the following error. I get the same error when I try to run the examples in the repo.
   
   ```
   Error: A JNI error has occurred, please check your installation and try again
   Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Seq
   	at java.lang.Class.getDeclaredMethods0(Native Method)
   	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
   	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
   	at java.lang.Class.getMethod0(Class.java:3018)
   	at java.lang.Class.getMethod(Class.java:1784)
   	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
   	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
   Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
   	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
   	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
   	... 7 more
   Exception in thread "Thread-21" java.lang.IllegalArgumentException: requirement failed: Failed to start ps scheduler process with exit code 1
   	at scala.Predef$.require(Predef.scala:224)
   	at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$startPSSchedulerInner$1(MXNet.scala:159)
   	at org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
   	at org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
   	at org.apache.mxnet.spark.MXNet$MXNetControllingThread.run(MXNet.scala:38)
   Error: A JNI error has occurred, please check your installation and try again
   Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Seq
   	at java.lang.Class.getDeclaredMethods0(Native Method)
   	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
   	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
   	at java.lang.Class.getMethod0(Class.java:3018)
   	at java.lang.Class.getMethod(Class.java:1784)
   	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
   	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
   Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
   	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
   	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
   	... 7 more
   ```
   
   I have seen this error generally when there’s a mismatch between the scala versions in the api, but in this case I am using the using the pom file that’s in the project and not including any external libraries.
   I have also looked at the pom file but I’ve not found any lib that might have a mismatch in this case, all the libraries in pom are 2.11 version
   
   I am building it from source, running it with this vm parameter : -
   
   ```
   Djava.library.path=/path_to_mxnet_source/incubator-mxnet/scala-package/native/osx-x86_64-cpu/target
   ```
   
   to let it find the native library.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services