You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by agg212 <al...@brown.edu> on 2014/12/01 17:49:44 UTC
Re: Mllib native netlib-java/OpenBLAS
Thanks for your reply, but I'm still running into issues
installing/configuring the native libraries for MLlib. Here are the steps
I've taken, please let me know if anything is incorrect.
- Download Spark source
- unzip and compile using `mvn -Pnetlib-lgpl -DskipTests clean package `
- Run `sbt/sbt publish-local`
The last step fails with the following error (full stack trace is attached
here: error.txt
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n20110/error.txt>
):
[error] (sql/compile:compile) java.lang.AssertionError: assertion failed:
List(object package$DebugNode, object package$DebugNode)
Do I still have to install OPENBLAS/anything else if I build Spark from the
source using the -Pnetlib-lgpl flag? Also, do I change the Spark version
(from 1.1.0 to 1.2.0-SNAPSHOT) in the .sbt file for my app?
Thanks!
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Mllib-native-netlib-java-OpenBLAS-tp19662p20110.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Mllib native netlib-java/OpenBLAS
Posted by xhudik <xh...@gmail.com>.
I'm half-way there
follow
1. compiled and installed open blas library
2. ln -s libopenblas_sandybridgep-r0.2.13.so /usr/lib/libblas.so.3
3. compiled and built spark:
mvn -Pnetlib-lgpl -DskipTests clean compile package
So far so fine. Then I run into problems by testing the solution:
bin/run-example mllib.LinearRegression data/mllib/sample_libsvm_data.txt
/14/12/30 18:39:57 INFO BlockManagerMaster: Registered BlockManager
14/12/30 18:39:58 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/12/30 18:39:58 WARN LoadSnappy: Snappy native library not loaded
Training: 80, test: 20.
*/usr/local/lib/jdk1.8.0//bin/java: symbol lookup error:
/tmp/jniloader1826801168744171087netlib-native_system-linux-x86_64.so:
undefined symbol: cblas_dscal*/
I created a issue report:
https://issues.apache.org/jira/browse/SPARK-5010
any help is deeply appreciated, Tomas
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Mllib-native-netlib-java-OpenBLAS-tp19662p20912.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Maven profile in MLLib netlib-lgpl not working (1.1.1)
Posted by Guillaume Pitel <gu...@exensa.com>.
Hi
Issue created https://issues.apache.org/jira/browse/SPARK-4816
Probably a maven-related question for profiles in child modules
I couldn't find a clean solution, just a workaround : modify pom.xml in
mllib module to force activation of netlib-lgpl module.
Hope a maven expert will help.
Guillaume
> +1 with 1.3-SNAPSHOT.
>
> On Mon, Dec 1, 2014 at 5:49 PM, agg212 <alexander_galakatos@brown.edu
> <ma...@brown.edu>> wrote:
>
> Thanks for your reply, but I'm still running into issues
> installing/configuring the native libraries for MLlib. Here are
> the steps
> I've taken, please let me know if anything is incorrect.
>
> - Download Spark source
> - unzip and compile using `mvn -Pnetlib-lgpl -DskipTests clean
> package `
> - Run `sbt/sbt publish-local`
>
> The last step fails with the following error (full stack trace is
> attached
> here: error.txt
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n20110/error.txt>
> ):
> [error] (sql/compile:compile) java.lang.AssertionError: assertion
> failed:
> List(object package$DebugNode, object package$DebugNode)
>
> Do I still have to install OPENBLAS/anything else if I build Spark
> from the
> source using the -Pnetlib-lgpl flag? Also, do I change the Spark
> version
> (from 1.1.0 to 1.2.0-SNAPSHOT) in the .sbt file for my app?
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Mllib-native-netlib-java-OpenBLAS-tp19662p20110.html
> Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> <ma...@spark.apache.org>
> For additional commands, e-mail: user-help@spark.apache.org
> <ma...@spark.apache.org>
>
>
--
eXenSa
*Guillaume PITEL, Président*
+33(0)626 222 431
eXenSa S.A.S. <http://www.exensa.com/>
41, rue Périer - 92120 Montrouge - FRANCE
Tel +33(0)184 163 677 / Fax +33(0)972 283 705
Maven profile in MLLib netlib-lgpl not working (1.1.1)
Posted by Guillaume Pitel <gu...@exensa.com>.
Hi
Issue created https://issues.apache.org/jira/browse/SPARK-4816
Probably a maven-related question for profiles in child modules
I couldn't find a clean solution, just a workaround : modify pom.xml in
mllib module to force activation of netlib-lgpl module.
Hope a maven expert will help.
Guillaume
> +1 with 1.3-SNAPSHOT.
>
> On Mon, Dec 1, 2014 at 5:49 PM, agg212 <alexander_galakatos@brown.edu
> <ma...@brown.edu>> wrote:
>
> Thanks for your reply, but I'm still running into issues
> installing/configuring the native libraries for MLlib. Here are
> the steps
> I've taken, please let me know if anything is incorrect.
>
> - Download Spark source
> - unzip and compile using `mvn -Pnetlib-lgpl -DskipTests clean
> package `
> - Run `sbt/sbt publish-local`
>
> The last step fails with the following error (full stack trace is
> attached
> here: error.txt
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n20110/error.txt>
> ):
> [error] (sql/compile:compile) java.lang.AssertionError: assertion
> failed:
> List(object package$DebugNode, object package$DebugNode)
>
> Do I still have to install OPENBLAS/anything else if I build Spark
> from the
> source using the -Pnetlib-lgpl flag? Also, do I change the Spark
> version
> (from 1.1.0 to 1.2.0-SNAPSHOT) in the .sbt file for my app?
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Mllib-native-netlib-java-OpenBLAS-tp19662p20110.html
> Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> <ma...@spark.apache.org>
> For additional commands, e-mail: user-help@spark.apache.org
> <ma...@spark.apache.org>
>
>
--
eXenSa
*Guillaume PITEL, Président*
+33(0)626 222 431
eXenSa S.A.S. <http://www.exensa.com/>
41, rue Périer - 92120 Montrouge - FRANCE
Tel +33(0)184 163 677 / Fax +33(0)972 283 705
Re: Mllib native netlib-java/OpenBLAS
Posted by Jaonary Rabarisoa <ja...@gmail.com>.
+1 with 1.3-SNAPSHOT.
On Mon, Dec 1, 2014 at 5:49 PM, agg212 <al...@brown.edu>
wrote:
> Thanks for your reply, but I'm still running into issues
> installing/configuring the native libraries for MLlib. Here are the steps
> I've taken, please let me know if anything is incorrect.
>
> - Download Spark source
> - unzip and compile using `mvn -Pnetlib-lgpl -DskipTests clean package `
> - Run `sbt/sbt publish-local`
>
> The last step fails with the following error (full stack trace is attached
> here: error.txt
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n20110/error.txt
> >
> ):
> [error] (sql/compile:compile) java.lang.AssertionError: assertion failed:
> List(object package$DebugNode, object package$DebugNode)
>
> Do I still have to install OPENBLAS/anything else if I build Spark from the
> source using the -Pnetlib-lgpl flag? Also, do I change the Spark version
> (from 1.1.0 to 1.2.0-SNAPSHOT) in the .sbt file for my app?
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Mllib-native-netlib-java-OpenBLAS-tp19662p20110.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>