You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by SK <sk...@gmail.com> on 2014/06/27 20:44:11 UTC
scopt.OptionParser
Hi,
I tried to develop some code to use Logistic Regression, following the code
in BinaryClassification.scala in examples/mllib. My code compiles, but at
runtime complains that scopt/OptionParser class cannot be found. I have the
following import statement in my code:
import scopt.OptionParser
My sbt file contains the following dependencies:
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.0.0"
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.0.0"
libraryDependencies += "com.github.scopt" %% "scopt" % "3.2.0"
resolvers += "Akka Repository" at "http://repo.akka.io/releases/"
Is there anything else I need to do to include the OptionParser?
thanks
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/scopt-OptionParser-tp8436.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: scopt.OptionParser
Posted by Xiangrui Meng <me...@gmail.com>.
Thanks for posting the solution! You can also append `% "provided"` to
the `spark-mllib` dependency line and remove `spark-core` (because
spark-mllib already depends on spark-core) to make the assembly jar
smaller. -Xiangrui
On Fri, Aug 8, 2014 at 10:05 AM, SK <sk...@gmail.com> wrote:
> i was using sbt package when I got this error. Then I switched to using sbt
> assembly and that solved the issue. To run "sbt assembly", you need to have
> a file called plugins.sbt in the "<project root>/project" directory and it
> has the following line:
>
> addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
>
> This is in addition to the <project name>.sbt file I mentioned in the
> earlier mail.
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/scopt-OptionParser-tp8436p11800.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
SOLVED -- Re: scopt.OptionParser
Posted by Caron <ca...@gmail.com>.
Update:
The issue in my previous post was solved:
I had to change the sbt file name from <project_name>.sbt to build.sbt.
-----
Thanks!
-Caron
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/scopt-OptionParser-tp8436p20581.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: scopt.OptionParser
Posted by Caron <ca...@gmail.com>.
Hi,
I ran into the same problem posted in this thread earlier when I tried to
write my own program:
"$ spark-submit --class "someClass" --master local[4]
target/scala-2.10/someclass_2.10-1.0.jar " gives me
"Exception in thread "main" java.lang.NoClassDefFoundError:
scopt/OptionParser"
I tried your suggestion of adding a "plugin.sbt" file under <project
root>/project directory, then I ran:
"$ sbt assembly" in the <project root> directory
I got the following ERROR:
[error] Not a valid command: assembly
[error] Not a valid project ID: assembly
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: assembly
[error] assembly
[error] ^
Could someone tell what I'm missing here? Much appreciated!
-Caron
-----
Thanks!
-Caron
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/scopt-OptionParser-tp8436p20409.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: scopt.OptionParser
Posted by SK <sk...@gmail.com>.
i was using sbt package when I got this error. Then I switched to using sbt
assembly and that solved the issue. To run "sbt assembly", you need to have
a file called plugins.sbt in the "<project root>/project" directory and it
has the following line:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
This is in addition to the <project name>.sbt file I mentioned in the
earlier mail.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/scopt-OptionParser-tp8436p11800.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org