You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yin Huai (JIRA)" <ji...@apache.org> on 2015/04/15 19:36:59 UTC
[jira] [Resolved] (SPARK-6060) List type missing for catalyst's
package.scala
[ https://issues.apache.org/jira/browse/SPARK-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai resolved SPARK-6060.
-----------------------------
Resolution: Not A Problem
[~stedre] Unfortunately, sometimes we need to do {{clean}} first. I am resolving it as "Not a Problem".
> List type missing for catalyst's package.scala
> ----------------------------------------------
>
> Key: SPARK-6060
> URL: https://issues.apache.org/jira/browse/SPARK-6060
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.3.0
> Environment: Linux zeno 3.18.5 #1 SMP Sun Feb 1 23:51:17 CET 2015 ppc64 GNU/Linux,
> java version "1.7.0_65"
> OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-2)
> OpenJDK Zero VM (build 24.65-b04, interpreted mode),
> sbt launcher version 0.13.7
> Reporter: Stephan Drescher
> Priority: Minor
> Labels: build, error
>
> Used command line:
> build/sbt -mem 1024 -Pyarn -Phive -Dhadoop.version=2.4.0 -Pbigtop-dist -DskipTests assembly
> Output:
> [error] while compiling: /home/spark/Developer/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/types/package.scala
> [error] during phase: jvm
> [error] library version: version 2.10.4
> [error] compiler version: version 2.10.4
> [error] reconstructed args: -bootclasspath /usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/resources.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/rt.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/jsse.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/jce.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/charsets.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/rhino.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/lib/jfr.jar:/usr/lib/jvm/java-7-openjdk-powerpc/jre/classes:/home/spark/.sbt/boot/scala-2.10.4/lib/scala-library.jar -deprecation -classpath /home/spark/Developer/spark/sql/catalyst/target/scala-2.10/classes:/home/spark/Developer/spark/core/target/scala-2.10/classes:/home/spark/Developer/spark/network/common/target/scala-2.10/classes:/home/spark/Developer/spark/network/shuffle/target/scala-2.10/classes:/home/spark/.sbt/boot/scala-2.10.4/lib/scala-compiler.jar:/home/spark/.sbt/boot/scala-2.10.4/lib/scala-reflect.jar:/home/spark/Developer/spark/lib_managed/jars/netty-all-4.0.23.Final.jar:/home/spark/Developer/spark/lib_managed/jars/unused-1.0.0.jar:/home/spark/Developer/spark/lib_managed/jars/chill_2.10-0.5.0.jar:/home/spark/Developer/spark/lib_managed/jars/chill-java-0.5.0.jar:/home/spark/Developer/spark/lib_managed/bundles/kryo-2.21.jar:/home/spark/Developer/spark/lib_managed/jars/reflectasm-1.07-shaded.jar:/home/spark/Developer/spark/lib_managed/jars/minlog-1.2.jar:/home/spark/Developer/spark/lib_managed/jars/objenesis-1.2.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-client-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-common-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-annotations-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/commons-cli-1.2.jar:/home/spark/Developer/spark/lib_managed/jars/commons-math3-3.1.1.jar:/home/spark/Developer/spark/lib_managed/jars/xmlenc-0.52.jar:/home/spark/Developer/spark/lib_managed/jars/commons-httpclient-3.1.jar:/home/spark/Developer/spark/lib_managed/jars/commons-logging-1.1.3.jar:/home/spark/Developer/spark/lib_managed/jars/commons-io-2.4.jar:/home/spark/Developer/spark/lib_managed/jars/commons-net-3.1.jar:/home/spark/Developer/spark/lib_managed/jars/commons-collections-3.2.1.jar:/home/spark/Developer/spark/lib_managed/bundles/log4j-1.2.17.jar:/home/spark/Developer/spark/lib_managed/jars/commons-lang-2.6.jar:/home/spark/Developer/spark/lib_managed/jars/commons-configuration-1.6.jar:/home/spark/Developer/spark/lib_managed/jars/commons-digester-1.8.jar:/home/spark/Developer/spark/lib_managed/jars/commons-beanutils-1.7.0.jar:/home/spark/Developer/spark/lib_managed/jars/commons-beanutils-core-1.8.0.jar:/home/spark/Developer/spark/lib_managed/jars/slf4j-api-1.7.10.jar:/home/spark/Developer/spark/lib_managed/jars/jackson-core-asl-1.8.8.jar:/home/spark/Developer/spark/lib_managed/jars/jackson-mapper-asl-1.8.8.jar:/home/spark/Developer/spark/lib_managed/jars/avro-1.7.4.jar:/home/spark/Developer/spark/lib_managed/jars/commons-compress-1.4.1.jar:/home/spark/Developer/spark/lib_managed/jars/xz-1.0.jar:/home/spark/Developer/spark/lib_managed/bundles/protobuf-java-2.5.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-auth-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/httpclient-4.2.5.jar:/home/spark/Developer/spark/lib_managed/jars/httpcore-4.2.4.jar:/home/spark/Developer/spark/lib_managed/jars/commons-codec-1.6.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-hdfs-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-util-6.1.26.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-mapreduce-client-app-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-mapreduce-client-common-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-yarn-common-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-yarn-api-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/jaxb-api-2.2.2.jar:/home/spark/Developer/spark/lib_managed/jars/stax-api-1.0-2.jar:/home/spark/Developer/spark/lib_managed/jars/activation-1.1.jar:/home/spark/Developer/spark/lib_managed/bundles/jersey-core-1.9.jar:/home/spark/Developer/spark/lib_managed/jars/guice-3.0.jar:/home/spark/Developer/spark/lib_managed/jars/javax.inject-1.jar:/home/spark/Developer/spark/lib_managed/jars/aopalliance-1.0.jar:/home/spark/Developer/spark/lib_managed/jars/cglib-2.2.1-v20090111.jar:/home/spark/Developer/spark/lib_managed/jars/asm-3.2.jar:/home/spark/Developer/spark/lib_managed/bundles/jersey-server-1.9.jar:/home/spark/Developer/spark/lib_managed/bundles/jersey-json-1.9.jar:/home/spark/Developer/spark/lib_managed/bundles/jettison-1.1.jar:/home/spark/Developer/spark/lib_managed/jars/jaxb-impl-2.2.3-1.jar:/home/spark/Developer/spark/lib_managed/jars/jackson-jaxrs-1.8.8.jar:/home/spark/Developer/spark/lib_managed/jars/jackson-xc-1.8.8.jar:/home/spark/Developer/spark/lib_managed/jars/jersey-guice-1.9.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-yarn-client-2.4.0.jar:/home/spark/Developer/spark/lib_managed/bundles/jersey-client-1.9.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-mapreduce-client-core-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-yarn-server-common-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-mapreduce-client-shuffle-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/hadoop-mapreduce-client-jobclient-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/jets3t-0.7.1.jar:/home/spark/Developer/spark/lib_managed/bundles/curator-recipes-2.4.0.jar:/home/spark/Developer/spark/lib_managed/bundles/curator-framework-2.4.0.jar:/home/spark/Developer/spark/lib_managed/bundles/curator-client-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/zookeeper-3.4.5.jar:/home/spark/Developer/spark/lib_managed/jars/jline-0.9.94.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-plus-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/orbits/javax.transaction-1.1.1.v201105210645.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-webapp-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-xml-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-util-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-servlet-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-security-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-server-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/orbits/javax.servlet-3.0.0.v201112011016.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-continuation-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-http-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-io-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/jars/jetty-jndi-8.1.14.v20131031.jar:/home/spark/Developer/spark/lib_managed/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/spark/Developer/spark/lib_managed/orbits/javax.activation-1.1.0.v201105071233.jar:/home/spark/Developer/spark/lib_managed/jars/commons-lang3-3.3.2.jar:/home/spark/Developer/spark/lib_managed/jars/jul-to-slf4j-1.7.10.jar:/home/spark/Developer/spark/lib_managed/jars/jcl-over-slf4j-1.7.10.jar:/home/spark/Developer/spark/lib_managed/jars/slf4j-log4j12-1.7.10.jar:/home/spark/Developer/spark/lib_managed/bundles/compress-lzf-1.0.0.jar:/home/spark/Developer/spark/lib_managed/bundles/snappy-java-1.1.1.6.jar:/home/spark/Developer/spark/lib_managed/jars/lz4-1.2.0.jar:/home/spark/Developer/spark/lib_managed/bundles/RoaringBitmap-0.4.5.jar:/home/spark/Developer/spark/lib_managed/jars/akka-remote_2.10-2.3.4-spark.jar:/home/spark/Developer/spark/lib_managed/jars/akka-actor_2.10-2.3.4-spark.jar:/home/spark/Developer/spark/lib_managed/bundles/config-1.2.1.jar:/home/spark/Developer/spark/lib_managed/bundles/netty-3.8.0.Final.jar:/home/spark/Developer/spark/lib_managed/bundles/protobuf-java-2.5.0-spark.jar:/home/spark/Developer/spark/lib_managed/jars/uncommons-maths-1.2.2a.jar:/home/spark/Developer/spark/lib_managed/jars/akka-slf4j_2.10-2.3.4-spark.jar:/home/spark/Developer/spark/lib_managed/jars/json4s-jackson_2.10-3.2.10.jar:/home/spark/Developer/spark/lib_managed/jars/json4s-core_2.10-3.2.10.jar:/home/spark/Developer/spark/lib_managed/jars/json4s-ast_2.10-3.2.10.jar:/home/spark/Developer/spark/lib_managed/jars/paranamer-2.6.jar:/home/spark/Developer/spark/lib_managed/jars/scalap-2.10.0.jar:/home/spark/Developer/spark/lib_managed/bundles/jackson-databind-2.4.4.jar:/home/spark/Developer/spark/lib_managed/bundles/jackson-core-2.4.4.jar:/home/spark/Developer/spark/lib_managed/jars/mesos-0.21.0-shaded-protobuf.jar:/home/spark/Developer/spark/lib_managed/jars/stream-2.7.0.jar:/home/spark/Developer/spark/lib_managed/bundles/metrics-core-3.1.0.jar:/home/spark/Developer/spark/lib_managed/bundles/metrics-jvm-3.1.0.jar:/home/spark/Developer/spark/lib_managed/bundles/metrics-json-3.1.0.jar:/home/spark/Developer/spark/lib_managed/bundles/metrics-graphite-3.1.0.jar:/home/spark/Developer/spark/lib_managed/bundles/jackson-module-scala_2.10-2.4.4.jar:/home/spark/Developer/spark/lib_managed/bundles/jackson-annotations-2.4.4.jar:/home/spark/Developer/spark/lib_managed/jars/jsr305-2.0.1.jar:/home/spark/Developer/spark/lib_managed/bundles/guava-15.0.jar:/home/spark/Developer/spark/lib_managed/jars/ivy-2.4.0.jar:/home/spark/Developer/spark/lib_managed/jars/oro-2.0.8.jar:/home/spark/Developer/spark/lib_managed/jars/tachyon-client-0.5.0.jar:/home/spark/Developer/spark/lib_managed/jars/tachyon-0.5.0.jar:/home/spark/Developer/spark/lib_managed/jars/pyrolite-2.0.1.jar:/home/spark/Developer/spark/lib_managed/jars/py4j-0.8.2.1.jar:/home/spark/Developer/spark/lib_managed/jars/quasiquotes_2.10-2.0.1.jar -feature -Xplugin:/home/spark/Developer/spark/lib_managed/jars/genjavadoc-plugin_2.10.4-0.8.jar -Xplugin:/home/spark/Developer/spark/lib_managed/jars/paradise_2.10.4-2.0.1.jar -unchecked -P:genjavadoc:out=/home/spark/Developer/spark/sql/catalyst/target/java
> [error]
> [error] last tree to typer: Literal(Constant(String))
> [error] symbol: null
> [error] symbol definition: null
> [error] tpe: Class(classOf[java.lang.String])
> [error] symbol owners:
> [error] context owners: package package -> package types
> [error]
> [error] == Enclosing template or block ==
> [error]
> [error] Template( // val <local package>: <notype> in package types, tree.tpe=org.apache.spark.sql.types.package.type
> [error] "java.lang.Object" // parents
> [error] ValDef(
> [error] private
> [error] "_"
> [error] <tpt>
> [error] <empty>
> [error] )
> [error] DefDef( // def <init>(): org.apache.spark.sql.types.package.type in package types
> [error] <method>
> [error] "<init>"
> [error] []
> [error] List(Nil)
> [error] <tpt> // tree.tpe=org.apache.spark.sql.types.package.type
> [error] Block( // tree.tpe=Unit
> [error] Apply( // def <init>(): Object in class Object, tree.tpe=Object
> [error] `package`.super."<init>" // def <init>(): Object in class Object, tree.tpe=()Object
> [error] Nil
> [error] )
> [error] ()
> [error] )
> [error] )
> [error] )
> [error]
> [error] == Expanded type of tree ==
> [error]
> [error] ConstantType(value = Constant(String))
> [error]
> [error] uncaught exception during compilation: java.lang.AssertionError
> java.lang.AssertionError: assertion failed: List(object package$ScalaUdfBuilder, object package$ScalaUdfBuilder)
> at scala.reflect.internal.Symbols$Symbol.suchThat(Symbols.scala:1678)
> at scala.reflect.internal.Symbols$ClassSymbol.companionModule0(Symbols.scala:2988)
> at scala.reflect.internal.Symbols$ClassSymbol.companionModule(Symbols.scala:2991)
> at scala.tools.nsc.backend.jvm.GenASM$JPlainBuilder.genClass(GenASM.scala:1371)
> at scala.tools.nsc.backend.jvm.GenASM$AsmPhase.run(GenASM.scala:120)
> at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1583)
> at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1557)
> at scala.tools.nsc.Global$Run.compileSources(Global.scala:1553)
> at scala.tools.nsc.Global$Run.compile(Global.scala:1662)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:116)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:95)
> at xsbt.CompilerInterface.run(CompilerInterface.scala:26)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:101)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:47)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
> at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply$mcV$sp(AggressiveCompile.scala:97)
> at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:97)
> at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:97)
> at sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:162)
> at sbt.compiler.AggressiveCompile$$anonfun$3.compileScala$1(AggressiveCompile.scala:96)
> at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.scala:139)
> at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.scala:86)
> at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:38)
> at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:36)
> at sbt.inc.IncrementalCommon.cycle(IncrementalCommon.scala:31)
> at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:39)
> at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:38)
> at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:66)
> at sbt.inc.Incremental$.compile(Incremental.scala:38)
> at sbt.inc.IncrementalCompile$.apply(Compile.scala:26)
> at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:153)
> at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:70)
> at sbt.compiler.AggressiveCompile.apply(AggressiveCompile.scala:45)
> at sbt.Compiler$.apply(Compiler.scala:74)
> at sbt.Compiler$.apply(Compiler.scala:65)
> at sbt.Defaults$.sbt$Defaults$$compileTaskImpl(Defaults.scala:789)
> at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:781)
> at sbt.Defaults$$anonfun$compileTask$1.apply(Defaults.scala:781)
> at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
> at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
> at sbt.std.Transform$$anon$4.work(System.scala:63)
> at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
> at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
> at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
> at sbt.Execute.work(Execute.scala:235)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
> at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
> at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> [error] (catalyst/compile:compile) java.lang.AssertionError: assertion failed: List(object package$ScalaUdfBuilder, object package$ScalaUdfBuilder)
> [error] Total time: 4149 s, completed Feb 27, 2015 9:58:30 AM
> Comment: The whole project stack of Spark had been compiled successfully two weeks ago, while using mvn (currently mvn and sbt fail on catalyst).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org