You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Guillermo Ortiz <ko...@gmail.com> on 2015/01/08 14:15:47 UTC
Executing Spark, Error creating path from empty String.
When I try to execute my task with Spark it starts to copy the jars it
needs to HDFS and it finally fails, I don't know exactly why. I have
checked HDFS and it copies the files, so, it seems to work that part.
I changed the log level to debug but there's nothing else to help.
What else does Spark need to copy that it could be an empty string?
Spark assembly has been built with Hive, including Datanucleus jars on classpath
15/01/08 14:06:32 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where
applicable
15/01/08 14:06:32 INFO RMProxy: Connecting to ResourceManager at
vmlbyarnl01.lvtc.gsnet.corp/180.133.240.174:8050
15/01/08 14:06:33 INFO Client: Got cluster metric info from
ResourceManager, number of NodeManagers: 3
15/01/08 14:06:33 INFO Client: Max mem capabililty of a single
resource in this cluster 97280
15/01/08 14:06:33 INFO Client: Preparing Local resources
15/01/08 14:06:34 WARN BlockReaderLocal: The short-circuit local reads
feature cannot be used because libhadoop cannot be loaded.
15/01/08 14:06:34 INFO Client: Uploading
file:/home/spark-1.1.1-bin-hadoop2.4/lib/spark-assembly-1.1.1-hadoop2.4.0.jar
to hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/spark-assembly-1.1.1-hadoop2.4.0.jar
15/01/08 14:06:42 INFO Client: Uploading
file:/user/local/etc/lib/my-spark-streaming-scala.jar to
hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/my-spark-streaming-scala.jar
Exception in thread "main" java.lang.IllegalArgumentException: Can not
create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
at org.apache.hadoop.fs.Path.<init>(Path.java:135)
at org.apache.hadoop.fs.Path.<init>(Path.java:94)
at org.apache.spark.deploy.yarn.ClientBase$class.copyRemoteFile(ClientBase.scala:159)
at org.apache.spark.deploy.yarn.Client.copyRemoteFile(Client.scala:37)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:236)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:231)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:231)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:229)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.apache.spark.deploy.yarn.ClientBase$class.prepareLocalResources(ClientBase.scala:229)
at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:37)
at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:74)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:96)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:176)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:329)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Executing Spark, Error creating path from empty String.
Posted by Guillermo Ortiz <ko...@gmail.com>.
I was adding some bad jars I guess. I deleted all the jars and copied
them again and it works.
2015-01-08 14:15 GMT+01:00 Guillermo Ortiz <ko...@gmail.com>:
> When I try to execute my task with Spark it starts to copy the jars it
> needs to HDFS and it finally fails, I don't know exactly why. I have
> checked HDFS and it copies the files, so, it seems to work that part.
> I changed the log level to debug but there's nothing else to help.
> What else does Spark need to copy that it could be an empty string?
>
> Spark assembly has been built with Hive, including Datanucleus jars on classpath
> 15/01/08 14:06:32 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where
> applicable
> 15/01/08 14:06:32 INFO RMProxy: Connecting to ResourceManager at
> vmlbyarnl01.lvtc.gsnet.corp/180.133.240.174:8050
> 15/01/08 14:06:33 INFO Client: Got cluster metric info from
> ResourceManager, number of NodeManagers: 3
> 15/01/08 14:06:33 INFO Client: Max mem capabililty of a single
> resource in this cluster 97280
> 15/01/08 14:06:33 INFO Client: Preparing Local resources
> 15/01/08 14:06:34 WARN BlockReaderLocal: The short-circuit local reads
> feature cannot be used because libhadoop cannot be loaded.
> 15/01/08 14:06:34 INFO Client: Uploading
> file:/home/spark-1.1.1-bin-hadoop2.4/lib/spark-assembly-1.1.1-hadoop2.4.0.jar
> to hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/spark-assembly-1.1.1-hadoop2.4.0.jar
> 15/01/08 14:06:42 INFO Client: Uploading
> file:/user/local/etc/lib/my-spark-streaming-scala.jar to
> hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/my-spark-streaming-scala.jar
> Exception in thread "main" java.lang.IllegalArgumentException: Can not
> create a Path from an empty string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
> at org.apache.hadoop.fs.Path.<init>(Path.java:135)
> at org.apache.hadoop.fs.Path.<init>(Path.java:94)
> at org.apache.spark.deploy.yarn.ClientBase$class.copyRemoteFile(ClientBase.scala:159)
> at org.apache.spark.deploy.yarn.Client.copyRemoteFile(Client.scala:37)
> at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:236)
> at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:231)
> at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
> at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:231)
> at org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:229)
> at scala.collection.immutable.List.foreach(List.scala:318)
> at org.apache.spark.deploy.yarn.ClientBase$class.prepareLocalResources(ClientBase.scala:229)
> at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:37)
> at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:74)
> at org.apache.spark.deploy.yarn.Client.run(Client.scala:96)
> at org.apache.spark.deploy.yarn.Client$.main(Client.scala:176)
> at org.apache.spark.deploy.yarn.Client.main(Client.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:329)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org