You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by "stephen mallette (JIRA)" <ji...@apache.org> on 2015/12/15 11:31:46 UTC
[jira] [Commented] (TINKERPOP-1043) Spark test InputRDDTest failing
on shouldReadFromArbitraryRDD
[ https://issues.apache.org/jira/browse/TINKERPOP-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057813#comment-15057813 ]
stephen mallette commented on TINKERPOP-1043:
---------------------------------------------
how are your running the tests? intellij? mvn clean install? i think i know what might be wrong, but it would be helpful if you could tell me the steps you use to get it in that state.
> Spark test InputRDDTest failing on shouldReadFromArbitraryRDD
> -------------------------------------------------------------
>
> Key: TINKERPOP-1043
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1043
> Project: TinkerPop
> Issue Type: Bug
> Affects Versions: 3.1.1-incubating
> Environment: Debian GNU/Linux 8, Java 8
> Reporter: Jonathan Ellithorpe
>
> Getting the following when running InputRDDTest.shouldReadFromArbitraryRDD:
> {code}
> java.lang.IllegalStateException: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing already exists
> at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:80)
> at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:126)
> at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:37)
> at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:158)
> at org.apache.tinkerpop.gremlin.spark.structure.io.InputRDDTest.shouldReadFromArbitraryRDD(InputRDDTest.java:56)
> Caused by: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing already exists
> at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1011)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:998)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:938)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
> at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:930)
> at org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:809)
> at org.apache.tinkerpop.gremlin.spark.process.computer.SparkExecutor.saveMapReduceRDD(SparkExecutor.java:203)
> at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$22(SparkGraphComputer.java:227)
> at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)