[jira] [Closed] (TINKERPOP-1043) Spark test InputRDDTest failing on shouldReadFromArbitraryRDD

["Marko A. Rodriguez (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TINKERPOP-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marko A. Rodriguez closed TINKERPOP-1043.
-----------------------------------------
    Resolution: Cannot Reproduce
      Assignee: Marko A. Rodriguez

I can't recreate this not can [~spmallette]... Please reopen if you can provide more information which will allow us to see the exception happen.

> Spark test InputRDDTest failing on shouldReadFromArbitraryRDD
> -------------------------------------------------------------
>
>                 Key: TINKERPOP-1043
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1043
>             Project: TinkerPop
>          Issue Type: Bug
>          Components: hadoop, test-suite
>    Affects Versions: 3.1.1-incubating
>         Environment: Debian GNU/Linux 8, Java 8
>            Reporter: Jonathan Ellithorpe
>            Assignee: Marko A. Rodriguez
>
> Getting the following when running InputRDDTest.shouldReadFromArbitraryRDD:
> {code}
> java.lang.IllegalStateException: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing already exists
>         at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
>         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
>         at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:80)
>         at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:126)
>         at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:37)
>         at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:158)
>         at org.apache.tinkerpop.gremlin.spark.structure.io.InputRDDTest.shouldReadFromArbitraryRDD(InputRDDTest.java:56)
> Caused by: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing already exists
>         at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1011)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
>         at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:998)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:938)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
>         at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
>         at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:930)
>         at org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:809)
>         at org.apache.tinkerpop.gremlin.spark.process.computer.SparkExecutor.saveMapReduceRDD(SparkExecutor.java:203)
>         at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$22(SparkGraphComputer.java:227)
>         at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)