You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@accumulo.apache.org by Jianshi Huang <ji...@gmail.com> on 2014/07/18 12:40:44 UTC

Need help (gets Error: Instance has not been configured for AccumuloOutputFormat)

I got the following errors using AccumuloOutputFormat from Spark:

14/07/18 18:32:21 ERROR Executor: Exception in task ID 0
java.io.IOException: java.lang.IllegalStateException: Instance has not been
configured for AccumuloOutputFormat
 at
org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.getRecordWriter(AccumuloOutputFormat.java:558)
at org.apache.spark.rdd.PairRDDFunctions.org
$apache$spark$rdd$PairRDDFunctions$$writeShard$1(PairRDDFunctions.scala:713)
 at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
 at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
at org.apache.spark.scheduler.Task.run(Task.scala:51)
 at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


However, AccumuloOutputFormat has already been set:

    val inst = new MockInstance("mock")
    val conn = inst.getConnector("test", new PasswordToken(""))
    conn.tableOperations().create(Conf.getString("accumulo.entity.table"))

    AccumuloOutputFormat.setMockInstance(job, inst.getInstanceName)
    AccumuloOutputFormat.setConnectorInfo(job, "test", new
PasswordToken(""))
    AccumuloOutputFormat.setCreateTables(job, true)
    AccumuloOutputFormat.setDefaultTableName(job,
Conf.getString("accumulo.entity.table"))


Any idea why it happened?

Cheers,
-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Re: Need help (gets Error: Instance has not been configured for AccumuloOutputFormat)

Posted by Josh Elser <jo...@gmail.com>.

For whatever reason, the Configuration that was passed to the
getRecordReader method is different than the one you called the
setMockInstance on.

I'm no spark expert, but I've experienced confusion in other frameworks
like Pig and Hive where the framework isn't always returning you the same
Configuration object you had before. You may have to invoke these
AccumuloOutputFormat methods more than once.
On Jul 18, 2014 6:41 AM, "Jianshi Huang" <ji...@gmail.com> wrote:

> I got the following errors using AccumuloOutputFormat from Spark:
>
> 14/07/18 18:32:21 ERROR Executor: Exception in task ID 0
> java.io.IOException: java.lang.IllegalStateException: Instance has not
> been configured for AccumuloOutputFormat
>  at
> org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.getRecordWriter(AccumuloOutputFormat.java:558)
> at org.apache.spark.rdd.PairRDDFunctions.org
> $apache$spark$rdd$PairRDDFunctions$$writeShard$1(PairRDDFunctions.scala:713)
>  at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
>  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
> at org.apache.spark.scheduler.Task.run(Task.scala:51)
>  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> However, AccumuloOutputFormat has already been set:
>
>     val inst = new MockInstance("mock")
>     val conn = inst.getConnector("test", new PasswordToken(""))
>     conn.tableOperations().create(Conf.getString("accumulo.entity.table"))
>
>     AccumuloOutputFormat.setMockInstance(job, inst.getInstanceName)
>     AccumuloOutputFormat.setConnectorInfo(job, "test", new
> PasswordToken(""))
>     AccumuloOutputFormat.setCreateTables(job, true)
>     AccumuloOutputFormat.setDefaultTableName(job,
> Conf.getString("accumulo.entity.table"))
>
>
> Any idea why it happened?
>
> Cheers,
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>