Solr + Hadoop

[Joan <jo...@gmail.com>]

Hi,

I'm trying build solr index with MapReduce (Hadoop) and I'm using
https://issues.apache.org/jira/browse/SOLR-1301 but I've a problem with
hadoop version and this patch.

When I compile this patch, I use 0.21.0 hadoop version, I don't have any
problem but when I'm trying to run my job in Hadoop (0.0.21) I get some
error like this:

*Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected*
 at
org.apache.solr.hadoop.SolrOutputFormat.checkOutputSpecs(SolrOutputFormat.java:147)
        at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:373)
        at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:334)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:960)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:976)



I try to override the next method:

  *public void checkOutputSpecs(JobContext job) throws IOException {
    super.checkOutputSpecs(job);
    if (job.getConfiguration().get(SETUP_OK) == null) {
      throw new IOException("Solr home cache not set up!");
    }
  }*

by

 * public void checkOutputSpecs(Job job) throws IOException {
    super.checkOutputSpecs(job);
    if (job.getConfiguration().get(SETUP_OK) == null) {
      throw new IOException("Solr home cache not set up!");
    }
  }*

but I continue receive some error:
*
        java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.solr.hadoop.SolrOutputFormat
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1128)
        at
org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:203)
        at org.apache.hadoop.mapred.Task.initialize(Task.java:487)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
        at java.security.AccessController.doPrivileged(Native Method)*



Please, Is someone using this patch with 0.21.0 Version Hadoop?. Can someone
help me?

Thanks,

Joan

Re: Solr + Hadoop

[Alexander Kanarsky <ka...@gmail.com>]

Joan,

make sure that you are running the job on Hadoop 0.21 cluster. (It
looks like you have compiled the apache-solr-hadoop jar with Hadoop
0.21 but using it on 0.20 cluster).

-Alexander

Re: Solr + Hadoop

[Em <ma...@yahoo.de>]

Hi Joan,

I am not sure whether it applies, but are you really using Solr 1.4 (not
1.4.1) and were also using the Hadoop-Jars provided by this patch (0.20.1
not 0.0.21)?
I ask, because I had some other issues with other classes that were related
to different package-definitions etc. - in short: some import-organization
failed and my IDE did not notice that when I build the files.

However, this is just a guess. 

Regards
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Hadoop-tp2247856p2249935.html
Sent from the Solr - User mailing list archive at Nabble.com.