You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Sandy <sn...@gmail.com> on 2009/01/29 18:49:02 UTC
beginner question about setting user defined variables in partition
function
Hi,
I am still relatively a beginner at java and hadoop. I am trying to do set a
user-defined variable in my own partition function, which I define in a file
called myPartitioner.java. The structure of the code is below:
----
package mypackage;
import java.util.*;
import java.io.*;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.Partitioner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
public class myPartitioner extends Configured implements
Partitioner<IntWritable, IntWritable> {
private int T;
public void configure(JobConf job) {
super.configure(job);
String myT = job.get("tval");
T = Integer.parseInt(myT);
}
public int getPartition(IntWritable key, IntWritable value, int
numPartitions) {
...
}
}
When I try to compile, I get the following error pointing to the line
super.configure(job):
myPartitioner.java: cannot find symbol
[javac] symbol : method configure(org.apache.hadoop.mapred.JobConf)
[javac] location: class org.apache.hadoop.conf.Configured
[javac] super.configure(job);
[javac] ^
[javac] 1 error
In my regular M/R program I use the public void configure(JobConf) function
like this liberally. I'm thinking I am missing importing some class
somewhere, but I am not sure which. Could someone please point me in the
right direction? This is hadoop-0.19.0.
Thanks in advance.
-SM
Re: beginner question about setting user defined variables in
partition function
Posted by Sandy <sn...@gmail.com>.
The error was that the superclass was in MapReduceBase rather than
Configured, like I had originally thought.
-SM
On Thu, Jan 29, 2009 at 11:49 AM, Sandy <sn...@gmail.com> wrote:
> Hi,
>
> I am still relatively a beginner at java and hadoop. I am trying to do set
> a user-defined variable in my own partition function, which I define in a
> file called myPartitioner.java. The structure of the code is below:
>
> ----
> package mypackage;
> import java.util.*;
> import java.io.*;
> import org.apache.hadoop.io.IntWritable;
> import org.apache.hadoop.mapred.JobConf;
> import org.apache.hadoop.mapred.Partitioner;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.conf.Configured;
>
> public class myPartitioner extends Configured implements
> Partitioner<IntWritable, IntWritable> {
>
> private int T;
>
> public void configure(JobConf job) {
> super.configure(job);
> String myT = job.get("tval");
> T = Integer.parseInt(myT);
> }
>
> public int getPartition(IntWritable key, IntWritable value, int
> numPartitions) {
> ...
> }
> }
>
> When I try to compile, I get the following error pointing to the line
> super.configure(job):
> myPartitioner.java: cannot find symbol
> [javac] symbol : method configure(org.apache.hadoop.mapred.JobConf)
> [javac] location: class org.apache.hadoop.conf.Configured
> [javac] super.configure(job);
> [javac] ^
> [javac] 1 error
>
> In my regular M/R program I use the public void configure(JobConf) function
> like this liberally. I'm thinking I am missing importing some class
> somewhere, but I am not sure which. Could someone please point me in the
> right direction? This is hadoop-0.19.0.
>
> Thanks in advance.
>
> -SM
>