You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by unmesha sreeveni <un...@gmail.com> on 2013/12/19 04:24:46 UTC
setup() and cleanup() methods for mapred api.
Is there setup() and cleanup() methods for mapred api.
Is there a sample code for reference
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: setup() and cleanup() methods for mapred api.
Posted by 梁李印 <li...@aliyun-inc.com>.
With old-api (mapred.*), there are no setup() and cleanup(). But you can use
configure() and close(). Here is a sample code:
public static class MapClass extends MapReduceBase
implements Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
@Override
public void configure(JobConf job) {
}
@Override
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer itr = new StringTokenizer(line);
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
output.collect(word, one);
}
}
@Override
public void close() throws IOException {
}
}
Liyin Liang
发件人: unmesha sreeveni [mailto:unmeshabiju@gmail.com]
发送时间: 2013年12月19日 11:25
收件人: User Hadoop
主题: setup() and cleanup() methods for mapred api.
Is there setup() and cleanup() methods for mapred api.
Is there a sample code for reference
--
Thanks & Regards
Unmesha Sreeveni U.B
Junior Developer
Re: setup() and cleanup() methods for mapred api.
Posted by 梁李印 <li...@aliyun-inc.com>.
With old-api (mapred.*), there are no setup() and cleanup(). But you can use
configure() and close(). Here is a sample code:
public static class MapClass extends MapReduceBase
implements Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
@Override
public void configure(JobConf job) {
}
@Override
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer itr = new StringTokenizer(line);
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
output.collect(word, one);
}
}
@Override
public void close() throws IOException {
}
}
Liyin Liang
发件人: unmesha sreeveni [mailto:unmeshabiju@gmail.com]
发送时间: 2013年12月19日 11:25
收件人: User Hadoop
主题: setup() and cleanup() methods for mapred api.
Is there setup() and cleanup() methods for mapred api.
Is there a sample code for reference
--
Thanks & Regards
Unmesha Sreeveni U.B
Junior Developer
Re: setup() and cleanup() methods for mapred api.
Posted by 梁李印 <li...@aliyun-inc.com>.
With old-api (mapred.*), there are no setup() and cleanup(). But you can use
configure() and close(). Here is a sample code:
public static class MapClass extends MapReduceBase
implements Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
@Override
public void configure(JobConf job) {
}
@Override
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer itr = new StringTokenizer(line);
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
output.collect(word, one);
}
}
@Override
public void close() throws IOException {
}
}
Liyin Liang
发件人: unmesha sreeveni [mailto:unmeshabiju@gmail.com]
发送时间: 2013年12月19日 11:25
收件人: User Hadoop
主题: setup() and cleanup() methods for mapred api.
Is there setup() and cleanup() methods for mapred api.
Is there a sample code for reference
--
Thanks & Regards
Unmesha Sreeveni U.B
Junior Developer
Re: setup() and cleanup() methods for mapred api.
Posted by 梁李印 <li...@aliyun-inc.com>.
With old-api (mapred.*), there are no setup() and cleanup(). But you can use
configure() and close(). Here is a sample code:
public static class MapClass extends MapReduceBase
implements Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
@Override
public void configure(JobConf job) {
}
@Override
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer itr = new StringTokenizer(line);
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
output.collect(word, one);
}
}
@Override
public void close() throws IOException {
}
}
Liyin Liang
发件人: unmesha sreeveni [mailto:unmeshabiju@gmail.com]
发送时间: 2013年12月19日 11:25
收件人: User Hadoop
主题: setup() and cleanup() methods for mapred api.
Is there setup() and cleanup() methods for mapred api.
Is there a sample code for reference
--
Thanks & Regards
Unmesha Sreeveni U.B
Junior Developer
Re: setup() and cleanup() methods for mapred api.
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks for Your reply Liyin. close() get executed after each reducer right?
Are we able to capture a dynamic value in close()?
On Thu, Dec 19, 2013 at 9:20 AM, 梁李印 <li...@aliyun-inc.com> wrote:
> With old-api (mapred.*), there are no setup() and cleanup(). But you can
> use configure() and close(). Here is a sample code:
>
>
>
> *public* *static* *class* MapClass *extends* MapReduceBase
>
> *implements* Mapper<LongWritable, Text, Text, IntWritable> {
>
>
>
> *private* *final* *static* IntWritable *one* = *new* IntWritable(1);
>
> *private* Text word = *new* Text();
>
>
>
> @Override
>
> *public* *void* configure(JobConf job) {
>
>
>
> }
>
>
>
> @Override
>
> *public* *void* map(LongWritable key, Text value,
>
> OutputCollector<Text, IntWritable> output,
>
> Reporter reporter) *throws* IOException {
>
> String line = value.toString();
>
> StringTokenizer itr = *new* StringTokenizer(line);
>
> *while* (itr.hasMoreTokens()) {
>
> word.set(itr.nextToken());
>
> output.collect(word, *one*);
>
> }
>
> }
>
>
>
> @Override
>
> *public* *void* close() *throws* IOException {
>
>
>
> }
>
> }
>
>
>
> Liyin Liang
>
> *发件人:* unmesha sreeveni [mailto:unmeshabiju@gmail.com]
> *发送时间:* 2013年12月19日 11:25
> *收件人:* User Hadoop
> *主题:* setup() and cleanup() methods for mapred api.
>
>
>
> Is there setup() and cleanup() methods for mapred api.
>
>
>
> Is there a sample code for reference
>
>
>
> --
>
> *Thanks & Regards*
>
>
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: setup() and cleanup() methods for mapred api.
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks for Your reply Liyin. close() get executed after each reducer right?
Are we able to capture a dynamic value in close()?
On Thu, Dec 19, 2013 at 9:20 AM, 梁李印 <li...@aliyun-inc.com> wrote:
> With old-api (mapred.*), there are no setup() and cleanup(). But you can
> use configure() and close(). Here is a sample code:
>
>
>
> *public* *static* *class* MapClass *extends* MapReduceBase
>
> *implements* Mapper<LongWritable, Text, Text, IntWritable> {
>
>
>
> *private* *final* *static* IntWritable *one* = *new* IntWritable(1);
>
> *private* Text word = *new* Text();
>
>
>
> @Override
>
> *public* *void* configure(JobConf job) {
>
>
>
> }
>
>
>
> @Override
>
> *public* *void* map(LongWritable key, Text value,
>
> OutputCollector<Text, IntWritable> output,
>
> Reporter reporter) *throws* IOException {
>
> String line = value.toString();
>
> StringTokenizer itr = *new* StringTokenizer(line);
>
> *while* (itr.hasMoreTokens()) {
>
> word.set(itr.nextToken());
>
> output.collect(word, *one*);
>
> }
>
> }
>
>
>
> @Override
>
> *public* *void* close() *throws* IOException {
>
>
>
> }
>
> }
>
>
>
> Liyin Liang
>
> *发件人:* unmesha sreeveni [mailto:unmeshabiju@gmail.com]
> *发送时间:* 2013年12月19日 11:25
> *收件人:* User Hadoop
> *主题:* setup() and cleanup() methods for mapred api.
>
>
>
> Is there setup() and cleanup() methods for mapred api.
>
>
>
> Is there a sample code for reference
>
>
>
> --
>
> *Thanks & Regards*
>
>
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: setup() and cleanup() methods for mapred api.
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks for Your reply Liyin. close() get executed after each reducer right?
Are we able to capture a dynamic value in close()?
On Thu, Dec 19, 2013 at 9:20 AM, 梁李印 <li...@aliyun-inc.com> wrote:
> With old-api (mapred.*), there are no setup() and cleanup(). But you can
> use configure() and close(). Here is a sample code:
>
>
>
> *public* *static* *class* MapClass *extends* MapReduceBase
>
> *implements* Mapper<LongWritable, Text, Text, IntWritable> {
>
>
>
> *private* *final* *static* IntWritable *one* = *new* IntWritable(1);
>
> *private* Text word = *new* Text();
>
>
>
> @Override
>
> *public* *void* configure(JobConf job) {
>
>
>
> }
>
>
>
> @Override
>
> *public* *void* map(LongWritable key, Text value,
>
> OutputCollector<Text, IntWritable> output,
>
> Reporter reporter) *throws* IOException {
>
> String line = value.toString();
>
> StringTokenizer itr = *new* StringTokenizer(line);
>
> *while* (itr.hasMoreTokens()) {
>
> word.set(itr.nextToken());
>
> output.collect(word, *one*);
>
> }
>
> }
>
>
>
> @Override
>
> *public* *void* close() *throws* IOException {
>
>
>
> }
>
> }
>
>
>
> Liyin Liang
>
> *发件人:* unmesha sreeveni [mailto:unmeshabiju@gmail.com]
> *发送时间:* 2013年12月19日 11:25
> *收件人:* User Hadoop
> *主题:* setup() and cleanup() methods for mapred api.
>
>
>
> Is there setup() and cleanup() methods for mapred api.
>
>
>
> Is there a sample code for reference
>
>
>
> --
>
> *Thanks & Regards*
>
>
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: setup() and cleanup() methods for mapred api.
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks for Your reply Liyin. close() get executed after each reducer right?
Are we able to capture a dynamic value in close()?
On Thu, Dec 19, 2013 at 9:20 AM, 梁李印 <li...@aliyun-inc.com> wrote:
> With old-api (mapred.*), there are no setup() and cleanup(). But you can
> use configure() and close(). Here is a sample code:
>
>
>
> *public* *static* *class* MapClass *extends* MapReduceBase
>
> *implements* Mapper<LongWritable, Text, Text, IntWritable> {
>
>
>
> *private* *final* *static* IntWritable *one* = *new* IntWritable(1);
>
> *private* Text word = *new* Text();
>
>
>
> @Override
>
> *public* *void* configure(JobConf job) {
>
>
>
> }
>
>
>
> @Override
>
> *public* *void* map(LongWritable key, Text value,
>
> OutputCollector<Text, IntWritable> output,
>
> Reporter reporter) *throws* IOException {
>
> String line = value.toString();
>
> StringTokenizer itr = *new* StringTokenizer(line);
>
> *while* (itr.hasMoreTokens()) {
>
> word.set(itr.nextToken());
>
> output.collect(word, *one*);
>
> }
>
> }
>
>
>
> @Override
>
> *public* *void* close() *throws* IOException {
>
>
>
> }
>
> }
>
>
>
> Liyin Liang
>
> *发件人:* unmesha sreeveni [mailto:unmeshabiju@gmail.com]
> *发送时间:* 2013年12月19日 11:25
> *收件人:* User Hadoop
> *主题:* setup() and cleanup() methods for mapred api.
>
>
>
> Is there setup() and cleanup() methods for mapred api.
>
>
>
> Is there a sample code for reference
>
>
>
> --
>
> *Thanks & Regards*
>
>
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*