You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2013/10/27 00:55:45 UTC
MRUNIT basic question
Hi,
I have been searching in mrunit documentation but hasnt been able to find
it so far..
How do i pass configuration parameters in my mrunit.
So for example, if i take the wordcount example.
Lets say, in my driver code I am setting this parameter...
conf.set("delimiter",args[2])
And in my mapper code I am calling this as:
String delimiter = conf.get("delimiter");
String [] tokens = value.toString().split(delimiter);
for (String token:tokens)
context.write(token,one);
How do I set up this configuration parameter.
I have been looking into this example:
https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
Thanks
Re: MRUNIT basic question
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
There is a withConfiguration method.
http://mrunit.apache.org/documentation/javadocs/1.0.0/org/apache/hadoop/mrunit/TestDriver.html#withConfiguration%28org.apache.hadoop.conf.Configuration%29
But now it is recommended to actually get the default Configuration and
modify it (see documentation).
Bertrand
PS : There is a MRUnit mailing list which might have been more appropriate.
On Sun, Oct 27, 2013 at 12:55 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I have been searching in mrunit documentation but hasnt been able to
> find it so far..
> How do i pass configuration parameters in my mrunit.
>
> So for example, if i take the wordcount example.
>
> Lets say, in my driver code I am setting this parameter...
>
> conf.set("delimiter",args[2])
>
> And in my mapper code I am calling this as:
>
> String delimiter = conf.get("delimiter");
> String [] tokens = value.toString().split(delimiter);
> for (String token:tokens)
> context.write(token,one);
>
> How do I set up this configuration parameter.
>
> I have been looking into this example:
>
> https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
>
> Thanks
>
--
Bertrand Dechoux
Re: MRUNIT basic question
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
There is a withConfiguration method.
http://mrunit.apache.org/documentation/javadocs/1.0.0/org/apache/hadoop/mrunit/TestDriver.html#withConfiguration%28org.apache.hadoop.conf.Configuration%29
But now it is recommended to actually get the default Configuration and
modify it (see documentation).
Bertrand
PS : There is a MRUnit mailing list which might have been more appropriate.
On Sun, Oct 27, 2013 at 12:55 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I have been searching in mrunit documentation but hasnt been able to
> find it so far..
> How do i pass configuration parameters in my mrunit.
>
> So for example, if i take the wordcount example.
>
> Lets say, in my driver code I am setting this parameter...
>
> conf.set("delimiter",args[2])
>
> And in my mapper code I am calling this as:
>
> String delimiter = conf.get("delimiter");
> String [] tokens = value.toString().split(delimiter);
> for (String token:tokens)
> context.write(token,one);
>
> How do I set up this configuration parameter.
>
> I have been looking into this example:
>
> https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
>
> Thanks
>
--
Bertrand Dechoux
Re: MRUNIT basic question
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
There is a withConfiguration method.
http://mrunit.apache.org/documentation/javadocs/1.0.0/org/apache/hadoop/mrunit/TestDriver.html#withConfiguration%28org.apache.hadoop.conf.Configuration%29
But now it is recommended to actually get the default Configuration and
modify it (see documentation).
Bertrand
PS : There is a MRUnit mailing list which might have been more appropriate.
On Sun, Oct 27, 2013 at 12:55 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I have been searching in mrunit documentation but hasnt been able to
> find it so far..
> How do i pass configuration parameters in my mrunit.
>
> So for example, if i take the wordcount example.
>
> Lets say, in my driver code I am setting this parameter...
>
> conf.set("delimiter",args[2])
>
> And in my mapper code I am calling this as:
>
> String delimiter = conf.get("delimiter");
> String [] tokens = value.toString().split(delimiter);
> for (String token:tokens)
> context.write(token,one);
>
> How do I set up this configuration parameter.
>
> I have been looking into this example:
>
> https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
>
> Thanks
>
--
Bertrand Dechoux
Re: MRUNIT basic question
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
There is a withConfiguration method.
http://mrunit.apache.org/documentation/javadocs/1.0.0/org/apache/hadoop/mrunit/TestDriver.html#withConfiguration%28org.apache.hadoop.conf.Configuration%29
But now it is recommended to actually get the default Configuration and
modify it (see documentation).
Bertrand
PS : There is a MRUnit mailing list which might have been more appropriate.
On Sun, Oct 27, 2013 at 12:55 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I have been searching in mrunit documentation but hasnt been able to
> find it so far..
> How do i pass configuration parameters in my mrunit.
>
> So for example, if i take the wordcount example.
>
> Lets say, in my driver code I am setting this parameter...
>
> conf.set("delimiter",args[2])
>
> And in my mapper code I am calling this as:
>
> String delimiter = conf.get("delimiter");
> String [] tokens = value.toString().split(delimiter);
> for (String token:tokens)
> context.write(token,one);
>
> How do I set up this configuration parameter.
>
> I have been looking into this example:
>
> https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
>
> Thanks
>
--
Bertrand Dechoux
Re: MRUNIT basic question
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
There is a withConfiguration method.
http://mrunit.apache.org/documentation/javadocs/1.0.0/org/apache/hadoop/mrunit/TestDriver.html#withConfiguration%28org.apache.hadoop.conf.Configuration%29
But now it is recommended to actually get the default Configuration and
modify it (see documentation).
Bertrand
PS : There is a MRUnit mailing list which might have been more appropriate.
On Sun, Oct 27, 2013 at 12:55 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I have been searching in mrunit documentation but hasnt been able to
> find it so far..
> How do i pass configuration parameters in my mrunit.
>
> So for example, if i take the wordcount example.
>
> Lets say, in my driver code I am setting this parameter...
>
> conf.set("delimiter",args[2])
>
> And in my mapper code I am calling this as:
>
> String delimiter = conf.get("delimiter");
> String [] tokens = value.toString().split(delimiter);
> for (String token:tokens)
> context.write(token,one);
>
> How do I set up this configuration parameter.
>
> I have been looking into this example:
>
> https://github.com/wpm/Hadoop-Word-Count/blob/master/src/test/java/wpmcn/hadoop/WordCountTest.java
>
> Thanks
>
--
Bertrand Dechoux