You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Remil Mohanan <re...@gmail.com> on 2017/08/16 15:17:23 UTC
Map reduce sample program
>
>
>
>>
>>>
>>> Hi there,
>>>
>>> Please help me to get a sample program for each scenario.
>>>
>>> 1) need a Java map reducer sample program where multiple parameters
>>> are passed from mapper to reducer.
>>> 2) need a Java map reducer program where there is a write to a file inside
>>> hdfs filesystem as well as a read from a file inside hdfs other than
>>> the normal input file and output file mentioned in the mapper and reducer.
>>>
>>> Have a nice day
>>>
>>> Thanks
>>>
>>> Remil
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: Map reduce sample program
Posted by Daniel Templeton <da...@cloudera.com>.
On 8/19/17 3:28 AM, Remil Mohanan wrote:
> I am trying to pass multiple non key values from mapper to reducer.
The only way to pass data from the mapper to the reducer is through
passing key-values. One common trick is to designate a special key as
the out-of-band information key and then use a custom sorting comparator
to make sure that key comes first in the sort order. I'm sure you can
find examples online.
> Similarly for reading and writing a file inside the hdfs system other than normal read and write.
>
I don't understand. Reading and writing a file in HDFS from an MR task
works exactly the same as doing it from a stand-alone program. You
probably want to do it in the setup() method, though.
Daniel
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: Map reduce sample program
Posted by Remil Mohanan <re...@gmail.com>.
Hello Daniel,
I am trying to pass multiple non key values from mapper to reducer. I can pass key columns by concatenating the text key values. Can you please give a working sample code. Similarly for reading and writing a file inside the hdfs system other than normal read and write.
Thanks
Remil
Sent from my iPhone
> On 17 Aug 2017, at 12:34 AM, Daniel Templeton <da...@cloudera.com> wrote:
>
> Can you clarify what you mean for #1? For #2, try this:
>
> https://tutorials.techmytalk.com/2014/08/16/hadoop-hdfs-java-api/
>
> Daniel
>
>> On 8/16/17 8:17 AM, Remil Mohanan wrote:
>>
>>
>>>
>>>
>>>>> Hi there,
>>>>>
>>>>> Please help me to get a sample program for each scenario.
>>>>>
>>>>> 1) need a Java map reducer sample program where multiple parameters
>>>>> are passed from mapper to reducer.
>>>>> 2) need a Java map reducer program where there is a write to a file inside
>>>>> hdfs filesystem as well as a read from a file inside hdfs other than
>>>>> the normal input file and output file mentioned in the mapper and reducer.
>>>>>
>>>>> Have a nice day
>>>>>
>>>>> Thanks
>>>>>
>>>>> Remil
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: Map reduce sample program
Posted by Daniel Templeton <da...@cloudera.com>.
Can you clarify what you mean for #1? For #2, try this:
https://tutorials.techmytalk.com/2014/08/16/hadoop-hdfs-java-api/
Daniel
On 8/16/17 8:17 AM, Remil Mohanan wrote:
>
>
>>
>>
>>>> Hi there,
>>>>
>>>> Please help me to get a sample program for each scenario.
>>>>
>>>> 1) need a Java map reducer sample program where multiple parameters
>>>> are passed from mapper to reducer.
>>>> 2) need a Java map reducer program where there is a write to a file inside
>>>> hdfs filesystem as well as a read from a file inside hdfs other than
>>>> the normal input file and output file mentioned in the mapper and reducer.
>>>>
>>>> Have a nice day
>>>>
>>>> Thanks
>>>>
>>>> Remil
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org