Creating a File From Reducer

[dinesh shan <cd...@gmail.com>]

Hi all ,

I am a beginner to hadoop .
I am passing a FileName as the Key and the Contents of he FIle as the Value
I want to write the contents of the FIle to a text file with Key as the
fileName,  from the reducer .
The approach that i used is

public static class myjobReducer extends Reducer<Text,Text,Text,Text>
    {
        FileOutputStream out;
        Text tex=new Text();
        public void reduce(Text key,Iterable <Text> values,Context
context)throws IOException, InterruptedException
        {

            String str=key.toString();

            out = new FileOutputStream(str);
            PrintStream p = new PrintStream( out );

            for(Text value:values)
            {
                p.println(value.toString());
                tex.set(value.toString());
                context.write(tex,tex);//Not Necessary
            }

            out.close();
        }

    }


I am not able to create the file as specified though i set the output path
in

 FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));


PLease through some light
-- 
Dinesh Shaan

Re: Creating a File From Reducer

[souri datta <so...@gmail.com>]

what error are you getting?can you copy paste the error here?

On Thu, Mar 24, 2011 at 5:49 PM, dinesh shan <cd...@gmail.com> wrote:

> Hi all ,
>
> I am a beginner to hadoop .
> I am passing a FileName as the Key and the Contents of he FIle as the Value
> I want to write the contents of the FIle to a text file with Key as the
> fileName,  from the reducer .
> The approach that i used is
>
> public static class myjobReducer extends Reducer<Text,Text,Text,Text>
>     {
>         FileOutputStream out;
>         Text tex=new Text();
>         public void reduce(Text key,Iterable <Text> values,Context
> context)throws IOException, InterruptedException
>         {
>
>             String str=key.toString();
>
>             out = new FileOutputStream(str);
>             PrintStream p = new PrintStream( out );
>
>             for(Text value:values)
>             {
>                 p.println(value.toString());
>                 tex.set(value.toString());
>                 context.write(tex,tex);//Not Necessary
>             }
>
>             out.close();
>         }
>
>     }
>
>
> I am not able to create the file as specified though i set the output path
> in
>
>  FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
>
>
> PLease through some light
> --
> Dinesh Shaan
>

Re: Creating a File From Reducer

[dinesh shan <cd...@gmail.com>]

Thank u Harsh ,
I will have to work on that ..

Re: Creating a File From Reducer

[Harsh J <qw...@gmail.com>]

Hello,

On Thu, Mar 24, 2011 at 5:49 PM, dinesh shan <cd...@gmail.com> wrote:
>         FileOutputStream out;

Using plain Java API will not really work right out-of-the-box.
Please see http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample

-- 
Harsh J
http://harshj.com