How to pass object between jobs

[Joan <jo...@gmail.com>]

Hi,

I would like pass one object from job1 to job2

Someone can I help me, please?

Thanks

Joan

Re: How to pass object between jobs

[Chase Bradford <ch...@gmail.com>]

You can make you object implement o.a.h.io.Writable and use the SequenceFileOutputFormat and SequenceFileInputFormat in job 1 and 2 respectively.

If something prevents you from implementing writable, but you can implement Serializable then you can add o.a.h.io.serialization.JavaSerialization to the list of serializations defined by the io.serialization conf option.  You would still use the sequence file input and output formats.

Chase

On Jan 25, 2011, at 11:21 PM, Joan <jo...@gmail.com> wrote:

> Hi harsh
> 
> I want to create many pairs <k,v> where v will be custom object so this will happen when my reduce's finished, so the output from job1 will be file with <k,v> = <Text,MyObject>
> 
> Then In second job I want to get MyObject from first job, ¿you know? So now, I have:
> 
> job1:
> 
> <Input,Output> <MyObject, Text>
> 
> job2;
> <Input,Output> <Text, MyObject>
> 
> Then I have to convert Text object, from first job, to MyObject in second job and I wan't to do this, I want to can work with MyObject directly.
> 
> Thanks,
> 
> Joan
> 
> 
> 2011/1/25 Harsh J <qw...@gmail.com>
> Could you describe your need with an example?
> 
> If you want the output of a Job (Job1), say being a single file with a
> single value in it, to be read and used as a scalar in the next Job
> (Job2), you can do it by reading and creating the object(s) manually
> from Job1's end, and then using it via a configuration property in
> Job2. This is straightforward.
> 
> But what exact type of passing are you talking about? It would be much
> easier to answer that if we know.
> 
> On Tue, Jan 25, 2011 at 8:00 PM, Joan <jo...@gmail.com> wrote:
> > Hi,
> >
> > I would like pass one object from job1 to job2
> >
> > Someone can I help me, please?
> >
> > Thanks
> >
> > Joan
> >
> 
> 
> 
> --
> Harsh J
> www.harshj.com
> 

Re: How to pass object between jobs

[Joan <jo...@gmail.com>]

Hi harsh

I want to create many pairs <k,v> where v will be custom object so this will
happen when my reduce's finished, so the output from job1 will be file with
<k,v> = <Text,MyObject>

Then In second job I want to get MyObject from first job, ¿you know? So now,
I have:

job1:

<Input,Output> <MyObject, Text>

job2;
<Input,Output> <Text, MyObject>

Then I have to convert Text object, from first job, to MyObject in second
job and I wan't to do this, I want to can work with MyObject directly.

Thanks,

Joan


2011/1/25 Harsh J <qw...@gmail.com>

> Could you describe your need with an example?
>
> If you want the output of a Job (Job1), say being a single file with a
> single value in it, to be read and used as a scalar in the next Job
> (Job2), you can do it by reading and creating the object(s) manually
> from Job1's end, and then using it via a configuration property in
> Job2. This is straightforward.
>
> But what exact type of passing are you talking about? It would be much
> easier to answer that if we know.
>
> On Tue, Jan 25, 2011 at 8:00 PM, Joan <jo...@gmail.com> wrote:
> > Hi,
> >
> > I would like pass one object from job1 to job2
> >
> > Someone can I help me, please?
> >
> > Thanks
> >
> > Joan
> >
>
>
>
> --
> Harsh J
> www.harshj.com
>

Re: How to pass object between jobs

[Harsh J <qw...@gmail.com>]

Could you describe your need with an example?

If you want the output of a Job (Job1), say being a single file with a
single value in it, to be read and used as a scalar in the next Job
(Job2), you can do it by reading and creating the object(s) manually
from Job1's end, and then using it via a configuration property in
Job2. This is straightforward.

But what exact type of passing are you talking about? It would be much
easier to answer that if we know.

On Tue, Jan 25, 2011 at 8:00 PM, Joan <jo...@gmail.com> wrote:
> Hi,
>
> I would like pass one object from job1 to job2
>
> Someone can I help me, please?
>
> Thanks
>
> Joan
>



-- 
Harsh J
www.harshj.com