Re: Passing Constants from One Job to the Next

[Aaron Kimball <aa...@cloudera.com>]

The Mapper and Reducer interfaces both provide a method 'void
configure(JobConf conf) throws IOException'; if you extend MapReduceBase,
this will provide a dummy implementation of configure(). You can add your
own implementation; it will be called before the first call to map() or
reduce(). You can read your initialization data at this time.

- Aaron

On Thu, Oct 30, 2008 at 4:02 PM, Erik Holstad <er...@gmail.com> wrote:

> Hi!
> Is there a way of using the value read in the configure() in the Map or
> Reduce phase?
>
> Erik
>
> On Thu, Oct 23, 2008 at 2:40 AM, Aaron Kimball <aa...@cloudera.com> wrote:
>
> > See Configuration.setInt() in the API. (JobConf inherits from
> > Configuration). You can read it back in the configure() method of your
> > mappers/reducers
> > - Aaron
> >
> > On Wed, Oct 22, 2008 at 3:03 PM, Yih Sun Khoo <ys...@gmail.com> wrote:
> >
> > > Are you saying that I can pass, say, a single integer constant with
> > either
> > > of these three: JobConf? A HDFS file? DistributedCache?
> > > Or are you asking if I can pass given the context of: JobConf? A HDFS
> > file?
> > > DistributedCache?
> > > I'm thinking of how to pass a single int so from one Jobconf to the
> next
> > >
> > > On Wed, Oct 22, 2008 at 2:57 PM, Arun C Murthy <ac...@yahoo-inc.com>
> > wrote:
> > >
> > > >
> > > > On Oct 22, 2008, at 2:52 PM, Yih Sun Khoo wrote:
> > > >
> > > >  I like to hear some good ways of passing constants from one job to
> the
> > > >> next.
> > > >>
> > > >
> > > > Unless I'm missing something: JobConf? A HDFS file? DistributedCache?
> > > >
> > > > Arun
> > > >
> > > >
> > > >
> > > >> These are some ways that I can think of:
> > > >> 1)  The obvious solution is to carry the constant as part of your
> > value
> > > >> from
> > > >> one job to the next, but that would mean every value would hold that
> > > >> constant
> > > >> 2)  Use the reporter as a hack so that you can set the status
> message
> > > and
> > > >> then get the status message back when u need the constant
> > > >>
> > > >> Any other ideas?  (Also please do not include code)
> > > >>
> > > >
> > > >
> > >
> >
>