You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Raghavendra Chandra <ra...@gmail.com> on 2014/12/05 06:16:01 UTC
Query regarding the object creation in map reduce code
Hi All,
I need your help in writing the map reduce program in Java.
I am creating a mapper and reducer classes for reading and processing a log
file.
I also have many other class files which acts as supporting classes to
mapper and will be instantiated from mapper class within the map function.
*Problem statement : *
Since there are 20 other objects which will be instantiated from mapper
class within the map function, we think this could create a performance hit
because of multiple object creation .
Please let us know what could be best approach/design to instantiate these
20 classes from Mapper class without compromising on the performance.
Your suggestions/comments are welcome.
Thanks and regards,
Raghav Chandra
Re: Query regarding the object creation in map reduce code
Posted by Gautam Hegde <ga...@aralitechnologies.com>.
Hi Raghavendra,
1. Make your objects reusable.
2. Instantiate these Reusable objects in the setup method of the
Mapper/Reducer, so that these objects are instantiated only once.
With Regards,
Gautam Hegde
On Fri, Dec 5, 2014 at 10:46 AM, Raghavendra Chandra <
raghavchandra.learning@gmail.com> wrote:
> Hi All,
>
> I need your help in writing the map reduce program in Java.
>
> I am creating a mapper and reducer classes for reading and processing a
> log file.
>
> I also have many other class files which acts as supporting classes to
> mapper and will be instantiated from mapper class within the map function.
>
>
> *Problem statement : *
>
> Since there are 20 other objects which will be instantiated from mapper
> class within the map function, we think this could create a performance hit
> because of multiple object creation .
>
> Please let us know what could be best approach/design to instantiate these
> 20 classes from Mapper class without compromising on the performance.
>
> Your suggestions/comments are welcome.
>
> Thanks and regards,
> Raghav Chandra
>
Re: Query regarding the object creation in map reduce code
Posted by Gautam Hegde <ga...@aralitechnologies.com>.
Hi Raghavendra,
1. Make your objects reusable.
2. Instantiate these Reusable objects in the setup method of the
Mapper/Reducer, so that these objects are instantiated only once.
With Regards,
Gautam Hegde
On Fri, Dec 5, 2014 at 10:46 AM, Raghavendra Chandra <
raghavchandra.learning@gmail.com> wrote:
> Hi All,
>
> I need your help in writing the map reduce program in Java.
>
> I am creating a mapper and reducer classes for reading and processing a
> log file.
>
> I also have many other class files which acts as supporting classes to
> mapper and will be instantiated from mapper class within the map function.
>
>
> *Problem statement : *
>
> Since there are 20 other objects which will be instantiated from mapper
> class within the map function, we think this could create a performance hit
> because of multiple object creation .
>
> Please let us know what could be best approach/design to instantiate these
> 20 classes from Mapper class without compromising on the performance.
>
> Your suggestions/comments are welcome.
>
> Thanks and regards,
> Raghav Chandra
>
Re: Query regarding the object creation in map reduce code
Posted by Gautam Hegde <ga...@aralitechnologies.com>.
Hi Raghavendra,
1. Make your objects reusable.
2. Instantiate these Reusable objects in the setup method of the
Mapper/Reducer, so that these objects are instantiated only once.
With Regards,
Gautam Hegde
On Fri, Dec 5, 2014 at 10:46 AM, Raghavendra Chandra <
raghavchandra.learning@gmail.com> wrote:
> Hi All,
>
> I need your help in writing the map reduce program in Java.
>
> I am creating a mapper and reducer classes for reading and processing a
> log file.
>
> I also have many other class files which acts as supporting classes to
> mapper and will be instantiated from mapper class within the map function.
>
>
> *Problem statement : *
>
> Since there are 20 other objects which will be instantiated from mapper
> class within the map function, we think this could create a performance hit
> because of multiple object creation .
>
> Please let us know what could be best approach/design to instantiate these
> 20 classes from Mapper class without compromising on the performance.
>
> Your suggestions/comments are welcome.
>
> Thanks and regards,
> Raghav Chandra
>
Re: Query regarding the object creation in map reduce code
Posted by Gautam Hegde <ga...@aralitechnologies.com>.
Hi Raghavendra,
1. Make your objects reusable.
2. Instantiate these Reusable objects in the setup method of the
Mapper/Reducer, so that these objects are instantiated only once.
With Regards,
Gautam Hegde
On Fri, Dec 5, 2014 at 10:46 AM, Raghavendra Chandra <
raghavchandra.learning@gmail.com> wrote:
> Hi All,
>
> I need your help in writing the map reduce program in Java.
>
> I am creating a mapper and reducer classes for reading and processing a
> log file.
>
> I also have many other class files which acts as supporting classes to
> mapper and will be instantiated from mapper class within the map function.
>
>
> *Problem statement : *
>
> Since there are 20 other objects which will be instantiated from mapper
> class within the map function, we think this could create a performance hit
> because of multiple object creation .
>
> Please let us know what could be best approach/design to instantiate these
> 20 classes from Mapper class without compromising on the performance.
>
> Your suggestions/comments are welcome.
>
> Thanks and regards,
> Raghav Chandra
>