You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by unmesha sreeveni <un...@gmail.com> on 2013/11/13 05:21:29 UTC
How to Set No of Mappers in program
How to set the no of mappers?
I am running a job with input file of 5000 records.but actually it is a
small file for hadoop..but i need to know whether it can be exceuted faster.
when i tried to run my job it took nearly 30 min to complete my job...
why so?
If i set my mappers to a lower count whether any reflection happens to my
job performance?
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: How to Set No of Mappers in program
Posted by Dieter De Witte <dr...@gmail.com>.
you can configure the max split size, since you have one mapper per splits
this might be the way to go..
2013/11/13 unmesha sreeveni <un...@gmail.com>
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Dieter De Witte <dr...@gmail.com>.
you can configure the max split size, since you have one mapper per splits
this might be the way to go..
2013/11/13 unmesha sreeveni <un...@gmail.com>
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Dieter De Witte <dr...@gmail.com>.
you can configure the max split size, since you have one mapper per splits
this might be the way to go..
2013/11/13 unmesha sreeveni <un...@gmail.com>
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
number of mappers depend on the number of dfs blocks you file spans across
, considering standard file input formatter. unlike reducers , where you
configure as how many reducers you want for your job.
thanks,
Rahul
On Tue, Nov 12, 2013 at 9:00 PM, unmesha sreeveni <un...@gmail.com>wrote:
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
number of mappers depend on the number of dfs blocks you file spans across
, considering standard file input formatter. unlike reducers , where you
configure as how many reducers you want for your job.
thanks,
Rahul
On Tue, Nov 12, 2013 at 9:00 PM, unmesha sreeveni <un...@gmail.com>wrote:
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
number of mappers depend on the number of dfs blocks you file spans across
, considering standard file input formatter. unlike reducers , where you
configure as how many reducers you want for your job.
thanks,
Rahul
On Tue, Nov 12, 2013 at 9:00 PM, unmesha sreeveni <un...@gmail.com>wrote:
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Dieter De Witte <dr...@gmail.com>.
you can configure the max split size, since you have one mapper per splits
this might be the way to go..
2013/11/13 unmesha sreeveni <un...@gmail.com>
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by Rahul Bhattacharjee <ra...@gmail.com>.
number of mappers depend on the number of dfs blocks you file spans across
, considering standard file input formatter. unlike reducers , where you
configure as how many reducers you want for your job.
thanks,
Rahul
On Tue, Nov 12, 2013 at 9:00 PM, unmesha sreeveni <un...@gmail.com>wrote:
> For my 5000 recorded file .. How many mappers will be running?
>
> And is there a way to know how my files are splited?
>
>
> On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
>
>> How to set the no of mappers?
>>
>> I am running a job with input file of 5000 records.but actually it is a
>> small file for hadoop..but i need to know whether it can be exceuted faster.
>>
>> when i tried to run my job it took nearly 30 min to complete my job...
>> why so?
>>
>> If i set my mappers to a lower count whether any reflection happens to my
>> job performance?
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: How to Set No of Mappers in program
Posted by unmesha sreeveni <un...@gmail.com>.
For my 5000 recorded file .. How many mappers will be running?
And is there a way to know how my files are splited?
On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
> How to set the no of mappers?
>
> I am running a job with input file of 5000 records.but actually it is a
> small file for hadoop..but i need to know whether it can be exceuted faster.
>
> when i tried to run my job it took nearly 30 min to complete my job...
> why so?
>
> If i set my mappers to a lower count whether any reflection happens to my
> job performance?
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: How to Set No of Mappers in program
Posted by unmesha sreeveni <un...@gmail.com>.
For my 5000 recorded file .. How many mappers will be running?
And is there a way to know how my files are splited?
On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
> How to set the no of mappers?
>
> I am running a job with input file of 5000 records.but actually it is a
> small file for hadoop..but i need to know whether it can be exceuted faster.
>
> when i tried to run my job it took nearly 30 min to complete my job...
> why so?
>
> If i set my mappers to a lower count whether any reflection happens to my
> job performance?
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: How to Set No of Mappers in program
Posted by unmesha sreeveni <un...@gmail.com>.
For my 5000 recorded file .. How many mappers will be running?
And is there a way to know how my files are splited?
On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
> How to set the no of mappers?
>
> I am running a job with input file of 5000 records.but actually it is a
> small file for hadoop..but i need to know whether it can be exceuted faster.
>
> when i tried to run my job it took nearly 30 min to complete my job...
> why so?
>
> If i set my mappers to a lower count whether any reflection happens to my
> job performance?
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: How to Set No of Mappers in program
Posted by unmesha sreeveni <un...@gmail.com>.
For my 5000 recorded file .. How many mappers will be running?
And is there a way to know how my files are splited?
On Wed, Nov 13, 2013 at 9:51 AM, unmesha sreeveni <un...@gmail.com>wrote:
> How to set the no of mappers?
>
> I am running a job with input file of 5000 records.but actually it is a
> small file for hadoop..but i need to know whether it can be exceuted faster.
>
> when i tried to run my job it took nearly 30 min to complete my job...
> why so?
>
> If i set my mappers to a lower count whether any reflection happens to my
> job performance?
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*