-m Number of Mappers

["Martin, Nick" <Ni...@pssd.com>]

Kind of a newbie question but one that's been stumping me...

I'm noticing when I set the number of mappers to use (i.e. -m in CLI Sqoop command) that isn't the number of mappers my cluster is using to execute the Sqoop job.

For instance, I just ran an import with -m 8 specified and the number of mappers used is 4 until the import hits 50% and then drops to 3 for the rest of the import process (my cluster has 16 available map slots; no other concurrent jobs processing).

I read through the "Controlling Parallelism" piece of the user guide but didn't seem to find my answer.

Thanks in advanced for the help,
Nick

Re: -m Number of Mappers

[Abraham Elmahrek <ab...@cloudera.com>]

Nick,

Could you provide your sqoop command?

Also, checkout the Sqoop Cookbook on a really good description on how
--num-mappers option works:
http://books.google.com/books?id=3qKAW063BhoC&pg=PA19&lpg=PA19&dq=controlling+parallelism+sqoop&source=bl&ots=HWYeBk36l5&sig=Cy7-AOHQO3ON7R0Kv4Pobdj-FNg&hl=en&sa=X&ei=oo-rUuGXNYrmoAS5sIK4BQ&ved=0CDIQ6AEwAQ#v=onepage&q=controlling%20parallelism%20sqoop&f=false
.


On Tue, Dec 10, 2013 at 6:36 PM, Martin, Nick <Ni...@pssd.com> wrote:

>  Kind of a newbie question but one that’s been stumping me…
>
>
>
> I’m noticing when I set the number of mappers to use (i.e. –m in CLI Sqoop
> command) that isn’t the number of mappers my cluster is using to execute
> the Sqoop job.
>
>
>
> For instance, I just ran an import with –m 8 specified and the number of
> mappers used is 4 until the import hits 50% and then drops to 3 for the
> rest of the import process (my cluster has 16 available map slots; no other
> concurrent jobs processing).
>
>
>
> I read through the “Controlling Parallelism” piece of the user guide but
> didn’t seem to find my answer.
>
>
>
> Thanks in advanced for the help,
>
> Nick
>
>
>
>
>