You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2013/08/07 01:46:46 UTC
Passing arguments in hadoop
Hi,
I am trying to pass a parameter to multiple mappers
So, I do this in my driver
conf.set("delimiter", args[3]);
In mapper1, I am retreiving this as:
Configuration conf = context.getConfiguration();
String[] values = value.toString().split(conf.get("delimiter"));
and same is my mapper2
But I get this error:
13/08/06 16:38:09 INFO mapred.JobClient: Task Id :
attempt_201306191416_1384_m_000001_0, Status : FAILED
java.lang.NullPointerException
at java.util.regex.Pattern.<init>(Pattern.java:1149)
at java.util.regex.Pattern.compile(Pattern.java:840)
at java.lang.String.split(String.java:2304)
at java.lang.String.split(String.java:2346)
at DataSourceOneMapper.map(DataSourceOneMapper.java:22)
at .DataSourceOneMapper.map(DataSourceOneMapper.java:1)
And the line 22 corresponds to this conf.get("...") line
What am i doing wrong?
Also, just to add, in this particular case delimiter is "@"
So, if i do
String[] values = value.toString().split("@");
everything works fine??
How do i resolve this?
Thanks
Re: Passing arguments in hadoop
Posted by jamal sasha <ja...@gmail.com>.
Never mind guys.
I had a typo when I was trying to set configuration param.
Sorry.
On Tue, Aug 6, 2013 at 4:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I am trying to pass a parameter to multiple mappers
>
> So, I do this in my driver
>
> conf.set("delimiter", args[3]);
>
> In mapper1, I am retreiving this as:
>
> Configuration conf = context.getConfiguration();
> String[] values = value.toString().split(conf.get("delimiter"));
>
> and same is my mapper2
>
>
> But I get this error:
> 13/08/06 16:38:09 INFO mapred.JobClient: Task Id :
> attempt_201306191416_1384_m_000001_0, Status : FAILED
> java.lang.NullPointerException
> at java.util.regex.Pattern.<init>(Pattern.java:1149)
> at java.util.regex.Pattern.compile(Pattern.java:840)
> at java.lang.String.split(String.java:2304)
> at java.lang.String.split(String.java:2346)
> at DataSourceOneMapper.map(DataSourceOneMapper.java:22)
> at .DataSourceOneMapper.map(DataSourceOneMapper.java:1)
>
>
> And the line 22 corresponds to this conf.get("...") line
>
> What am i doing wrong?
> Also, just to add, in this particular case delimiter is "@"
>
> So, if i do
>
> String[] values = value.toString().split("@");
>
> everything works fine??
> How do i resolve this?
> Thanks
>
>
>
Re: Passing arguments in hadoop
Posted by jamal sasha <ja...@gmail.com>.
Never mind guys.
I had a typo when I was trying to set configuration param.
Sorry.
On Tue, Aug 6, 2013 at 4:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I am trying to pass a parameter to multiple mappers
>
> So, I do this in my driver
>
> conf.set("delimiter", args[3]);
>
> In mapper1, I am retreiving this as:
>
> Configuration conf = context.getConfiguration();
> String[] values = value.toString().split(conf.get("delimiter"));
>
> and same is my mapper2
>
>
> But I get this error:
> 13/08/06 16:38:09 INFO mapred.JobClient: Task Id :
> attempt_201306191416_1384_m_000001_0, Status : FAILED
> java.lang.NullPointerException
> at java.util.regex.Pattern.<init>(Pattern.java:1149)
> at java.util.regex.Pattern.compile(Pattern.java:840)
> at java.lang.String.split(String.java:2304)
> at java.lang.String.split(String.java:2346)
> at DataSourceOneMapper.map(DataSourceOneMapper.java:22)
> at .DataSourceOneMapper.map(DataSourceOneMapper.java:1)
>
>
> And the line 22 corresponds to this conf.get("...") line
>
> What am i doing wrong?
> Also, just to add, in this particular case delimiter is "@"
>
> So, if i do
>
> String[] values = value.toString().split("@");
>
> everything works fine??
> How do i resolve this?
> Thanks
>
>
>
Re: Passing arguments in hadoop
Posted by jamal sasha <ja...@gmail.com>.
Never mind guys.
I had a typo when I was trying to set configuration param.
Sorry.
On Tue, Aug 6, 2013 at 4:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I am trying to pass a parameter to multiple mappers
>
> So, I do this in my driver
>
> conf.set("delimiter", args[3]);
>
> In mapper1, I am retreiving this as:
>
> Configuration conf = context.getConfiguration();
> String[] values = value.toString().split(conf.get("delimiter"));
>
> and same is my mapper2
>
>
> But I get this error:
> 13/08/06 16:38:09 INFO mapred.JobClient: Task Id :
> attempt_201306191416_1384_m_000001_0, Status : FAILED
> java.lang.NullPointerException
> at java.util.regex.Pattern.<init>(Pattern.java:1149)
> at java.util.regex.Pattern.compile(Pattern.java:840)
> at java.lang.String.split(String.java:2304)
> at java.lang.String.split(String.java:2346)
> at DataSourceOneMapper.map(DataSourceOneMapper.java:22)
> at .DataSourceOneMapper.map(DataSourceOneMapper.java:1)
>
>
> And the line 22 corresponds to this conf.get("...") line
>
> What am i doing wrong?
> Also, just to add, in this particular case delimiter is "@"
>
> So, if i do
>
> String[] values = value.toString().split("@");
>
> everything works fine??
> How do i resolve this?
> Thanks
>
>
>
Re: Passing arguments in hadoop
Posted by jamal sasha <ja...@gmail.com>.
Never mind guys.
I had a typo when I was trying to set configuration param.
Sorry.
On Tue, Aug 6, 2013 at 4:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> I am trying to pass a parameter to multiple mappers
>
> So, I do this in my driver
>
> conf.set("delimiter", args[3]);
>
> In mapper1, I am retreiving this as:
>
> Configuration conf = context.getConfiguration();
> String[] values = value.toString().split(conf.get("delimiter"));
>
> and same is my mapper2
>
>
> But I get this error:
> 13/08/06 16:38:09 INFO mapred.JobClient: Task Id :
> attempt_201306191416_1384_m_000001_0, Status : FAILED
> java.lang.NullPointerException
> at java.util.regex.Pattern.<init>(Pattern.java:1149)
> at java.util.regex.Pattern.compile(Pattern.java:840)
> at java.lang.String.split(String.java:2304)
> at java.lang.String.split(String.java:2346)
> at DataSourceOneMapper.map(DataSourceOneMapper.java:22)
> at .DataSourceOneMapper.map(DataSourceOneMapper.java:1)
>
>
> And the line 22 corresponds to this conf.get("...") line
>
> What am i doing wrong?
> Also, just to add, in this particular case delimiter is "@"
>
> So, if i do
>
> String[] values = value.toString().split("@");
>
> everything works fine??
> How do i resolve this?
> Thanks
>
>
>