You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by unmesha sreeveni <un...@gmail.com> on 2014/05/21 09:33:47 UTC
Issue with conf.set and conf.get method
Hi,
I am having an issue with conf.set and conf.get method
Driver
Configuration conf=new Configuration();
conf.set("delimiter",args[2]); //File delimiter as user argument
Map/Reduce
Configuration conf = context.getConfiguration();
String delim = conf.get("delimiter");
All things works fine with this.I am able to get the delimiter("," ";"
.....) and process accordingly except "TAB"
If I give
1. \t as an argument it wont work any operations
eg: will not be able to do
1. StringTokenizer st = new StringTokenizer(value.toString,delim)
<but works for split>
<String[] parts = value.toString.split(delim);>
2. String classLabel =
value.toString.substring(value.toString.lastIndexOf(delim)+1);
2. "\t" as argument also wont work
3. \\t and "\\t" also wont work
4. " " this WORKS FINE as an argument.
Anybody came across with this issue?
If so can any one tell me a workaround.
Regards
Unmesha
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Regards,
*Stanley Shi,*
On Thu, May 22, 2014 at 10:46 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> seems my guess is correct; I mean in your program, you can call:
>
>
> *hadoop jar myjar.jar input output " " *
>
> *instead you use:*
>
> *hadoop jar myjar.jar input output TAB*
>
> *or *
>
>
> *hadoop jar myjar.jar input output WHITESPACE*
> *and then in your program, *
> now you use: conf.set("delimiter",args[2]);
> instead you can use:
> if (args[2] == "TAG") conf.set("delimiter","\t");
>
>
>
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, May 21, 2014 at 7:53 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, May 21, 2014 at 3:37 PM, Stanley Shi <ss...@gopivotal.com> wrote:
>>
>>> Are you trying to pass arguments from user input? reading input from
>>> stdin?
>>>
>>> I suggest you use some special characters to express; for example, let
>>> args[2] == "TAB" and in your program, you can convert this "TAB" to the
>>> real delimeter you want to use ("\t")
>>>
>> *Sorry I did nt get you*
>> *I am running my job like this*
>> * hadoop jar myjar.jar input output ","*
>> *or*
>> *hadoop jar myjar.jar input output " "*
>> *And i am accessing these delimiters in mapper for spliting my inputs.*
>>
>>>
>>> Regards,
>>> *Stanley Shi,*
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <unmeshabiju@gmail.com
>>> > wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> I am having an issue with conf.set and conf.get method
>>>> Driver
>>>> Configuration conf=new Configuration();
>>>> conf.set("delimiter",args[2]); //File delimiter as user argument
>>>>
>>>> Map/Reduce
>>>> Configuration conf = context.getConfiguration();
>>>> String delim = conf.get("delimiter");
>>>>
>>>> All things works fine with this.I am able to get the delimiter("," ";"
>>>> .....) and process accordingly except "TAB"
>>>>
>>>> If I give
>>>> 1. \t as an argument it wont work any operations
>>>> eg: will not be able to do
>>>> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
>>>> <but works for split>
>>>> <String[] parts = value.toString.split(delim);>
>>>> 2. String classLabel =
>>>> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>>>>
>>>> 2. "\t" as argument also wont work
>>>> 3. \\t and "\\t" also wont work
>>>> 4. " " this WORKS FINE as an argument.
>>>>
>>>> Anybody came across with this issue?
>>>> If so can any one tell me a workaround.
>>>>
>>>> Regards
>>>> Unmesha
>>>>
>>>> --
>>>> *Thanks & Regards *
>>>>
>>>>
>>>> *Unmesha Sreeveni U.B*
>>>> *Hadoop, Bigdata Developer*
>>>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>>>> http://www.unmeshasreeveni.blogspot.in/
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> *Thanks & Regards *
>>
>>
>> *Unmesha Sreeveni U.B*
>> *Hadoop, Bigdata Developer*
>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Regards,
*Stanley Shi,*
On Thu, May 22, 2014 at 10:46 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> seems my guess is correct; I mean in your program, you can call:
>
>
> *hadoop jar myjar.jar input output " " *
>
> *instead you use:*
>
> *hadoop jar myjar.jar input output TAB*
>
> *or *
>
>
> *hadoop jar myjar.jar input output WHITESPACE*
> *and then in your program, *
> now you use: conf.set("delimiter",args[2]);
> instead you can use:
> if (args[2] == "TAG") conf.set("delimiter","\t");
>
>
>
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, May 21, 2014 at 7:53 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, May 21, 2014 at 3:37 PM, Stanley Shi <ss...@gopivotal.com> wrote:
>>
>>> Are you trying to pass arguments from user input? reading input from
>>> stdin?
>>>
>>> I suggest you use some special characters to express; for example, let
>>> args[2] == "TAB" and in your program, you can convert this "TAB" to the
>>> real delimeter you want to use ("\t")
>>>
>> *Sorry I did nt get you*
>> *I am running my job like this*
>> * hadoop jar myjar.jar input output ","*
>> *or*
>> *hadoop jar myjar.jar input output " "*
>> *And i am accessing these delimiters in mapper for spliting my inputs.*
>>
>>>
>>> Regards,
>>> *Stanley Shi,*
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <unmeshabiju@gmail.com
>>> > wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> I am having an issue with conf.set and conf.get method
>>>> Driver
>>>> Configuration conf=new Configuration();
>>>> conf.set("delimiter",args[2]); //File delimiter as user argument
>>>>
>>>> Map/Reduce
>>>> Configuration conf = context.getConfiguration();
>>>> String delim = conf.get("delimiter");
>>>>
>>>> All things works fine with this.I am able to get the delimiter("," ";"
>>>> .....) and process accordingly except "TAB"
>>>>
>>>> If I give
>>>> 1. \t as an argument it wont work any operations
>>>> eg: will not be able to do
>>>> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
>>>> <but works for split>
>>>> <String[] parts = value.toString.split(delim);>
>>>> 2. String classLabel =
>>>> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>>>>
>>>> 2. "\t" as argument also wont work
>>>> 3. \\t and "\\t" also wont work
>>>> 4. " " this WORKS FINE as an argument.
>>>>
>>>> Anybody came across with this issue?
>>>> If so can any one tell me a workaround.
>>>>
>>>> Regards
>>>> Unmesha
>>>>
>>>> --
>>>> *Thanks & Regards *
>>>>
>>>>
>>>> *Unmesha Sreeveni U.B*
>>>> *Hadoop, Bigdata Developer*
>>>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>>>> http://www.unmeshasreeveni.blogspot.in/
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> *Thanks & Regards *
>>
>>
>> *Unmesha Sreeveni U.B*
>> *Hadoop, Bigdata Developer*
>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Regards,
*Stanley Shi,*
On Thu, May 22, 2014 at 10:46 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> seems my guess is correct; I mean in your program, you can call:
>
>
> *hadoop jar myjar.jar input output " " *
>
> *instead you use:*
>
> *hadoop jar myjar.jar input output TAB*
>
> *or *
>
>
> *hadoop jar myjar.jar input output WHITESPACE*
> *and then in your program, *
> now you use: conf.set("delimiter",args[2]);
> instead you can use:
> if (args[2] == "TAG") conf.set("delimiter","\t");
>
>
>
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, May 21, 2014 at 7:53 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, May 21, 2014 at 3:37 PM, Stanley Shi <ss...@gopivotal.com> wrote:
>>
>>> Are you trying to pass arguments from user input? reading input from
>>> stdin?
>>>
>>> I suggest you use some special characters to express; for example, let
>>> args[2] == "TAB" and in your program, you can convert this "TAB" to the
>>> real delimeter you want to use ("\t")
>>>
>> *Sorry I did nt get you*
>> *I am running my job like this*
>> * hadoop jar myjar.jar input output ","*
>> *or*
>> *hadoop jar myjar.jar input output " "*
>> *And i am accessing these delimiters in mapper for spliting my inputs.*
>>
>>>
>>> Regards,
>>> *Stanley Shi,*
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <unmeshabiju@gmail.com
>>> > wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> I am having an issue with conf.set and conf.get method
>>>> Driver
>>>> Configuration conf=new Configuration();
>>>> conf.set("delimiter",args[2]); //File delimiter as user argument
>>>>
>>>> Map/Reduce
>>>> Configuration conf = context.getConfiguration();
>>>> String delim = conf.get("delimiter");
>>>>
>>>> All things works fine with this.I am able to get the delimiter("," ";"
>>>> .....) and process accordingly except "TAB"
>>>>
>>>> If I give
>>>> 1. \t as an argument it wont work any operations
>>>> eg: will not be able to do
>>>> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
>>>> <but works for split>
>>>> <String[] parts = value.toString.split(delim);>
>>>> 2. String classLabel =
>>>> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>>>>
>>>> 2. "\t" as argument also wont work
>>>> 3. \\t and "\\t" also wont work
>>>> 4. " " this WORKS FINE as an argument.
>>>>
>>>> Anybody came across with this issue?
>>>> If so can any one tell me a workaround.
>>>>
>>>> Regards
>>>> Unmesha
>>>>
>>>> --
>>>> *Thanks & Regards *
>>>>
>>>>
>>>> *Unmesha Sreeveni U.B*
>>>> *Hadoop, Bigdata Developer*
>>>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>>>> http://www.unmeshasreeveni.blogspot.in/
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> *Thanks & Regards *
>>
>>
>> *Unmesha Sreeveni U.B*
>> *Hadoop, Bigdata Developer*
>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Regards,
*Stanley Shi,*
On Thu, May 22, 2014 at 10:46 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> seems my guess is correct; I mean in your program, you can call:
>
>
> *hadoop jar myjar.jar input output " " *
>
> *instead you use:*
>
> *hadoop jar myjar.jar input output TAB*
>
> *or *
>
>
> *hadoop jar myjar.jar input output WHITESPACE*
> *and then in your program, *
> now you use: conf.set("delimiter",args[2]);
> instead you can use:
> if (args[2] == "TAG") conf.set("delimiter","\t");
>
>
>
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, May 21, 2014 at 7:53 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, May 21, 2014 at 3:37 PM, Stanley Shi <ss...@gopivotal.com> wrote:
>>
>>> Are you trying to pass arguments from user input? reading input from
>>> stdin?
>>>
>>> I suggest you use some special characters to express; for example, let
>>> args[2] == "TAB" and in your program, you can convert this "TAB" to the
>>> real delimeter you want to use ("\t")
>>>
>> *Sorry I did nt get you*
>> *I am running my job like this*
>> * hadoop jar myjar.jar input output ","*
>> *or*
>> *hadoop jar myjar.jar input output " "*
>> *And i am accessing these delimiters in mapper for spliting my inputs.*
>>
>>>
>>> Regards,
>>> *Stanley Shi,*
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <unmeshabiju@gmail.com
>>> > wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> I am having an issue with conf.set and conf.get method
>>>> Driver
>>>> Configuration conf=new Configuration();
>>>> conf.set("delimiter",args[2]); //File delimiter as user argument
>>>>
>>>> Map/Reduce
>>>> Configuration conf = context.getConfiguration();
>>>> String delim = conf.get("delimiter");
>>>>
>>>> All things works fine with this.I am able to get the delimiter("," ";"
>>>> .....) and process accordingly except "TAB"
>>>>
>>>> If I give
>>>> 1. \t as an argument it wont work any operations
>>>> eg: will not be able to do
>>>> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
>>>> <but works for split>
>>>> <String[] parts = value.toString.split(delim);>
>>>> 2. String classLabel =
>>>> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>>>>
>>>> 2. "\t" as argument also wont work
>>>> 3. \\t and "\\t" also wont work
>>>> 4. " " this WORKS FINE as an argument.
>>>>
>>>> Anybody came across with this issue?
>>>> If so can any one tell me a workaround.
>>>>
>>>> Regards
>>>> Unmesha
>>>>
>>>> --
>>>> *Thanks & Regards *
>>>>
>>>>
>>>> *Unmesha Sreeveni U.B*
>>>> *Hadoop, Bigdata Developer*
>>>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>>>> http://www.unmeshasreeveni.blogspot.in/
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> *Thanks & Regards *
>>
>>
>> *Unmesha Sreeveni U.B*
>> *Hadoop, Bigdata Developer*
>> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Are you trying to pass arguments from user input? reading input from stdin?
I suggest you use some special characters to express; for example, let
args[2] == "TAB" and in your program, you can convert this "TAB" to the
real delimeter you want to use ("\t")
Regards,
*Stanley Shi,*
On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
> Hi,
>
> I am having an issue with conf.set and conf.get method
> Driver
> Configuration conf=new Configuration();
> conf.set("delimiter",args[2]); //File delimiter as user argument
>
> Map/Reduce
> Configuration conf = context.getConfiguration();
> String delim = conf.get("delimiter");
>
> All things works fine with this.I am able to get the delimiter("," ";"
> .....) and process accordingly except "TAB"
>
> If I give
> 1. \t as an argument it wont work any operations
> eg: will not be able to do
> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
> <but works for split>
> <String[] parts = value.toString.split(delim);>
> 2. String classLabel =
> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>
> 2. "\t" as argument also wont work
> 3. \\t and "\\t" also wont work
> 4. " " this WORKS FINE as an argument.
>
> Anybody came across with this issue?
> If so can any one tell me a workaround.
>
> Regards
> Unmesha
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Are you trying to pass arguments from user input? reading input from stdin?
I suggest you use some special characters to express; for example, let
args[2] == "TAB" and in your program, you can convert this "TAB" to the
real delimeter you want to use ("\t")
Regards,
*Stanley Shi,*
On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
> Hi,
>
> I am having an issue with conf.set and conf.get method
> Driver
> Configuration conf=new Configuration();
> conf.set("delimiter",args[2]); //File delimiter as user argument
>
> Map/Reduce
> Configuration conf = context.getConfiguration();
> String delim = conf.get("delimiter");
>
> All things works fine with this.I am able to get the delimiter("," ";"
> .....) and process accordingly except "TAB"
>
> If I give
> 1. \t as an argument it wont work any operations
> eg: will not be able to do
> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
> <but works for split>
> <String[] parts = value.toString.split(delim);>
> 2. String classLabel =
> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>
> 2. "\t" as argument also wont work
> 3. \\t and "\\t" also wont work
> 4. " " this WORKS FINE as an argument.
>
> Anybody came across with this issue?
> If so can any one tell me a workaround.
>
> Regards
> Unmesha
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Are you trying to pass arguments from user input? reading input from stdin?
I suggest you use some special characters to express; for example, let
args[2] == "TAB" and in your program, you can convert this "TAB" to the
real delimeter you want to use ("\t")
Regards,
*Stanley Shi,*
On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
> Hi,
>
> I am having an issue with conf.set and conf.get method
> Driver
> Configuration conf=new Configuration();
> conf.set("delimiter",args[2]); //File delimiter as user argument
>
> Map/Reduce
> Configuration conf = context.getConfiguration();
> String delim = conf.get("delimiter");
>
> All things works fine with this.I am able to get the delimiter("," ";"
> .....) and process accordingly except "TAB"
>
> If I give
> 1. \t as an argument it wont work any operations
> eg: will not be able to do
> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
> <but works for split>
> <String[] parts = value.toString.split(delim);>
> 2. String classLabel =
> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>
> 2. "\t" as argument also wont work
> 3. \\t and "\\t" also wont work
> 4. " " this WORKS FINE as an argument.
>
> Anybody came across with this issue?
> If so can any one tell me a workaround.
>
> Regards
> Unmesha
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: Issue with conf.set and conf.get method
Posted by Stanley Shi <ss...@gopivotal.com>.
Are you trying to pass arguments from user input? reading input from stdin?
I suggest you use some special characters to express; for example, let
args[2] == "TAB" and in your program, you can convert this "TAB" to the
real delimeter you want to use ("\t")
Regards,
*Stanley Shi,*
On Wed, May 21, 2014 at 3:33 PM, unmesha sreeveni <un...@gmail.com>wrote:
>
> Hi,
>
> I am having an issue with conf.set and conf.get method
> Driver
> Configuration conf=new Configuration();
> conf.set("delimiter",args[2]); //File delimiter as user argument
>
> Map/Reduce
> Configuration conf = context.getConfiguration();
> String delim = conf.get("delimiter");
>
> All things works fine with this.I am able to get the delimiter("," ";"
> .....) and process accordingly except "TAB"
>
> If I give
> 1. \t as an argument it wont work any operations
> eg: will not be able to do
> 1. StringTokenizer st = new StringTokenizer(value.toString,delim)
> <but works for split>
> <String[] parts = value.toString.split(delim);>
> 2. String classLabel =
> value.toString.substring(value.toString.lastIndexOf(delim)+1);
>
> 2. "\t" as argument also wont work
> 3. \\t and "\\t" also wont work
> 4. " " this WORKS FINE as an argument.
>
> Anybody came across with this issue?
> If so can any one tell me a workaround.
>
> Regards
> Unmesha
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>