You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Ranjini Rathinam <ra...@gmail.com> on 2013/12/06 12:02:26 UTC
Hadoop-MapReduce
Hi,
How to read xml file via mapreduce and load them in hbase and hive using
java.
Please provide sample code.
I am using hadoop 0.20 version and java 1.6. Which parser version should be
used.
Thanks in advance.
Ranjini
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
First Option: Put the jar in $HADOOP_HOME/lib folder
And then run "hadoop classpath" command on your terminal to check
whether the jar has been added
Second OPtion: PUt the jar path in HADOOP_CLASSPATH variable (
hadoop-env.sh file) and restart your cluster..
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
First Option: Put the jar in $HADOOP_HOME/lib folder
And then run "hadoop classpath" command on your terminal to check
whether the jar has been added
Second OPtion: PUt the jar path in HADOOP_CLASSPATH variable (
hadoop-env.sh file) and restart your cluster..
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I want to know , when should i use Mapper , Reduce and Combiner.
What all methods are there in them.
Please suggest for study in detail. As I am fresher .
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:34 PM, unmesha sreeveni <un...@gmail.com>wrote:
> Ranjini can u pls check this. This is not perfect ..I simply did to
> check my xml data.
>
> https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
>
>
> On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
>> Hi,
>>
>> The driver class and my Mapper class i have used
>> org.apache.hadoop.mapreduce.lib
>>
>> and in the XmlInputFormat.java class also i have used the
>> org.apache.hadoop.mapreduce.lib
>>
>> but still iam getting this error.
>>
>> Please suggest.
>>
>> Thanks in advance
>>
>> Ranjini
>>
>> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>>
>>> Hello Ranjini,
>>> This error will come when you use mix and match newer and older API..
>>>
>>> You might have written program using newer API and the the XML input
>>> format is using older api..
>>> The older api has package structure of org.apache.hadoop.mapred
>>>
>>> The newer api has package structure package of
>>> org.apache.hadoop.mapreduce.lib
>>>
>>> Check out the XMLINputFormat.java, which package of FileInputFormat
>>> they have used...
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>>> <ra...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am using hadoop 0.20 version
>>> >
>>> > In that while exceuting the XmlInformat class
>>> > I am getting the error as
>>> >
>>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext,
>>> but
>>> > interface was excepted,."
>>> >
>>> > Please suggest to fix the error.
>>> >
>>> > Thanks in advance.
>>> >
>>> > Ranjini
>>> >
>>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> > wrote:
>>> >>
>>> >> hi,
>>> >>
>>> >> I have fixed the error , the code is running fine, but this code just
>>> >> split the part of the tag.
>>> >>
>>> >> i want to convert into text format so that i can load them into
>>> tables of
>>> >> hbase and hive.
>>> >>
>>> >> I have used the DOM Parser but this parser uses File as Object but
>>> hdfs
>>> >> uses FileSystem.
>>> >>
>>> >> Eg,
>>> >>
>>> >> File fXmlFile = new File("D:/elango/test.xml");
>>> >>
>>> >> System.out.println(g);
>>> >> DocumentBuilderFactory dbFactory =
>>> DocumentBuilderFactory.newInstance();
>>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>>> >> Document doc = dBuilder.parse(fXmlFile);
>>> >>
>>> >>
>>> >> This cant be used as hdfs, because hdfs path is accessed through
>>> >> FileSystem.
>>> >>
>>> >> I kindly request u to , Please suggest me to fix the above issue.
>>> >>
>>> >> Thanks in advance
>>> >>
>>> >> Ranjini R
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>>> >> <ra...@gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> ---------- Forwarded message ----------
>>> >>> From: Shekhar Sharma <sh...@gmail.com>
>>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> >>> Subject: Re: Hadoop-MapReduce
>>> >>> To: user@hadoop.apache.org
>>> >>> Cc: ssanyal@datameer.com
>>> >>>
>>> >>>
>>> >>> It does work i have used it long back..
>>> >>>
>>> >>> BTW if it is not working, write the custom input format and implement
>>> >>> your record reader. That would be far more easy than breaking your
>>> >>> head with others code.
>>> >>>
>>> >>> Break your problem in step:
>>> >>>
>>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> >>> single record for you...May be a record for you would be
>>> >>>
>>> >>> <person>
>>> >>> <fname>x</fname>
>>> >>> <lname>y</lname>
>>> >>> </person>
>>> >>>
>>> >>> (2) Implement a record reader that looks out for the starting and
>>> >>> ending person tag ( Checkout how RecordReader.java is written)
>>> >>>
>>> >>> (3) Once you got the contents between starting and ending tag, now
>>> you
>>> >>> can use a xml parser to parse the contents into an java object and
>>> >>> form your own key value pairs ( custom key and custom value)
>>> >>>
>>> >>>
>>> >>> Hope you have enough pointers to write the code.
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>> Som Shekhar Sharma
>>> >>> +91-8197243810
>>> >>>
>>> >>>
>>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> >>> wrote:
>>> >>> > Hi Subroto Sanyal,
>>> >>> >
>>> >>> > The link provided about xml, it does not work . The Class written
>>> >>> > XmlContent is not allowed in the XmlInputFormat.
>>> >>> >
>>> >>> > I request you to help , whether this scenaio some one has coded,
>>> and
>>> >>> > needed
>>> >>> > working code.
>>> >>> >
>>> >>> > I have written using SAX Parser too, but eventhough the jars are
>>> added
>>> >>> > in
>>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >>> >
>>> >>> > Please provide sample code for the same.
>>> >>> >
>>> >>> > Thanks in advance,
>>> >>> > Ranjini.R
>>> >>> >
>>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> >>> > <ra...@gmail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >>
>>> >>> >>>> Hi,
>>> >>> >>>>
>>> >>> >>>> As suggest by the link below , i have used for my program ,
>>> >>> >>>>
>>> >>> >>>> but i am facing the below issues, please help me to fix these
>>> error.
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>> >>>> override
>>> >>> >>>> abstract method
>>> >>> >>>>
>>> >>> >>>>
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>> >>>> <LongWritable, Text, Text, Text> {
>>> >>> >>>> ^
>>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>> >>>> abstract
>>> >>> >>>> and does not override abstract method
>>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>> >>>> ^
>>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>> >>>> 2 errors
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>> >>>>
>>> >>> >>>> Please suggest.
>>> >>> >>>>
>>> >>> >>>> Thanks in advance.
>>> >>> >>>>
>>> >>> >>>> Regrads,
>>> >>> >>>> Ranjini. R
>>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>> >>>> <ra...@gmail.com> wrote:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> ---------- Forwarded message ----------
>>> >>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>> >>>>> To: user@hadoop.apache.org
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Hi Ranjini,
>>> >>> >>>>>
>>> >>> >>>>> A good example to look into :
>>> >>> >>>>> http://www.undercloud.org/?p=408
>>> >>> >>>>>
>>> >>> >>>>> Cheers,
>>> >>> >>>>> Subroto Sanyal
>>> >>> >>>>>
>>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>> >>>>>
>>> >>> >>>>> Hi,
>>> >>> >>>>>
>>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>>> hive
>>> >>> >>>>> using java.
>>> >>> >>>>>
>>> >>> >>>>> Please provide sample code.
>>> >>> >>>>>
>>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>>> version
>>> >>> >>>>> should be used.
>>> >>> >>>>>
>>> >>> >>>>> Thanks in advance.
>>> >>> >>>>>
>>> >>> >>>>> Ranjini
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>
>>> >>> >>>
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I want to know , when should i use Mapper , Reduce and Combiner.
What all methods are there in them.
Please suggest for study in detail. As I am fresher .
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:34 PM, unmesha sreeveni <un...@gmail.com>wrote:
> Ranjini can u pls check this. This is not perfect ..I simply did to
> check my xml data.
>
> https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
>
>
> On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
>> Hi,
>>
>> The driver class and my Mapper class i have used
>> org.apache.hadoop.mapreduce.lib
>>
>> and in the XmlInputFormat.java class also i have used the
>> org.apache.hadoop.mapreduce.lib
>>
>> but still iam getting this error.
>>
>> Please suggest.
>>
>> Thanks in advance
>>
>> Ranjini
>>
>> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>>
>>> Hello Ranjini,
>>> This error will come when you use mix and match newer and older API..
>>>
>>> You might have written program using newer API and the the XML input
>>> format is using older api..
>>> The older api has package structure of org.apache.hadoop.mapred
>>>
>>> The newer api has package structure package of
>>> org.apache.hadoop.mapreduce.lib
>>>
>>> Check out the XMLINputFormat.java, which package of FileInputFormat
>>> they have used...
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>>> <ra...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am using hadoop 0.20 version
>>> >
>>> > In that while exceuting the XmlInformat class
>>> > I am getting the error as
>>> >
>>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext,
>>> but
>>> > interface was excepted,."
>>> >
>>> > Please suggest to fix the error.
>>> >
>>> > Thanks in advance.
>>> >
>>> > Ranjini
>>> >
>>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> > wrote:
>>> >>
>>> >> hi,
>>> >>
>>> >> I have fixed the error , the code is running fine, but this code just
>>> >> split the part of the tag.
>>> >>
>>> >> i want to convert into text format so that i can load them into
>>> tables of
>>> >> hbase and hive.
>>> >>
>>> >> I have used the DOM Parser but this parser uses File as Object but
>>> hdfs
>>> >> uses FileSystem.
>>> >>
>>> >> Eg,
>>> >>
>>> >> File fXmlFile = new File("D:/elango/test.xml");
>>> >>
>>> >> System.out.println(g);
>>> >> DocumentBuilderFactory dbFactory =
>>> DocumentBuilderFactory.newInstance();
>>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>>> >> Document doc = dBuilder.parse(fXmlFile);
>>> >>
>>> >>
>>> >> This cant be used as hdfs, because hdfs path is accessed through
>>> >> FileSystem.
>>> >>
>>> >> I kindly request u to , Please suggest me to fix the above issue.
>>> >>
>>> >> Thanks in advance
>>> >>
>>> >> Ranjini R
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>>> >> <ra...@gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> ---------- Forwarded message ----------
>>> >>> From: Shekhar Sharma <sh...@gmail.com>
>>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> >>> Subject: Re: Hadoop-MapReduce
>>> >>> To: user@hadoop.apache.org
>>> >>> Cc: ssanyal@datameer.com
>>> >>>
>>> >>>
>>> >>> It does work i have used it long back..
>>> >>>
>>> >>> BTW if it is not working, write the custom input format and implement
>>> >>> your record reader. That would be far more easy than breaking your
>>> >>> head with others code.
>>> >>>
>>> >>> Break your problem in step:
>>> >>>
>>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> >>> single record for you...May be a record for you would be
>>> >>>
>>> >>> <person>
>>> >>> <fname>x</fname>
>>> >>> <lname>y</lname>
>>> >>> </person>
>>> >>>
>>> >>> (2) Implement a record reader that looks out for the starting and
>>> >>> ending person tag ( Checkout how RecordReader.java is written)
>>> >>>
>>> >>> (3) Once you got the contents between starting and ending tag, now
>>> you
>>> >>> can use a xml parser to parse the contents into an java object and
>>> >>> form your own key value pairs ( custom key and custom value)
>>> >>>
>>> >>>
>>> >>> Hope you have enough pointers to write the code.
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>> Som Shekhar Sharma
>>> >>> +91-8197243810
>>> >>>
>>> >>>
>>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> >>> wrote:
>>> >>> > Hi Subroto Sanyal,
>>> >>> >
>>> >>> > The link provided about xml, it does not work . The Class written
>>> >>> > XmlContent is not allowed in the XmlInputFormat.
>>> >>> >
>>> >>> > I request you to help , whether this scenaio some one has coded,
>>> and
>>> >>> > needed
>>> >>> > working code.
>>> >>> >
>>> >>> > I have written using SAX Parser too, but eventhough the jars are
>>> added
>>> >>> > in
>>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >>> >
>>> >>> > Please provide sample code for the same.
>>> >>> >
>>> >>> > Thanks in advance,
>>> >>> > Ranjini.R
>>> >>> >
>>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> >>> > <ra...@gmail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >>
>>> >>> >>>> Hi,
>>> >>> >>>>
>>> >>> >>>> As suggest by the link below , i have used for my program ,
>>> >>> >>>>
>>> >>> >>>> but i am facing the below issues, please help me to fix these
>>> error.
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>> >>>> override
>>> >>> >>>> abstract method
>>> >>> >>>>
>>> >>> >>>>
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>> >>>> <LongWritable, Text, Text, Text> {
>>> >>> >>>> ^
>>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>> >>>> abstract
>>> >>> >>>> and does not override abstract method
>>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>> >>>> ^
>>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>> >>>> 2 errors
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>> >>>>
>>> >>> >>>> Please suggest.
>>> >>> >>>>
>>> >>> >>>> Thanks in advance.
>>> >>> >>>>
>>> >>> >>>> Regrads,
>>> >>> >>>> Ranjini. R
>>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>> >>>> <ra...@gmail.com> wrote:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> ---------- Forwarded message ----------
>>> >>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>> >>>>> To: user@hadoop.apache.org
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Hi Ranjini,
>>> >>> >>>>>
>>> >>> >>>>> A good example to look into :
>>> >>> >>>>> http://www.undercloud.org/?p=408
>>> >>> >>>>>
>>> >>> >>>>> Cheers,
>>> >>> >>>>> Subroto Sanyal
>>> >>> >>>>>
>>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>> >>>>>
>>> >>> >>>>> Hi,
>>> >>> >>>>>
>>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>>> hive
>>> >>> >>>>> using java.
>>> >>> >>>>>
>>> >>> >>>>> Please provide sample code.
>>> >>> >>>>>
>>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>>> version
>>> >>> >>>>> should be used.
>>> >>> >>>>>
>>> >>> >>>>> Thanks in advance.
>>> >>> >>>>>
>>> >>> >>>>> Ranjini
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>
>>> >>> >>>
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I want to know , when should i use Mapper , Reduce and Combiner.
What all methods are there in them.
Please suggest for study in detail. As I am fresher .
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:34 PM, unmesha sreeveni <un...@gmail.com>wrote:
> Ranjini can u pls check this. This is not perfect ..I simply did to
> check my xml data.
>
> https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
>
>
> On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
>> Hi,
>>
>> The driver class and my Mapper class i have used
>> org.apache.hadoop.mapreduce.lib
>>
>> and in the XmlInputFormat.java class also i have used the
>> org.apache.hadoop.mapreduce.lib
>>
>> but still iam getting this error.
>>
>> Please suggest.
>>
>> Thanks in advance
>>
>> Ranjini
>>
>> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>>
>>> Hello Ranjini,
>>> This error will come when you use mix and match newer and older API..
>>>
>>> You might have written program using newer API and the the XML input
>>> format is using older api..
>>> The older api has package structure of org.apache.hadoop.mapred
>>>
>>> The newer api has package structure package of
>>> org.apache.hadoop.mapreduce.lib
>>>
>>> Check out the XMLINputFormat.java, which package of FileInputFormat
>>> they have used...
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>>> <ra...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am using hadoop 0.20 version
>>> >
>>> > In that while exceuting the XmlInformat class
>>> > I am getting the error as
>>> >
>>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext,
>>> but
>>> > interface was excepted,."
>>> >
>>> > Please suggest to fix the error.
>>> >
>>> > Thanks in advance.
>>> >
>>> > Ranjini
>>> >
>>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> > wrote:
>>> >>
>>> >> hi,
>>> >>
>>> >> I have fixed the error , the code is running fine, but this code just
>>> >> split the part of the tag.
>>> >>
>>> >> i want to convert into text format so that i can load them into
>>> tables of
>>> >> hbase and hive.
>>> >>
>>> >> I have used the DOM Parser but this parser uses File as Object but
>>> hdfs
>>> >> uses FileSystem.
>>> >>
>>> >> Eg,
>>> >>
>>> >> File fXmlFile = new File("D:/elango/test.xml");
>>> >>
>>> >> System.out.println(g);
>>> >> DocumentBuilderFactory dbFactory =
>>> DocumentBuilderFactory.newInstance();
>>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>>> >> Document doc = dBuilder.parse(fXmlFile);
>>> >>
>>> >>
>>> >> This cant be used as hdfs, because hdfs path is accessed through
>>> >> FileSystem.
>>> >>
>>> >> I kindly request u to , Please suggest me to fix the above issue.
>>> >>
>>> >> Thanks in advance
>>> >>
>>> >> Ranjini R
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>>> >> <ra...@gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> ---------- Forwarded message ----------
>>> >>> From: Shekhar Sharma <sh...@gmail.com>
>>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> >>> Subject: Re: Hadoop-MapReduce
>>> >>> To: user@hadoop.apache.org
>>> >>> Cc: ssanyal@datameer.com
>>> >>>
>>> >>>
>>> >>> It does work i have used it long back..
>>> >>>
>>> >>> BTW if it is not working, write the custom input format and implement
>>> >>> your record reader. That would be far more easy than breaking your
>>> >>> head with others code.
>>> >>>
>>> >>> Break your problem in step:
>>> >>>
>>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> >>> single record for you...May be a record for you would be
>>> >>>
>>> >>> <person>
>>> >>> <fname>x</fname>
>>> >>> <lname>y</lname>
>>> >>> </person>
>>> >>>
>>> >>> (2) Implement a record reader that looks out for the starting and
>>> >>> ending person tag ( Checkout how RecordReader.java is written)
>>> >>>
>>> >>> (3) Once you got the contents between starting and ending tag, now
>>> you
>>> >>> can use a xml parser to parse the contents into an java object and
>>> >>> form your own key value pairs ( custom key and custom value)
>>> >>>
>>> >>>
>>> >>> Hope you have enough pointers to write the code.
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>> Som Shekhar Sharma
>>> >>> +91-8197243810
>>> >>>
>>> >>>
>>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> >>> wrote:
>>> >>> > Hi Subroto Sanyal,
>>> >>> >
>>> >>> > The link provided about xml, it does not work . The Class written
>>> >>> > XmlContent is not allowed in the XmlInputFormat.
>>> >>> >
>>> >>> > I request you to help , whether this scenaio some one has coded,
>>> and
>>> >>> > needed
>>> >>> > working code.
>>> >>> >
>>> >>> > I have written using SAX Parser too, but eventhough the jars are
>>> added
>>> >>> > in
>>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >>> >
>>> >>> > Please provide sample code for the same.
>>> >>> >
>>> >>> > Thanks in advance,
>>> >>> > Ranjini.R
>>> >>> >
>>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> >>> > <ra...@gmail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >>
>>> >>> >>>> Hi,
>>> >>> >>>>
>>> >>> >>>> As suggest by the link below , i have used for my program ,
>>> >>> >>>>
>>> >>> >>>> but i am facing the below issues, please help me to fix these
>>> error.
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>> >>>> override
>>> >>> >>>> abstract method
>>> >>> >>>>
>>> >>> >>>>
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>> >>>> <LongWritable, Text, Text, Text> {
>>> >>> >>>> ^
>>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>> >>>> abstract
>>> >>> >>>> and does not override abstract method
>>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>> >>>> ^
>>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>> >>>> 2 errors
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>> >>>>
>>> >>> >>>> Please suggest.
>>> >>> >>>>
>>> >>> >>>> Thanks in advance.
>>> >>> >>>>
>>> >>> >>>> Regrads,
>>> >>> >>>> Ranjini. R
>>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>> >>>> <ra...@gmail.com> wrote:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> ---------- Forwarded message ----------
>>> >>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>> >>>>> To: user@hadoop.apache.org
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Hi Ranjini,
>>> >>> >>>>>
>>> >>> >>>>> A good example to look into :
>>> >>> >>>>> http://www.undercloud.org/?p=408
>>> >>> >>>>>
>>> >>> >>>>> Cheers,
>>> >>> >>>>> Subroto Sanyal
>>> >>> >>>>>
>>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>> >>>>>
>>> >>> >>>>> Hi,
>>> >>> >>>>>
>>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>>> hive
>>> >>> >>>>> using java.
>>> >>> >>>>>
>>> >>> >>>>> Please provide sample code.
>>> >>> >>>>>
>>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>>> version
>>> >>> >>>>> should be used.
>>> >>> >>>>>
>>> >>> >>>>> Thanks in advance.
>>> >>> >>>>>
>>> >>> >>>>> Ranjini
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>
>>> >>> >>>
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I want to know , when should i use Mapper , Reduce and Combiner.
What all methods are there in them.
Please suggest for study in detail. As I am fresher .
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:34 PM, unmesha sreeveni <un...@gmail.com>wrote:
> Ranjini can u pls check this. This is not perfect ..I simply did to
> check my xml data.
>
> https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
>
>
> On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
>> Hi,
>>
>> The driver class and my Mapper class i have used
>> org.apache.hadoop.mapreduce.lib
>>
>> and in the XmlInputFormat.java class also i have used the
>> org.apache.hadoop.mapreduce.lib
>>
>> but still iam getting this error.
>>
>> Please suggest.
>>
>> Thanks in advance
>>
>> Ranjini
>>
>> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>>
>>> Hello Ranjini,
>>> This error will come when you use mix and match newer and older API..
>>>
>>> You might have written program using newer API and the the XML input
>>> format is using older api..
>>> The older api has package structure of org.apache.hadoop.mapred
>>>
>>> The newer api has package structure package of
>>> org.apache.hadoop.mapreduce.lib
>>>
>>> Check out the XMLINputFormat.java, which package of FileInputFormat
>>> they have used...
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>>> <ra...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am using hadoop 0.20 version
>>> >
>>> > In that while exceuting the XmlInformat class
>>> > I am getting the error as
>>> >
>>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext,
>>> but
>>> > interface was excepted,."
>>> >
>>> > Please suggest to fix the error.
>>> >
>>> > Thanks in advance.
>>> >
>>> > Ranjini
>>> >
>>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> > wrote:
>>> >>
>>> >> hi,
>>> >>
>>> >> I have fixed the error , the code is running fine, but this code just
>>> >> split the part of the tag.
>>> >>
>>> >> i want to convert into text format so that i can load them into
>>> tables of
>>> >> hbase and hive.
>>> >>
>>> >> I have used the DOM Parser but this parser uses File as Object but
>>> hdfs
>>> >> uses FileSystem.
>>> >>
>>> >> Eg,
>>> >>
>>> >> File fXmlFile = new File("D:/elango/test.xml");
>>> >>
>>> >> System.out.println(g);
>>> >> DocumentBuilderFactory dbFactory =
>>> DocumentBuilderFactory.newInstance();
>>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>>> >> Document doc = dBuilder.parse(fXmlFile);
>>> >>
>>> >>
>>> >> This cant be used as hdfs, because hdfs path is accessed through
>>> >> FileSystem.
>>> >>
>>> >> I kindly request u to , Please suggest me to fix the above issue.
>>> >>
>>> >> Thanks in advance
>>> >>
>>> >> Ranjini R
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>>> >> <ra...@gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> ---------- Forwarded message ----------
>>> >>> From: Shekhar Sharma <sh...@gmail.com>
>>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> >>> Subject: Re: Hadoop-MapReduce
>>> >>> To: user@hadoop.apache.org
>>> >>> Cc: ssanyal@datameer.com
>>> >>>
>>> >>>
>>> >>> It does work i have used it long back..
>>> >>>
>>> >>> BTW if it is not working, write the custom input format and implement
>>> >>> your record reader. That would be far more easy than breaking your
>>> >>> head with others code.
>>> >>>
>>> >>> Break your problem in step:
>>> >>>
>>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> >>> single record for you...May be a record for you would be
>>> >>>
>>> >>> <person>
>>> >>> <fname>x</fname>
>>> >>> <lname>y</lname>
>>> >>> </person>
>>> >>>
>>> >>> (2) Implement a record reader that looks out for the starting and
>>> >>> ending person tag ( Checkout how RecordReader.java is written)
>>> >>>
>>> >>> (3) Once you got the contents between starting and ending tag, now
>>> you
>>> >>> can use a xml parser to parse the contents into an java object and
>>> >>> form your own key value pairs ( custom key and custom value)
>>> >>>
>>> >>>
>>> >>> Hope you have enough pointers to write the code.
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>> Som Shekhar Sharma
>>> >>> +91-8197243810
>>> >>>
>>> >>>
>>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com>
>>> >>> wrote:
>>> >>> > Hi Subroto Sanyal,
>>> >>> >
>>> >>> > The link provided about xml, it does not work . The Class written
>>> >>> > XmlContent is not allowed in the XmlInputFormat.
>>> >>> >
>>> >>> > I request you to help , whether this scenaio some one has coded,
>>> and
>>> >>> > needed
>>> >>> > working code.
>>> >>> >
>>> >>> > I have written using SAX Parser too, but eventhough the jars are
>>> added
>>> >>> > in
>>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >>> >
>>> >>> > Please provide sample code for the same.
>>> >>> >
>>> >>> > Thanks in advance,
>>> >>> > Ranjini.R
>>> >>> >
>>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> >>> > <ra...@gmail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >>
>>> >>> >>>> Hi,
>>> >>> >>>>
>>> >>> >>>> As suggest by the link below , i have used for my program ,
>>> >>> >>>>
>>> >>> >>>> but i am facing the below issues, please help me to fix these
>>> error.
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>> >>>> override
>>> >>> >>>> abstract method
>>> >>> >>>>
>>> >>> >>>>
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>> >>>> <LongWritable, Text, Text, Text> {
>>> >>> >>>> ^
>>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>> >>>> abstract
>>> >>> >>>> and does not override abstract method
>>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>> >>>> ^
>>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>> >>>> 2 errors
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>> >>>>
>>> >>> >>>> Please suggest.
>>> >>> >>>>
>>> >>> >>>> Thanks in advance.
>>> >>> >>>>
>>> >>> >>>> Regrads,
>>> >>> >>>> Ranjini. R
>>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>> >>>> <ra...@gmail.com> wrote:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> ---------- Forwarded message ----------
>>> >>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>> >>>>> To: user@hadoop.apache.org
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Hi Ranjini,
>>> >>> >>>>>
>>> >>> >>>>> A good example to look into :
>>> >>> >>>>> http://www.undercloud.org/?p=408
>>> >>> >>>>>
>>> >>> >>>>> Cheers,
>>> >>> >>>>> Subroto Sanyal
>>> >>> >>>>>
>>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>> >>>>>
>>> >>> >>>>> Hi,
>>> >>> >>>>>
>>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>>> hive
>>> >>> >>>>> using java.
>>> >>> >>>>>
>>> >>> >>>>> Please provide sample code.
>>> >>> >>>>>
>>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>>> version
>>> >>> >>>>> should be used.
>>> >>> >>>>>
>>> >>> >>>>> Thanks in advance.
>>> >>> >>>>>
>>> >>> >>>>> Ranjini
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>
>>> >>> >>>
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
>
>
Re: Hadoop-MapReduce
Posted by unmesha sreeveni <un...@gmail.com>.
Ranjini can u pls check this. This is not perfect ..I simply did to check
my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> The driver class and my Mapper class i have used
> org.apache.hadoop.mapreduce.lib
>
> and in the XmlInputFormat.java class also i have used the
> org.apache.hadoop.mapreduce.lib
>
> but still iam getting this error.
>
> Please suggest.
>
> Thanks in advance
>
> Ranjini
>
> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>
>> Hello Ranjini,
>> This error will come when you use mix and match newer and older API..
>>
>> You might have written program using newer API and the the XML input
>> format is using older api..
>> The older api has package structure of org.apache.hadoop.mapred
>>
>> The newer api has package structure package of
>> org.apache.hadoop.mapreduce.lib
>>
>> Check out the XMLINputFormat.java, which package of FileInputFormat
>> they have used...
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am using hadoop 0.20 version
>> >
>> > In that while exceuting the XmlInformat class
>> > I am getting the error as
>> >
>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
>> > interface was excepted,."
>> >
>> > Please suggest to fix the error.
>> >
>> > Thanks in advance.
>> >
>> > Ranjini
>> >
>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >> hi,
>> >>
>> >> I have fixed the error , the code is running fine, but this code just
>> >> split the part of the tag.
>> >>
>> >> i want to convert into text format so that i can load them into tables
>> of
>> >> hbase and hive.
>> >>
>> >> I have used the DOM Parser but this parser uses File as Object but
>> hdfs
>> >> uses FileSystem.
>> >>
>> >> Eg,
>> >>
>> >> File fXmlFile = new File("D:/elango/test.xml");
>> >>
>> >> System.out.println(g);
>> >> DocumentBuilderFactory dbFactory =
>> DocumentBuilderFactory.newInstance();
>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> >> Document doc = dBuilder.parse(fXmlFile);
>> >>
>> >>
>> >> This cant be used as hdfs, because hdfs path is accessed through
>> >> FileSystem.
>> >>
>> >> I kindly request u to , Please suggest me to fix the above issue.
>> >>
>> >> Thanks in advance
>> >>
>> >> Ranjini R
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> >> <ra...@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> ---------- Forwarded message ----------
>> >>> From: Shekhar Sharma <sh...@gmail.com>
>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> >>> Subject: Re: Hadoop-MapReduce
>> >>> To: user@hadoop.apache.org
>> >>> Cc: ssanyal@datameer.com
>> >>>
>> >>>
>> >>> It does work i have used it long back..
>> >>>
>> >>> BTW if it is not working, write the custom input format and implement
>> >>> your record reader. That would be far more easy than breaking your
>> >>> head with others code.
>> >>>
>> >>> Break your problem in step:
>> >>>
>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> >>> single record for you...May be a record for you would be
>> >>>
>> >>> <person>
>> >>> <fname>x</fname>
>> >>> <lname>y</lname>
>> >>> </person>
>> >>>
>> >>> (2) Implement a record reader that looks out for the starting and
>> >>> ending person tag ( Checkout how RecordReader.java is written)
>> >>>
>> >>> (3) Once you got the contents between starting and ending tag, now you
>> >>> can use a xml parser to parse the contents into an java object and
>> >>> form your own key value pairs ( custom key and custom value)
>> >>>
>> >>>
>> >>> Hope you have enough pointers to write the code.
>> >>>
>> >>>
>> >>> Regards,
>> >>> Som Shekhar Sharma
>> >>> +91-8197243810
>> >>>
>> >>>
>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> >>> wrote:
>> >>> > Hi Subroto Sanyal,
>> >>> >
>> >>> > The link provided about xml, it does not work . The Class written
>> >>> > XmlContent is not allowed in the XmlInputFormat.
>> >>> >
>> >>> > I request you to help , whether this scenaio some one has coded, and
>> >>> > needed
>> >>> > working code.
>> >>> >
>> >>> > I have written using SAX Parser too, but eventhough the jars are
>> added
>> >>> > in
>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>> >>> >
>> >>> > Please provide sample code for the same.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Ranjini.R
>> >>> >
>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>> >>> > <ra...@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> As suggest by the link below , i have used for my program ,
>> >>> >>>>
>> >>> >>>> but i am facing the below issues, please help me to fix these
>> error.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>> >>> >>>> override
>> >>> >>>> abstract method
>> >>> >>>>
>> >>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>> >>>> <LongWritable, Text, Text, Text> {
>> >>> >>>> ^
>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> >>> >>>> abstract
>> >>> >>>> and does not override abstract method
>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>> >>>> public class XmlRecordReader implements RecordReader {
>> >>> >>>> ^
>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>> >>>> 2 errors
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>> >>>>
>> >>> >>>> Please suggest.
>> >>> >>>>
>> >>> >>>> Thanks in advance.
>> >>> >>>>
>> >>> >>>> Regrads,
>> >>> >>>> Ranjini. R
>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>> >>>> <ra...@gmail.com> wrote:
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> ---------- Forwarded message ----------
>> >>> >>>>> From: Subroto <ss...@datameer.com>
>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>> >>>>> To: user@hadoop.apache.org
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> Hi Ranjini,
>> >>> >>>>>
>> >>> >>>>> A good example to look into :
>> >>> >>>>> http://www.undercloud.org/?p=408
>> >>> >>>>>
>> >>> >>>>> Cheers,
>> >>> >>>>> Subroto Sanyal
>> >>> >>>>>
>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>> >>>>>
>> >>> >>>>> Hi,
>> >>> >>>>>
>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>> hive
>> >>> >>>>> using java.
>> >>> >>>>>
>> >>> >>>>> Please provide sample code.
>> >>> >>>>>
>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>> version
>> >>> >>>>> should be used.
>> >>> >>>>>
>> >>> >>>>> Thanks in advance.
>> >>> >>>>>
>> >>> >>>>> Ranjini
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: Hadoop-MapReduce
Posted by unmesha sreeveni <un...@gmail.com>.
Ranjini can u pls check this. This is not perfect ..I simply did to check
my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> The driver class and my Mapper class i have used
> org.apache.hadoop.mapreduce.lib
>
> and in the XmlInputFormat.java class also i have used the
> org.apache.hadoop.mapreduce.lib
>
> but still iam getting this error.
>
> Please suggest.
>
> Thanks in advance
>
> Ranjini
>
> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>
>> Hello Ranjini,
>> This error will come when you use mix and match newer and older API..
>>
>> You might have written program using newer API and the the XML input
>> format is using older api..
>> The older api has package structure of org.apache.hadoop.mapred
>>
>> The newer api has package structure package of
>> org.apache.hadoop.mapreduce.lib
>>
>> Check out the XMLINputFormat.java, which package of FileInputFormat
>> they have used...
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am using hadoop 0.20 version
>> >
>> > In that while exceuting the XmlInformat class
>> > I am getting the error as
>> >
>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
>> > interface was excepted,."
>> >
>> > Please suggest to fix the error.
>> >
>> > Thanks in advance.
>> >
>> > Ranjini
>> >
>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >> hi,
>> >>
>> >> I have fixed the error , the code is running fine, but this code just
>> >> split the part of the tag.
>> >>
>> >> i want to convert into text format so that i can load them into tables
>> of
>> >> hbase and hive.
>> >>
>> >> I have used the DOM Parser but this parser uses File as Object but
>> hdfs
>> >> uses FileSystem.
>> >>
>> >> Eg,
>> >>
>> >> File fXmlFile = new File("D:/elango/test.xml");
>> >>
>> >> System.out.println(g);
>> >> DocumentBuilderFactory dbFactory =
>> DocumentBuilderFactory.newInstance();
>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> >> Document doc = dBuilder.parse(fXmlFile);
>> >>
>> >>
>> >> This cant be used as hdfs, because hdfs path is accessed through
>> >> FileSystem.
>> >>
>> >> I kindly request u to , Please suggest me to fix the above issue.
>> >>
>> >> Thanks in advance
>> >>
>> >> Ranjini R
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> >> <ra...@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> ---------- Forwarded message ----------
>> >>> From: Shekhar Sharma <sh...@gmail.com>
>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> >>> Subject: Re: Hadoop-MapReduce
>> >>> To: user@hadoop.apache.org
>> >>> Cc: ssanyal@datameer.com
>> >>>
>> >>>
>> >>> It does work i have used it long back..
>> >>>
>> >>> BTW if it is not working, write the custom input format and implement
>> >>> your record reader. That would be far more easy than breaking your
>> >>> head with others code.
>> >>>
>> >>> Break your problem in step:
>> >>>
>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> >>> single record for you...May be a record for you would be
>> >>>
>> >>> <person>
>> >>> <fname>x</fname>
>> >>> <lname>y</lname>
>> >>> </person>
>> >>>
>> >>> (2) Implement a record reader that looks out for the starting and
>> >>> ending person tag ( Checkout how RecordReader.java is written)
>> >>>
>> >>> (3) Once you got the contents between starting and ending tag, now you
>> >>> can use a xml parser to parse the contents into an java object and
>> >>> form your own key value pairs ( custom key and custom value)
>> >>>
>> >>>
>> >>> Hope you have enough pointers to write the code.
>> >>>
>> >>>
>> >>> Regards,
>> >>> Som Shekhar Sharma
>> >>> +91-8197243810
>> >>>
>> >>>
>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> >>> wrote:
>> >>> > Hi Subroto Sanyal,
>> >>> >
>> >>> > The link provided about xml, it does not work . The Class written
>> >>> > XmlContent is not allowed in the XmlInputFormat.
>> >>> >
>> >>> > I request you to help , whether this scenaio some one has coded, and
>> >>> > needed
>> >>> > working code.
>> >>> >
>> >>> > I have written using SAX Parser too, but eventhough the jars are
>> added
>> >>> > in
>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>> >>> >
>> >>> > Please provide sample code for the same.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Ranjini.R
>> >>> >
>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>> >>> > <ra...@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> As suggest by the link below , i have used for my program ,
>> >>> >>>>
>> >>> >>>> but i am facing the below issues, please help me to fix these
>> error.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>> >>> >>>> override
>> >>> >>>> abstract method
>> >>> >>>>
>> >>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>> >>>> <LongWritable, Text, Text, Text> {
>> >>> >>>> ^
>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> >>> >>>> abstract
>> >>> >>>> and does not override abstract method
>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>> >>>> public class XmlRecordReader implements RecordReader {
>> >>> >>>> ^
>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>> >>>> 2 errors
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>> >>>>
>> >>> >>>> Please suggest.
>> >>> >>>>
>> >>> >>>> Thanks in advance.
>> >>> >>>>
>> >>> >>>> Regrads,
>> >>> >>>> Ranjini. R
>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>> >>>> <ra...@gmail.com> wrote:
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> ---------- Forwarded message ----------
>> >>> >>>>> From: Subroto <ss...@datameer.com>
>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>> >>>>> To: user@hadoop.apache.org
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> Hi Ranjini,
>> >>> >>>>>
>> >>> >>>>> A good example to look into :
>> >>> >>>>> http://www.undercloud.org/?p=408
>> >>> >>>>>
>> >>> >>>>> Cheers,
>> >>> >>>>> Subroto Sanyal
>> >>> >>>>>
>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>> >>>>>
>> >>> >>>>> Hi,
>> >>> >>>>>
>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>> hive
>> >>> >>>>> using java.
>> >>> >>>>>
>> >>> >>>>> Please provide sample code.
>> >>> >>>>>
>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>> version
>> >>> >>>>> should be used.
>> >>> >>>>>
>> >>> >>>>> Thanks in advance.
>> >>> >>>>>
>> >>> >>>>> Ranjini
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: Hadoop-MapReduce
Posted by unmesha sreeveni <un...@gmail.com>.
Ranjini can u pls check this. This is not perfect ..I simply did to check
my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> The driver class and my Mapper class i have used
> org.apache.hadoop.mapreduce.lib
>
> and in the XmlInputFormat.java class also i have used the
> org.apache.hadoop.mapreduce.lib
>
> but still iam getting this error.
>
> Please suggest.
>
> Thanks in advance
>
> Ranjini
>
> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>
>> Hello Ranjini,
>> This error will come when you use mix and match newer and older API..
>>
>> You might have written program using newer API and the the XML input
>> format is using older api..
>> The older api has package structure of org.apache.hadoop.mapred
>>
>> The newer api has package structure package of
>> org.apache.hadoop.mapreduce.lib
>>
>> Check out the XMLINputFormat.java, which package of FileInputFormat
>> they have used...
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am using hadoop 0.20 version
>> >
>> > In that while exceuting the XmlInformat class
>> > I am getting the error as
>> >
>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
>> > interface was excepted,."
>> >
>> > Please suggest to fix the error.
>> >
>> > Thanks in advance.
>> >
>> > Ranjini
>> >
>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >> hi,
>> >>
>> >> I have fixed the error , the code is running fine, but this code just
>> >> split the part of the tag.
>> >>
>> >> i want to convert into text format so that i can load them into tables
>> of
>> >> hbase and hive.
>> >>
>> >> I have used the DOM Parser but this parser uses File as Object but
>> hdfs
>> >> uses FileSystem.
>> >>
>> >> Eg,
>> >>
>> >> File fXmlFile = new File("D:/elango/test.xml");
>> >>
>> >> System.out.println(g);
>> >> DocumentBuilderFactory dbFactory =
>> DocumentBuilderFactory.newInstance();
>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> >> Document doc = dBuilder.parse(fXmlFile);
>> >>
>> >>
>> >> This cant be used as hdfs, because hdfs path is accessed through
>> >> FileSystem.
>> >>
>> >> I kindly request u to , Please suggest me to fix the above issue.
>> >>
>> >> Thanks in advance
>> >>
>> >> Ranjini R
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> >> <ra...@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> ---------- Forwarded message ----------
>> >>> From: Shekhar Sharma <sh...@gmail.com>
>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> >>> Subject: Re: Hadoop-MapReduce
>> >>> To: user@hadoop.apache.org
>> >>> Cc: ssanyal@datameer.com
>> >>>
>> >>>
>> >>> It does work i have used it long back..
>> >>>
>> >>> BTW if it is not working, write the custom input format and implement
>> >>> your record reader. That would be far more easy than breaking your
>> >>> head with others code.
>> >>>
>> >>> Break your problem in step:
>> >>>
>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> >>> single record for you...May be a record for you would be
>> >>>
>> >>> <person>
>> >>> <fname>x</fname>
>> >>> <lname>y</lname>
>> >>> </person>
>> >>>
>> >>> (2) Implement a record reader that looks out for the starting and
>> >>> ending person tag ( Checkout how RecordReader.java is written)
>> >>>
>> >>> (3) Once you got the contents between starting and ending tag, now you
>> >>> can use a xml parser to parse the contents into an java object and
>> >>> form your own key value pairs ( custom key and custom value)
>> >>>
>> >>>
>> >>> Hope you have enough pointers to write the code.
>> >>>
>> >>>
>> >>> Regards,
>> >>> Som Shekhar Sharma
>> >>> +91-8197243810
>> >>>
>> >>>
>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> >>> wrote:
>> >>> > Hi Subroto Sanyal,
>> >>> >
>> >>> > The link provided about xml, it does not work . The Class written
>> >>> > XmlContent is not allowed in the XmlInputFormat.
>> >>> >
>> >>> > I request you to help , whether this scenaio some one has coded, and
>> >>> > needed
>> >>> > working code.
>> >>> >
>> >>> > I have written using SAX Parser too, but eventhough the jars are
>> added
>> >>> > in
>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>> >>> >
>> >>> > Please provide sample code for the same.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Ranjini.R
>> >>> >
>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>> >>> > <ra...@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> As suggest by the link below , i have used for my program ,
>> >>> >>>>
>> >>> >>>> but i am facing the below issues, please help me to fix these
>> error.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>> >>> >>>> override
>> >>> >>>> abstract method
>> >>> >>>>
>> >>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>> >>>> <LongWritable, Text, Text, Text> {
>> >>> >>>> ^
>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> >>> >>>> abstract
>> >>> >>>> and does not override abstract method
>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>> >>>> public class XmlRecordReader implements RecordReader {
>> >>> >>>> ^
>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>> >>>> 2 errors
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>> >>>>
>> >>> >>>> Please suggest.
>> >>> >>>>
>> >>> >>>> Thanks in advance.
>> >>> >>>>
>> >>> >>>> Regrads,
>> >>> >>>> Ranjini. R
>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>> >>>> <ra...@gmail.com> wrote:
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> ---------- Forwarded message ----------
>> >>> >>>>> From: Subroto <ss...@datameer.com>
>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>> >>>>> To: user@hadoop.apache.org
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> Hi Ranjini,
>> >>> >>>>>
>> >>> >>>>> A good example to look into :
>> >>> >>>>> http://www.undercloud.org/?p=408
>> >>> >>>>>
>> >>> >>>>> Cheers,
>> >>> >>>>> Subroto Sanyal
>> >>> >>>>>
>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>> >>>>>
>> >>> >>>>> Hi,
>> >>> >>>>>
>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>> hive
>> >>> >>>>> using java.
>> >>> >>>>>
>> >>> >>>>> Please provide sample code.
>> >>> >>>>>
>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>> version
>> >>> >>>>> should be used.
>> >>> >>>>>
>> >>> >>>>> Thanks in advance.
>> >>> >>>>>
>> >>> >>>>> Ranjini
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: Hadoop-MapReduce
Posted by unmesha sreeveni <un...@gmail.com>.
Ranjini can u pls check this. This is not perfect ..I simply did to check
my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java
On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
> Hi,
>
> The driver class and my Mapper class i have used
> org.apache.hadoop.mapreduce.lib
>
> and in the XmlInputFormat.java class also i have used the
> org.apache.hadoop.mapreduce.lib
>
> but still iam getting this error.
>
> Please suggest.
>
> Thanks in advance
>
> Ranjini
>
> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
>
>> Hello Ranjini,
>> This error will come when you use mix and match newer and older API..
>>
>> You might have written program using newer API and the the XML input
>> format is using older api..
>> The older api has package structure of org.apache.hadoop.mapred
>>
>> The newer api has package structure package of
>> org.apache.hadoop.mapreduce.lib
>>
>> Check out the XMLINputFormat.java, which package of FileInputFormat
>> they have used...
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am using hadoop 0.20 version
>> >
>> > In that while exceuting the XmlInformat class
>> > I am getting the error as
>> >
>> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
>> > interface was excepted,."
>> >
>> > Please suggest to fix the error.
>> >
>> > Thanks in advance.
>> >
>> > Ranjini
>> >
>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >> hi,
>> >>
>> >> I have fixed the error , the code is running fine, but this code just
>> >> split the part of the tag.
>> >>
>> >> i want to convert into text format so that i can load them into tables
>> of
>> >> hbase and hive.
>> >>
>> >> I have used the DOM Parser but this parser uses File as Object but
>> hdfs
>> >> uses FileSystem.
>> >>
>> >> Eg,
>> >>
>> >> File fXmlFile = new File("D:/elango/test.xml");
>> >>
>> >> System.out.println(g);
>> >> DocumentBuilderFactory dbFactory =
>> DocumentBuilderFactory.newInstance();
>> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> >> Document doc = dBuilder.parse(fXmlFile);
>> >>
>> >>
>> >> This cant be used as hdfs, because hdfs path is accessed through
>> >> FileSystem.
>> >>
>> >> I kindly request u to , Please suggest me to fix the above issue.
>> >>
>> >> Thanks in advance
>> >>
>> >> Ranjini R
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> >> <ra...@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> ---------- Forwarded message ----------
>> >>> From: Shekhar Sharma <sh...@gmail.com>
>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> >>> Subject: Re: Hadoop-MapReduce
>> >>> To: user@hadoop.apache.org
>> >>> Cc: ssanyal@datameer.com
>> >>>
>> >>>
>> >>> It does work i have used it long back..
>> >>>
>> >>> BTW if it is not working, write the custom input format and implement
>> >>> your record reader. That would be far more easy than breaking your
>> >>> head with others code.
>> >>>
>> >>> Break your problem in step:
>> >>>
>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> >>> single record for you...May be a record for you would be
>> >>>
>> >>> <person>
>> >>> <fname>x</fname>
>> >>> <lname>y</lname>
>> >>> </person>
>> >>>
>> >>> (2) Implement a record reader that looks out for the starting and
>> >>> ending person tag ( Checkout how RecordReader.java is written)
>> >>>
>> >>> (3) Once you got the contents between starting and ending tag, now you
>> >>> can use a xml parser to parse the contents into an java object and
>> >>> form your own key value pairs ( custom key and custom value)
>> >>>
>> >>>
>> >>> Hope you have enough pointers to write the code.
>> >>>
>> >>>
>> >>> Regards,
>> >>> Som Shekhar Sharma
>> >>> +91-8197243810
>> >>>
>> >>>
>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> >>> wrote:
>> >>> > Hi Subroto Sanyal,
>> >>> >
>> >>> > The link provided about xml, it does not work . The Class written
>> >>> > XmlContent is not allowed in the XmlInputFormat.
>> >>> >
>> >>> > I request you to help , whether this scenaio some one has coded, and
>> >>> > needed
>> >>> > working code.
>> >>> >
>> >>> > I have written using SAX Parser too, but eventhough the jars are
>> added
>> >>> > in
>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>> >>> >
>> >>> > Please provide sample code for the same.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Ranjini.R
>> >>> >
>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>> >>> > <ra...@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> As suggest by the link below , i have used for my program ,
>> >>> >>>>
>> >>> >>>> but i am facing the below issues, please help me to fix these
>> error.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>> >>> >>>> override
>> >>> >>>> abstract method
>> >>> >>>>
>> >>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>> >>>> <LongWritable, Text, Text, Text> {
>> >>> >>>> ^
>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> >>> >>>> abstract
>> >>> >>>> and does not override abstract method
>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>> >>>> public class XmlRecordReader implements RecordReader {
>> >>> >>>> ^
>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>> >>>> 2 errors
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>> >>>>
>> >>> >>>> Please suggest.
>> >>> >>>>
>> >>> >>>> Thanks in advance.
>> >>> >>>>
>> >>> >>>> Regrads,
>> >>> >>>> Ranjini. R
>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>> >>>> <ra...@gmail.com> wrote:
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> ---------- Forwarded message ----------
>> >>> >>>>> From: Subroto <ss...@datameer.com>
>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>> >>>>> To: user@hadoop.apache.org
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> Hi Ranjini,
>> >>> >>>>>
>> >>> >>>>> A good example to look into :
>> >>> >>>>> http://www.undercloud.org/?p=408
>> >>> >>>>>
>> >>> >>>>> Cheers,
>> >>> >>>>> Subroto Sanyal
>> >>> >>>>>
>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>> >>>>>
>> >>> >>>>> Hi,
>> >>> >>>>>
>> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
>> hive
>> >>> >>>>> using java.
>> >>> >>>>>
>> >>> >>>>> Please provide sample code.
>> >>> >>>>>
>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser
>> version
>> >>> >>>>> should be used.
>> >>> >>>>>
>> >>> >>>>> Thanks in advance.
>> >>> >>>>>
>> >>> >>>>> Ranjini
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>
>
--
*Thanks & Regards*
Unmesha Sreeveni U.B
*Junior Developer*
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
The driver class and my Mapper class i have used
org.apache.hadoop.mapreduce.lib
and in the XmlInputFormat.java class also i have used the
org.apache.hadoop.mapreduce.lib
but still iam getting this error.
Please suggest.
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
> Hello Ranjini,
> This error will come when you use mix and match newer and older API..
>
> You might have written program using newer API and the the XML input
> format is using older api..
> The older api has package structure of org.apache.hadoop.mapred
>
> The newer api has package structure package of
> org.apache.hadoop.mapreduce.lib
>
> Check out the XMLINputFormat.java, which package of FileInputFormat
> they have used...
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
> <ra...@gmail.com> wrote:
> > Hi,
> >
> > I am using hadoop 0.20 version
> >
> > In that while exceuting the XmlInformat class
> > I am getting the error as
> >
> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> > interface was excepted,."
> >
> > Please suggest to fix the error.
> >
> > Thanks in advance.
> >
> > Ranjini
> >
> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >> hi,
> >>
> >> I have fixed the error , the code is running fine, but this code just
> >> split the part of the tag.
> >>
> >> i want to convert into text format so that i can load them into tables
> of
> >> hbase and hive.
> >>
> >> I have used the DOM Parser but this parser uses File as Object but hdfs
> >> uses FileSystem.
> >>
> >> Eg,
> >>
> >> File fXmlFile = new File("D:/elango/test.xml");
> >>
> >> System.out.println(g);
> >> DocumentBuilderFactory dbFactory =
> DocumentBuilderFactory.newInstance();
> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> >> Document doc = dBuilder.parse(fXmlFile);
> >>
> >>
> >> This cant be used as hdfs, because hdfs path is accessed through
> >> FileSystem.
> >>
> >> I kindly request u to , Please suggest me to fix the above issue.
> >>
> >> Thanks in advance
> >>
> >> Ranjini R
> >>
> >>
> >>
> >>
> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
> >> <ra...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> ---------- Forwarded message ----------
> >>> From: Shekhar Sharma <sh...@gmail.com>
> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
> >>> Subject: Re: Hadoop-MapReduce
> >>> To: user@hadoop.apache.org
> >>> Cc: ssanyal@datameer.com
> >>>
> >>>
> >>> It does work i have used it long back..
> >>>
> >>> BTW if it is not working, write the custom input format and implement
> >>> your record reader. That would be far more easy than breaking your
> >>> head with others code.
> >>>
> >>> Break your problem in step:
> >>>
> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
> >>> single record for you...May be a record for you would be
> >>>
> >>> <person>
> >>> <fname>x</fname>
> >>> <lname>y</lname>
> >>> </person>
> >>>
> >>> (2) Implement a record reader that looks out for the starting and
> >>> ending person tag ( Checkout how RecordReader.java is written)
> >>>
> >>> (3) Once you got the contents between starting and ending tag, now you
> >>> can use a xml parser to parse the contents into an java object and
> >>> form your own key value pairs ( custom key and custom value)
> >>>
> >>>
> >>> Hope you have enough pointers to write the code.
> >>>
> >>>
> >>> Regards,
> >>> Som Shekhar Sharma
> >>> +91-8197243810
> >>>
> >>>
> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> >>> wrote:
> >>> > Hi Subroto Sanyal,
> >>> >
> >>> > The link provided about xml, it does not work . The Class written
> >>> > XmlContent is not allowed in the XmlInputFormat.
> >>> >
> >>> > I request you to help , whether this scenaio some one has coded, and
> >>> > needed
> >>> > working code.
> >>> >
> >>> > I have written using SAX Parser too, but eventhough the jars are
> added
> >>> > in
> >>> > classpath THe error is is coming has NoClasFoung Exception.
> >>> >
> >>> > Please provide sample code for the same.
> >>> >
> >>> > Thanks in advance,
> >>> > Ranjini.R
> >>> >
> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
> >>> > <ra...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >>
> >>> >>>> Hi,
> >>> >>>>
> >>> >>>> As suggest by the link below , i have used for my program ,
> >>> >>>>
> >>> >>>> but i am facing the below issues, please help me to fix these
> error.
> >>> >>>>
> >>> >>>>
> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
> >>> >>>> override
> >>> >>>> abstract method
> >>> >>>>
> >>> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>> >>>> in org.apache.hadoop.mapred.Mapper
> >>> >>>> public static class Map extends MapReduceBase implements Mapper
> >>> >>>> <LongWritable, Text, Text, Text> {
> >>> >>>> ^
> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> >>> >>>> abstract
> >>> >>>> and does not override abstract method
> >>> >>>> next(java.lang.Object,java.lang.Object) in
> >>> >>>> org.apache.hadoop.mapred.RecordReader
> >>> >>>> public class XmlRecordReader implements RecordReader {
> >>> >>>> ^
> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>> >>>> 2 errors
> >>> >>>>
> >>> >>>>
> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>> >>>>
> >>> >>>> Please suggest.
> >>> >>>>
> >>> >>>> Thanks in advance.
> >>> >>>>
> >>> >>>> Regrads,
> >>> >>>> Ranjini. R
> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>> >>>> <ra...@gmail.com> wrote:
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> ---------- Forwarded message ----------
> >>> >>>>> From: Subroto <ss...@datameer.com>
> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>> >>>>> Subject: Re: Hadoop-MapReduce
> >>> >>>>> To: user@hadoop.apache.org
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> Hi Ranjini,
> >>> >>>>>
> >>> >>>>> A good example to look into :
> >>> >>>>> http://www.undercloud.org/?p=408
> >>> >>>>>
> >>> >>>>> Cheers,
> >>> >>>>> Subroto Sanyal
> >>> >>>>>
> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>> >>>>>
> >>> >>>>> Hi,
> >>> >>>>>
> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
> hive
> >>> >>>>> using java.
> >>> >>>>>
> >>> >>>>> Please provide sample code.
> >>> >>>>>
> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>> >>>>> should be used.
> >>> >>>>>
> >>> >>>>> Thanks in advance.
> >>> >>>>>
> >>> >>>>> Ranjini
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >
> >>>
> >>
> >
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
The driver class and my Mapper class i have used
org.apache.hadoop.mapreduce.lib
and in the XmlInputFormat.java class also i have used the
org.apache.hadoop.mapreduce.lib
but still iam getting this error.
Please suggest.
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
> Hello Ranjini,
> This error will come when you use mix and match newer and older API..
>
> You might have written program using newer API and the the XML input
> format is using older api..
> The older api has package structure of org.apache.hadoop.mapred
>
> The newer api has package structure package of
> org.apache.hadoop.mapreduce.lib
>
> Check out the XMLINputFormat.java, which package of FileInputFormat
> they have used...
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
> <ra...@gmail.com> wrote:
> > Hi,
> >
> > I am using hadoop 0.20 version
> >
> > In that while exceuting the XmlInformat class
> > I am getting the error as
> >
> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> > interface was excepted,."
> >
> > Please suggest to fix the error.
> >
> > Thanks in advance.
> >
> > Ranjini
> >
> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >> hi,
> >>
> >> I have fixed the error , the code is running fine, but this code just
> >> split the part of the tag.
> >>
> >> i want to convert into text format so that i can load them into tables
> of
> >> hbase and hive.
> >>
> >> I have used the DOM Parser but this parser uses File as Object but hdfs
> >> uses FileSystem.
> >>
> >> Eg,
> >>
> >> File fXmlFile = new File("D:/elango/test.xml");
> >>
> >> System.out.println(g);
> >> DocumentBuilderFactory dbFactory =
> DocumentBuilderFactory.newInstance();
> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> >> Document doc = dBuilder.parse(fXmlFile);
> >>
> >>
> >> This cant be used as hdfs, because hdfs path is accessed through
> >> FileSystem.
> >>
> >> I kindly request u to , Please suggest me to fix the above issue.
> >>
> >> Thanks in advance
> >>
> >> Ranjini R
> >>
> >>
> >>
> >>
> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
> >> <ra...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> ---------- Forwarded message ----------
> >>> From: Shekhar Sharma <sh...@gmail.com>
> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
> >>> Subject: Re: Hadoop-MapReduce
> >>> To: user@hadoop.apache.org
> >>> Cc: ssanyal@datameer.com
> >>>
> >>>
> >>> It does work i have used it long back..
> >>>
> >>> BTW if it is not working, write the custom input format and implement
> >>> your record reader. That would be far more easy than breaking your
> >>> head with others code.
> >>>
> >>> Break your problem in step:
> >>>
> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
> >>> single record for you...May be a record for you would be
> >>>
> >>> <person>
> >>> <fname>x</fname>
> >>> <lname>y</lname>
> >>> </person>
> >>>
> >>> (2) Implement a record reader that looks out for the starting and
> >>> ending person tag ( Checkout how RecordReader.java is written)
> >>>
> >>> (3) Once you got the contents between starting and ending tag, now you
> >>> can use a xml parser to parse the contents into an java object and
> >>> form your own key value pairs ( custom key and custom value)
> >>>
> >>>
> >>> Hope you have enough pointers to write the code.
> >>>
> >>>
> >>> Regards,
> >>> Som Shekhar Sharma
> >>> +91-8197243810
> >>>
> >>>
> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> >>> wrote:
> >>> > Hi Subroto Sanyal,
> >>> >
> >>> > The link provided about xml, it does not work . The Class written
> >>> > XmlContent is not allowed in the XmlInputFormat.
> >>> >
> >>> > I request you to help , whether this scenaio some one has coded, and
> >>> > needed
> >>> > working code.
> >>> >
> >>> > I have written using SAX Parser too, but eventhough the jars are
> added
> >>> > in
> >>> > classpath THe error is is coming has NoClasFoung Exception.
> >>> >
> >>> > Please provide sample code for the same.
> >>> >
> >>> > Thanks in advance,
> >>> > Ranjini.R
> >>> >
> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
> >>> > <ra...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >>
> >>> >>>> Hi,
> >>> >>>>
> >>> >>>> As suggest by the link below , i have used for my program ,
> >>> >>>>
> >>> >>>> but i am facing the below issues, please help me to fix these
> error.
> >>> >>>>
> >>> >>>>
> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
> >>> >>>> override
> >>> >>>> abstract method
> >>> >>>>
> >>> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>> >>>> in org.apache.hadoop.mapred.Mapper
> >>> >>>> public static class Map extends MapReduceBase implements Mapper
> >>> >>>> <LongWritable, Text, Text, Text> {
> >>> >>>> ^
> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> >>> >>>> abstract
> >>> >>>> and does not override abstract method
> >>> >>>> next(java.lang.Object,java.lang.Object) in
> >>> >>>> org.apache.hadoop.mapred.RecordReader
> >>> >>>> public class XmlRecordReader implements RecordReader {
> >>> >>>> ^
> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>> >>>> 2 errors
> >>> >>>>
> >>> >>>>
> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>> >>>>
> >>> >>>> Please suggest.
> >>> >>>>
> >>> >>>> Thanks in advance.
> >>> >>>>
> >>> >>>> Regrads,
> >>> >>>> Ranjini. R
> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>> >>>> <ra...@gmail.com> wrote:
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> ---------- Forwarded message ----------
> >>> >>>>> From: Subroto <ss...@datameer.com>
> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>> >>>>> Subject: Re: Hadoop-MapReduce
> >>> >>>>> To: user@hadoop.apache.org
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> Hi Ranjini,
> >>> >>>>>
> >>> >>>>> A good example to look into :
> >>> >>>>> http://www.undercloud.org/?p=408
> >>> >>>>>
> >>> >>>>> Cheers,
> >>> >>>>> Subroto Sanyal
> >>> >>>>>
> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>> >>>>>
> >>> >>>>> Hi,
> >>> >>>>>
> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
> hive
> >>> >>>>> using java.
> >>> >>>>>
> >>> >>>>> Please provide sample code.
> >>> >>>>>
> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>> >>>>> should be used.
> >>> >>>>>
> >>> >>>>> Thanks in advance.
> >>> >>>>>
> >>> >>>>> Ranjini
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >
> >>>
> >>
> >
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
The driver class and my Mapper class i have used
org.apache.hadoop.mapreduce.lib
and in the XmlInputFormat.java class also i have used the
org.apache.hadoop.mapreduce.lib
but still iam getting this error.
Please suggest.
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
> Hello Ranjini,
> This error will come when you use mix and match newer and older API..
>
> You might have written program using newer API and the the XML input
> format is using older api..
> The older api has package structure of org.apache.hadoop.mapred
>
> The newer api has package structure package of
> org.apache.hadoop.mapreduce.lib
>
> Check out the XMLINputFormat.java, which package of FileInputFormat
> they have used...
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
> <ra...@gmail.com> wrote:
> > Hi,
> >
> > I am using hadoop 0.20 version
> >
> > In that while exceuting the XmlInformat class
> > I am getting the error as
> >
> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> > interface was excepted,."
> >
> > Please suggest to fix the error.
> >
> > Thanks in advance.
> >
> > Ranjini
> >
> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >> hi,
> >>
> >> I have fixed the error , the code is running fine, but this code just
> >> split the part of the tag.
> >>
> >> i want to convert into text format so that i can load them into tables
> of
> >> hbase and hive.
> >>
> >> I have used the DOM Parser but this parser uses File as Object but hdfs
> >> uses FileSystem.
> >>
> >> Eg,
> >>
> >> File fXmlFile = new File("D:/elango/test.xml");
> >>
> >> System.out.println(g);
> >> DocumentBuilderFactory dbFactory =
> DocumentBuilderFactory.newInstance();
> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> >> Document doc = dBuilder.parse(fXmlFile);
> >>
> >>
> >> This cant be used as hdfs, because hdfs path is accessed through
> >> FileSystem.
> >>
> >> I kindly request u to , Please suggest me to fix the above issue.
> >>
> >> Thanks in advance
> >>
> >> Ranjini R
> >>
> >>
> >>
> >>
> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
> >> <ra...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> ---------- Forwarded message ----------
> >>> From: Shekhar Sharma <sh...@gmail.com>
> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
> >>> Subject: Re: Hadoop-MapReduce
> >>> To: user@hadoop.apache.org
> >>> Cc: ssanyal@datameer.com
> >>>
> >>>
> >>> It does work i have used it long back..
> >>>
> >>> BTW if it is not working, write the custom input format and implement
> >>> your record reader. That would be far more easy than breaking your
> >>> head with others code.
> >>>
> >>> Break your problem in step:
> >>>
> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
> >>> single record for you...May be a record for you would be
> >>>
> >>> <person>
> >>> <fname>x</fname>
> >>> <lname>y</lname>
> >>> </person>
> >>>
> >>> (2) Implement a record reader that looks out for the starting and
> >>> ending person tag ( Checkout how RecordReader.java is written)
> >>>
> >>> (3) Once you got the contents between starting and ending tag, now you
> >>> can use a xml parser to parse the contents into an java object and
> >>> form your own key value pairs ( custom key and custom value)
> >>>
> >>>
> >>> Hope you have enough pointers to write the code.
> >>>
> >>>
> >>> Regards,
> >>> Som Shekhar Sharma
> >>> +91-8197243810
> >>>
> >>>
> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> >>> wrote:
> >>> > Hi Subroto Sanyal,
> >>> >
> >>> > The link provided about xml, it does not work . The Class written
> >>> > XmlContent is not allowed in the XmlInputFormat.
> >>> >
> >>> > I request you to help , whether this scenaio some one has coded, and
> >>> > needed
> >>> > working code.
> >>> >
> >>> > I have written using SAX Parser too, but eventhough the jars are
> added
> >>> > in
> >>> > classpath THe error is is coming has NoClasFoung Exception.
> >>> >
> >>> > Please provide sample code for the same.
> >>> >
> >>> > Thanks in advance,
> >>> > Ranjini.R
> >>> >
> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
> >>> > <ra...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >>
> >>> >>>> Hi,
> >>> >>>>
> >>> >>>> As suggest by the link below , i have used for my program ,
> >>> >>>>
> >>> >>>> but i am facing the below issues, please help me to fix these
> error.
> >>> >>>>
> >>> >>>>
> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
> >>> >>>> override
> >>> >>>> abstract method
> >>> >>>>
> >>> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>> >>>> in org.apache.hadoop.mapred.Mapper
> >>> >>>> public static class Map extends MapReduceBase implements Mapper
> >>> >>>> <LongWritable, Text, Text, Text> {
> >>> >>>> ^
> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> >>> >>>> abstract
> >>> >>>> and does not override abstract method
> >>> >>>> next(java.lang.Object,java.lang.Object) in
> >>> >>>> org.apache.hadoop.mapred.RecordReader
> >>> >>>> public class XmlRecordReader implements RecordReader {
> >>> >>>> ^
> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>> >>>> 2 errors
> >>> >>>>
> >>> >>>>
> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>> >>>>
> >>> >>>> Please suggest.
> >>> >>>>
> >>> >>>> Thanks in advance.
> >>> >>>>
> >>> >>>> Regrads,
> >>> >>>> Ranjini. R
> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>> >>>> <ra...@gmail.com> wrote:
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> ---------- Forwarded message ----------
> >>> >>>>> From: Subroto <ss...@datameer.com>
> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>> >>>>> Subject: Re: Hadoop-MapReduce
> >>> >>>>> To: user@hadoop.apache.org
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> Hi Ranjini,
> >>> >>>>>
> >>> >>>>> A good example to look into :
> >>> >>>>> http://www.undercloud.org/?p=408
> >>> >>>>>
> >>> >>>>> Cheers,
> >>> >>>>> Subroto Sanyal
> >>> >>>>>
> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>> >>>>>
> >>> >>>>> Hi,
> >>> >>>>>
> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
> hive
> >>> >>>>> using java.
> >>> >>>>>
> >>> >>>>> Please provide sample code.
> >>> >>>>>
> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>> >>>>> should be used.
> >>> >>>>>
> >>> >>>>> Thanks in advance.
> >>> >>>>>
> >>> >>>>> Ranjini
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >
> >>>
> >>
> >
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
The driver class and my Mapper class i have used
org.apache.hadoop.mapreduce.lib
and in the XmlInputFormat.java class also i have used the
org.apache.hadoop.mapreduce.lib
but still iam getting this error.
Please suggest.
Thanks in advance
Ranjini
On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <sh...@gmail.com>wrote:
> Hello Ranjini,
> This error will come when you use mix and match newer and older API..
>
> You might have written program using newer API and the the XML input
> format is using older api..
> The older api has package structure of org.apache.hadoop.mapred
>
> The newer api has package structure package of
> org.apache.hadoop.mapreduce.lib
>
> Check out the XMLINputFormat.java, which package of FileInputFormat
> they have used...
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
> <ra...@gmail.com> wrote:
> > Hi,
> >
> > I am using hadoop 0.20 version
> >
> > In that while exceuting the XmlInformat class
> > I am getting the error as
> >
> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> > interface was excepted,."
> >
> > Please suggest to fix the error.
> >
> > Thanks in advance.
> >
> > Ranjini
> >
> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >> hi,
> >>
> >> I have fixed the error , the code is running fine, but this code just
> >> split the part of the tag.
> >>
> >> i want to convert into text format so that i can load them into tables
> of
> >> hbase and hive.
> >>
> >> I have used the DOM Parser but this parser uses File as Object but hdfs
> >> uses FileSystem.
> >>
> >> Eg,
> >>
> >> File fXmlFile = new File("D:/elango/test.xml");
> >>
> >> System.out.println(g);
> >> DocumentBuilderFactory dbFactory =
> DocumentBuilderFactory.newInstance();
> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> >> Document doc = dBuilder.parse(fXmlFile);
> >>
> >>
> >> This cant be used as hdfs, because hdfs path is accessed through
> >> FileSystem.
> >>
> >> I kindly request u to , Please suggest me to fix the above issue.
> >>
> >> Thanks in advance
> >>
> >> Ranjini R
> >>
> >>
> >>
> >>
> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
> >> <ra...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> ---------- Forwarded message ----------
> >>> From: Shekhar Sharma <sh...@gmail.com>
> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
> >>> Subject: Re: Hadoop-MapReduce
> >>> To: user@hadoop.apache.org
> >>> Cc: ssanyal@datameer.com
> >>>
> >>>
> >>> It does work i have used it long back..
> >>>
> >>> BTW if it is not working, write the custom input format and implement
> >>> your record reader. That would be far more easy than breaking your
> >>> head with others code.
> >>>
> >>> Break your problem in step:
> >>>
> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
> >>> single record for you...May be a record for you would be
> >>>
> >>> <person>
> >>> <fname>x</fname>
> >>> <lname>y</lname>
> >>> </person>
> >>>
> >>> (2) Implement a record reader that looks out for the starting and
> >>> ending person tag ( Checkout how RecordReader.java is written)
> >>>
> >>> (3) Once you got the contents between starting and ending tag, now you
> >>> can use a xml parser to parse the contents into an java object and
> >>> form your own key value pairs ( custom key and custom value)
> >>>
> >>>
> >>> Hope you have enough pointers to write the code.
> >>>
> >>>
> >>> Regards,
> >>> Som Shekhar Sharma
> >>> +91-8197243810
> >>>
> >>>
> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> >>> wrote:
> >>> > Hi Subroto Sanyal,
> >>> >
> >>> > The link provided about xml, it does not work . The Class written
> >>> > XmlContent is not allowed in the XmlInputFormat.
> >>> >
> >>> > I request you to help , whether this scenaio some one has coded, and
> >>> > needed
> >>> > working code.
> >>> >
> >>> > I have written using SAX Parser too, but eventhough the jars are
> added
> >>> > in
> >>> > classpath THe error is is coming has NoClasFoung Exception.
> >>> >
> >>> > Please provide sample code for the same.
> >>> >
> >>> > Thanks in advance,
> >>> > Ranjini.R
> >>> >
> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
> >>> > <ra...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >>
> >>> >>>> Hi,
> >>> >>>>
> >>> >>>> As suggest by the link below , i have used for my program ,
> >>> >>>>
> >>> >>>> but i am facing the below issues, please help me to fix these
> error.
> >>> >>>>
> >>> >>>>
> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
> >>> >>>> override
> >>> >>>> abstract method
> >>> >>>>
> >>> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>> >>>> in org.apache.hadoop.mapred.Mapper
> >>> >>>> public static class Map extends MapReduceBase implements Mapper
> >>> >>>> <LongWritable, Text, Text, Text> {
> >>> >>>> ^
> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> >>> >>>> abstract
> >>> >>>> and does not override abstract method
> >>> >>>> next(java.lang.Object,java.lang.Object) in
> >>> >>>> org.apache.hadoop.mapred.RecordReader
> >>> >>>> public class XmlRecordReader implements RecordReader {
> >>> >>>> ^
> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>> >>>> 2 errors
> >>> >>>>
> >>> >>>>
> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>> >>>>
> >>> >>>> Please suggest.
> >>> >>>>
> >>> >>>> Thanks in advance.
> >>> >>>>
> >>> >>>> Regrads,
> >>> >>>> Ranjini. R
> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>> >>>> <ra...@gmail.com> wrote:
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> ---------- Forwarded message ----------
> >>> >>>>> From: Subroto <ss...@datameer.com>
> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>> >>>>> Subject: Re: Hadoop-MapReduce
> >>> >>>>> To: user@hadoop.apache.org
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> Hi Ranjini,
> >>> >>>>>
> >>> >>>>> A good example to look into :
> >>> >>>>> http://www.undercloud.org/?p=408
> >>> >>>>>
> >>> >>>>> Cheers,
> >>> >>>>> Subroto Sanyal
> >>> >>>>>
> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>> >>>>>
> >>> >>>>> Hi,
> >>> >>>>>
> >>> >>>>> How to read xml file via mapreduce and load them in hbase and
> hive
> >>> >>>>> using java.
> >>> >>>>>
> >>> >>>>> Please provide sample code.
> >>> >>>>>
> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>> >>>>> should be used.
> >>> >>>>>
> >>> >>>>> Thanks in advance.
> >>> >>>>>
> >>> >>>>> Ranjini
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >
> >>>
> >>
> >
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Hello Ranjini,
This error will come when you use mix and match newer and older API..
You might have written program using newer API and the the XML input
format is using older api..
The older api has package structure of org.apache.hadoop.mapred
The newer api has package structure package of org.apache.hadoop.mapreduce.lib
Check out the XMLINputFormat.java, which package of FileInputFormat
they have used...
Regards,
Som Shekhar Sharma
+91-8197243810
On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
<ra...@gmail.com> wrote:
> Hi,
>
> I am using hadoop 0.20 version
>
> In that while exceuting the XmlInformat class
> I am getting the error as
>
> "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> interface was excepted,."
>
> Please suggest to fix the error.
>
> Thanks in advance.
>
> Ranjini
>
> On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>> hi,
>>
>> I have fixed the error , the code is running fine, but this code just
>> split the part of the tag.
>>
>> i want to convert into text format so that i can load them into tables of
>> hbase and hive.
>>
>> I have used the DOM Parser but this parser uses File as Object but hdfs
>> uses FileSystem.
>>
>> Eg,
>>
>> File fXmlFile = new File("D:/elango/test.xml");
>>
>> System.out.println(g);
>> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
>> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> Document doc = dBuilder.parse(fXmlFile);
>>
>>
>> This cant be used as hdfs, because hdfs path is accessed through
>> FileSystem.
>>
>> I kindly request u to , Please suggest me to fix the above issue.
>>
>> Thanks in advance
>>
>> Ranjini R
>>
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shekhar Sharma <sh...@gmail.com>
>>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>> Cc: ssanyal@datameer.com
>>>
>>>
>>> It does work i have used it long back..
>>>
>>> BTW if it is not working, write the custom input format and implement
>>> your record reader. That would be far more easy than breaking your
>>> head with others code.
>>>
>>> Break your problem in step:
>>>
>>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> single record for you...May be a record for you would be
>>>
>>> <person>
>>> <fname>x</fname>
>>> <lname>y</lname>
>>> </person>
>>>
>>> (2) Implement a record reader that looks out for the starting and
>>> ending person tag ( Checkout how RecordReader.java is written)
>>>
>>> (3) Once you got the contents between starting and ending tag, now you
>>> can use a xml parser to parse the contents into an java object and
>>> form your own key value pairs ( custom key and custom value)
>>>
>>>
>>> Hope you have enough pointers to write the code.
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>>> wrote:
>>> > Hi Subroto Sanyal,
>>> >
>>> > The link provided about xml, it does not work . The Class written
>>> > XmlContent is not allowed in the XmlInputFormat.
>>> >
>>> > I request you to help , whether this scenaio some one has coded, and
>>> > needed
>>> > working code.
>>> >
>>> > I have written using SAX Parser too, but eventhough the jars are added
>>> > in
>>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >
>>> > Please provide sample code for the same.
>>> >
>>> > Thanks in advance,
>>> > Ranjini.R
>>> >
>>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> > <ra...@gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >>>> Hi,
>>> >>>>
>>> >>>> As suggest by the link below , i have used for my program ,
>>> >>>>
>>> >>>> but i am facing the below issues, please help me to fix these error.
>>> >>>>
>>> >>>>
>>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>>> override
>>> >>>> abstract method
>>> >>>>
>>> >>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>>> <LongWritable, Text, Text, Text> {
>>> >>>> ^
>>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>>> abstract
>>> >>>> and does not override abstract method
>>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>>> ^
>>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>>> 2 errors
>>> >>>>
>>> >>>>
>>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>>>
>>> >>>> Please suggest.
>>> >>>>
>>> >>>> Thanks in advance.
>>> >>>>
>>> >>>> Regrads,
>>> >>>> Ranjini. R
>>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>>> <ra...@gmail.com> wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> ---------- Forwarded message ----------
>>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>>>> To: user@hadoop.apache.org
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi Ranjini,
>>> >>>>>
>>> >>>>> A good example to look into :
>>> >>>>> http://www.undercloud.org/?p=408
>>> >>>>>
>>> >>>>> Cheers,
>>> >>>>> Subroto Sanyal
>>> >>>>>
>>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>>>>
>>> >>>>> Hi,
>>> >>>>>
>>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>>> >>>>> using java.
>>> >>>>>
>>> >>>>> Please provide sample code.
>>> >>>>>
>>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>> >>>>> should be used.
>>> >>>>>
>>> >>>>> Thanks in advance.
>>> >>>>>
>>> >>>>> Ranjini
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Hello Ranjini,
This error will come when you use mix and match newer and older API..
You might have written program using newer API and the the XML input
format is using older api..
The older api has package structure of org.apache.hadoop.mapred
The newer api has package structure package of org.apache.hadoop.mapreduce.lib
Check out the XMLINputFormat.java, which package of FileInputFormat
they have used...
Regards,
Som Shekhar Sharma
+91-8197243810
On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
<ra...@gmail.com> wrote:
> Hi,
>
> I am using hadoop 0.20 version
>
> In that while exceuting the XmlInformat class
> I am getting the error as
>
> "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> interface was excepted,."
>
> Please suggest to fix the error.
>
> Thanks in advance.
>
> Ranjini
>
> On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>> hi,
>>
>> I have fixed the error , the code is running fine, but this code just
>> split the part of the tag.
>>
>> i want to convert into text format so that i can load them into tables of
>> hbase and hive.
>>
>> I have used the DOM Parser but this parser uses File as Object but hdfs
>> uses FileSystem.
>>
>> Eg,
>>
>> File fXmlFile = new File("D:/elango/test.xml");
>>
>> System.out.println(g);
>> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
>> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> Document doc = dBuilder.parse(fXmlFile);
>>
>>
>> This cant be used as hdfs, because hdfs path is accessed through
>> FileSystem.
>>
>> I kindly request u to , Please suggest me to fix the above issue.
>>
>> Thanks in advance
>>
>> Ranjini R
>>
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shekhar Sharma <sh...@gmail.com>
>>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>> Cc: ssanyal@datameer.com
>>>
>>>
>>> It does work i have used it long back..
>>>
>>> BTW if it is not working, write the custom input format and implement
>>> your record reader. That would be far more easy than breaking your
>>> head with others code.
>>>
>>> Break your problem in step:
>>>
>>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> single record for you...May be a record for you would be
>>>
>>> <person>
>>> <fname>x</fname>
>>> <lname>y</lname>
>>> </person>
>>>
>>> (2) Implement a record reader that looks out for the starting and
>>> ending person tag ( Checkout how RecordReader.java is written)
>>>
>>> (3) Once you got the contents between starting and ending tag, now you
>>> can use a xml parser to parse the contents into an java object and
>>> form your own key value pairs ( custom key and custom value)
>>>
>>>
>>> Hope you have enough pointers to write the code.
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>>> wrote:
>>> > Hi Subroto Sanyal,
>>> >
>>> > The link provided about xml, it does not work . The Class written
>>> > XmlContent is not allowed in the XmlInputFormat.
>>> >
>>> > I request you to help , whether this scenaio some one has coded, and
>>> > needed
>>> > working code.
>>> >
>>> > I have written using SAX Parser too, but eventhough the jars are added
>>> > in
>>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >
>>> > Please provide sample code for the same.
>>> >
>>> > Thanks in advance,
>>> > Ranjini.R
>>> >
>>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> > <ra...@gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >>>> Hi,
>>> >>>>
>>> >>>> As suggest by the link below , i have used for my program ,
>>> >>>>
>>> >>>> but i am facing the below issues, please help me to fix these error.
>>> >>>>
>>> >>>>
>>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>>> override
>>> >>>> abstract method
>>> >>>>
>>> >>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>>> <LongWritable, Text, Text, Text> {
>>> >>>> ^
>>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>>> abstract
>>> >>>> and does not override abstract method
>>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>>> ^
>>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>>> 2 errors
>>> >>>>
>>> >>>>
>>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>>>
>>> >>>> Please suggest.
>>> >>>>
>>> >>>> Thanks in advance.
>>> >>>>
>>> >>>> Regrads,
>>> >>>> Ranjini. R
>>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>>> <ra...@gmail.com> wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> ---------- Forwarded message ----------
>>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>>>> To: user@hadoop.apache.org
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi Ranjini,
>>> >>>>>
>>> >>>>> A good example to look into :
>>> >>>>> http://www.undercloud.org/?p=408
>>> >>>>>
>>> >>>>> Cheers,
>>> >>>>> Subroto Sanyal
>>> >>>>>
>>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>>>>
>>> >>>>> Hi,
>>> >>>>>
>>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>>> >>>>> using java.
>>> >>>>>
>>> >>>>> Please provide sample code.
>>> >>>>>
>>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>> >>>>> should be used.
>>> >>>>>
>>> >>>>> Thanks in advance.
>>> >>>>>
>>> >>>>> Ranjini
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Hello Ranjini,
This error will come when you use mix and match newer and older API..
You might have written program using newer API and the the XML input
format is using older api..
The older api has package structure of org.apache.hadoop.mapred
The newer api has package structure package of org.apache.hadoop.mapreduce.lib
Check out the XMLINputFormat.java, which package of FileInputFormat
they have used...
Regards,
Som Shekhar Sharma
+91-8197243810
On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
<ra...@gmail.com> wrote:
> Hi,
>
> I am using hadoop 0.20 version
>
> In that while exceuting the XmlInformat class
> I am getting the error as
>
> "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> interface was excepted,."
>
> Please suggest to fix the error.
>
> Thanks in advance.
>
> Ranjini
>
> On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>> hi,
>>
>> I have fixed the error , the code is running fine, but this code just
>> split the part of the tag.
>>
>> i want to convert into text format so that i can load them into tables of
>> hbase and hive.
>>
>> I have used the DOM Parser but this parser uses File as Object but hdfs
>> uses FileSystem.
>>
>> Eg,
>>
>> File fXmlFile = new File("D:/elango/test.xml");
>>
>> System.out.println(g);
>> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
>> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> Document doc = dBuilder.parse(fXmlFile);
>>
>>
>> This cant be used as hdfs, because hdfs path is accessed through
>> FileSystem.
>>
>> I kindly request u to , Please suggest me to fix the above issue.
>>
>> Thanks in advance
>>
>> Ranjini R
>>
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shekhar Sharma <sh...@gmail.com>
>>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>> Cc: ssanyal@datameer.com
>>>
>>>
>>> It does work i have used it long back..
>>>
>>> BTW if it is not working, write the custom input format and implement
>>> your record reader. That would be far more easy than breaking your
>>> head with others code.
>>>
>>> Break your problem in step:
>>>
>>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> single record for you...May be a record for you would be
>>>
>>> <person>
>>> <fname>x</fname>
>>> <lname>y</lname>
>>> </person>
>>>
>>> (2) Implement a record reader that looks out for the starting and
>>> ending person tag ( Checkout how RecordReader.java is written)
>>>
>>> (3) Once you got the contents between starting and ending tag, now you
>>> can use a xml parser to parse the contents into an java object and
>>> form your own key value pairs ( custom key and custom value)
>>>
>>>
>>> Hope you have enough pointers to write the code.
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>>> wrote:
>>> > Hi Subroto Sanyal,
>>> >
>>> > The link provided about xml, it does not work . The Class written
>>> > XmlContent is not allowed in the XmlInputFormat.
>>> >
>>> > I request you to help , whether this scenaio some one has coded, and
>>> > needed
>>> > working code.
>>> >
>>> > I have written using SAX Parser too, but eventhough the jars are added
>>> > in
>>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >
>>> > Please provide sample code for the same.
>>> >
>>> > Thanks in advance,
>>> > Ranjini.R
>>> >
>>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> > <ra...@gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >>>> Hi,
>>> >>>>
>>> >>>> As suggest by the link below , i have used for my program ,
>>> >>>>
>>> >>>> but i am facing the below issues, please help me to fix these error.
>>> >>>>
>>> >>>>
>>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>>> override
>>> >>>> abstract method
>>> >>>>
>>> >>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>>> <LongWritable, Text, Text, Text> {
>>> >>>> ^
>>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>>> abstract
>>> >>>> and does not override abstract method
>>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>>> ^
>>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>>> 2 errors
>>> >>>>
>>> >>>>
>>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>>>
>>> >>>> Please suggest.
>>> >>>>
>>> >>>> Thanks in advance.
>>> >>>>
>>> >>>> Regrads,
>>> >>>> Ranjini. R
>>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>>> <ra...@gmail.com> wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> ---------- Forwarded message ----------
>>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>>>> To: user@hadoop.apache.org
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi Ranjini,
>>> >>>>>
>>> >>>>> A good example to look into :
>>> >>>>> http://www.undercloud.org/?p=408
>>> >>>>>
>>> >>>>> Cheers,
>>> >>>>> Subroto Sanyal
>>> >>>>>
>>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>>>>
>>> >>>>> Hi,
>>> >>>>>
>>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>>> >>>>> using java.
>>> >>>>>
>>> >>>>> Please provide sample code.
>>> >>>>>
>>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>> >>>>> should be used.
>>> >>>>>
>>> >>>>> Thanks in advance.
>>> >>>>>
>>> >>>>> Ranjini
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
Hello Ranjini,
This error will come when you use mix and match newer and older API..
You might have written program using newer API and the the XML input
format is using older api..
The older api has package structure of org.apache.hadoop.mapred
The newer api has package structure package of org.apache.hadoop.mapreduce.lib
Check out the XMLINputFormat.java, which package of FileInputFormat
they have used...
Regards,
Som Shekhar Sharma
+91-8197243810
On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
<ra...@gmail.com> wrote:
> Hi,
>
> I am using hadoop 0.20 version
>
> In that while exceuting the XmlInformat class
> I am getting the error as
>
> "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
> interface was excepted,."
>
> Please suggest to fix the error.
>
> Thanks in advance.
>
> Ranjini
>
> On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>> hi,
>>
>> I have fixed the error , the code is running fine, but this code just
>> split the part of the tag.
>>
>> i want to convert into text format so that i can load them into tables of
>> hbase and hive.
>>
>> I have used the DOM Parser but this parser uses File as Object but hdfs
>> uses FileSystem.
>>
>> Eg,
>>
>> File fXmlFile = new File("D:/elango/test.xml");
>>
>> System.out.println(g);
>> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
>> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> Document doc = dBuilder.parse(fXmlFile);
>>
>>
>> This cant be used as hdfs, because hdfs path is accessed through
>> FileSystem.
>>
>> I kindly request u to , Please suggest me to fix the above issue.
>>
>> Thanks in advance
>>
>> Ranjini R
>>
>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> <ra...@gmail.com> wrote:
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shekhar Sharma <sh...@gmail.com>
>>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>> Cc: ssanyal@datameer.com
>>>
>>>
>>> It does work i have used it long back..
>>>
>>> BTW if it is not working, write the custom input format and implement
>>> your record reader. That would be far more easy than breaking your
>>> head with others code.
>>>
>>> Break your problem in step:
>>>
>>> (1) First the XML data is multiline...Meaning multiple lines makes a
>>> single record for you...May be a record for you would be
>>>
>>> <person>
>>> <fname>x</fname>
>>> <lname>y</lname>
>>> </person>
>>>
>>> (2) Implement a record reader that looks out for the starting and
>>> ending person tag ( Checkout how RecordReader.java is written)
>>>
>>> (3) Once you got the contents between starting and ending tag, now you
>>> can use a xml parser to parse the contents into an java object and
>>> form your own key value pairs ( custom key and custom value)
>>>
>>>
>>> Hope you have enough pointers to write the code.
>>>
>>>
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>>> wrote:
>>> > Hi Subroto Sanyal,
>>> >
>>> > The link provided about xml, it does not work . The Class written
>>> > XmlContent is not allowed in the XmlInputFormat.
>>> >
>>> > I request you to help , whether this scenaio some one has coded, and
>>> > needed
>>> > working code.
>>> >
>>> > I have written using SAX Parser too, but eventhough the jars are added
>>> > in
>>> > classpath THe error is is coming has NoClasFoung Exception.
>>> >
>>> > Please provide sample code for the same.
>>> >
>>> > Thanks in advance,
>>> > Ranjini.R
>>> >
>>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> > <ra...@gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >>>> Hi,
>>> >>>>
>>> >>>> As suggest by the link below , i have used for my program ,
>>> >>>>
>>> >>>> but i am facing the below issues, please help me to fix these error.
>>> >>>>
>>> >>>>
>>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not
>>> >>>> override
>>> >>>> abstract method
>>> >>>>
>>> >>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>>> in org.apache.hadoop.mapred.Mapper
>>> >>>> public static class Map extends MapReduceBase implements Mapper
>>> >>>> <LongWritable, Text, Text, Text> {
>>> >>>> ^
>>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>>> >>>> abstract
>>> >>>> and does not override abstract method
>>> >>>> next(java.lang.Object,java.lang.Object) in
>>> >>>> org.apache.hadoop.mapred.RecordReader
>>> >>>> public class XmlRecordReader implements RecordReader {
>>> >>>> ^
>>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> >>>> Note: Recompile with -Xlint:unchecked for details.
>>> >>>> 2 errors
>>> >>>>
>>> >>>>
>>> >>>> i am using hadoop 0.20 version and java 1.6 .
>>> >>>>
>>> >>>> Please suggest.
>>> >>>>
>>> >>>> Thanks in advance.
>>> >>>>
>>> >>>> Regrads,
>>> >>>> Ranjini. R
>>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>>> <ra...@gmail.com> wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> ---------- Forwarded message ----------
>>> >>>>> From: Subroto <ss...@datameer.com>
>>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>>>> Subject: Re: Hadoop-MapReduce
>>> >>>>> To: user@hadoop.apache.org
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi Ranjini,
>>> >>>>>
>>> >>>>> A good example to look into :
>>> >>>>> http://www.undercloud.org/?p=408
>>> >>>>>
>>> >>>>> Cheers,
>>> >>>>> Subroto Sanyal
>>> >>>>>
>>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>> >>>>>
>>> >>>>> Hi,
>>> >>>>>
>>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>>> >>>>> using java.
>>> >>>>>
>>> >>>>> Please provide sample code.
>>> >>>>>
>>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>> >>>>> should be used.
>>> >>>>>
>>> >>>>> Thanks in advance.
>>> >>>>>
>>> >>>>> Ranjini
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I am using hadoop 0.20 version
In that while exceuting the XmlInformat class
I am getting the error as
"Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
interface was excepted,."
Please suggest to fix the error.
Thanks in advance.
Ranjini
On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam
<ra...@gmail.com>wrote:
> hi,
>
> I have fixed the error , the code is running fine, but this code just
> split the part of the tag.
>
> i want to convert into text format so that i can load them into tables of
> hbase and hive.
>
> I have used the DOM Parser but this parser uses File as Object but hdfs
> uses FileSystem.
>
> Eg,
>
> File fXmlFile = new File("D:/elango/test.xml");
>
> System.out.println(g);
> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> Document doc = dBuilder.parse(fXmlFile);
>
>
> This cant be used as hdfs, because hdfs path is accessed through
> FileSystem.
>
> I kindly request u to , Please suggest me to fix the above issue.
>
> Thanks in advance
>
> Ranjini R
>
>
>
>
> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Shekhar Sharma <sh...@gmail.com>
>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>> Cc: ssanyal@datameer.com
>>
>>
>> It does work i have used it long back..
>>
>> BTW if it is not working, write the custom input format and implement
>> your record reader. That would be far more easy than breaking your
>> head with others code.
>>
>> Break your problem in step:
>>
>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> single record for you...May be a record for you would be
>>
>> <person>
>> <fname>x</fname>
>> <lname>y</lname>
>> </person>
>>
>> (2) Implement a record reader that looks out for the starting and
>> ending person tag ( Checkout how RecordReader.java is written)
>>
>> (3) Once you got the contents between starting and ending tag, now you
>> can use a xml parser to parse the contents into an java object and
>> form your own key value pairs ( custom key and custom value)
>>
>>
>> Hope you have enough pointers to write the code.
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>> wrote:
>> > Hi Subroto Sanyal,
>> >
>> > The link provided about xml, it does not work . The Class written
>> > XmlContent is not allowed in the XmlInputFormat.
>> >
>> > I request you to help , whether this scenaio some one has coded, and
>> needed
>> > working code.
>> >
>> > I have written using SAX Parser too, but eventhough the jars are added
>> in
>> > classpath THe error is is coming has NoClasFoung Exception.
>> >
>> > Please provide sample code for the same.
>> >
>> > Thanks in advance,
>> > Ranjini.R
>> >
>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >>
>> >>>> Hi,
>> >>>>
>> >>>> As suggest by the link below , i have used for my program ,
>> >>>>
>> >>>> but i am facing the below issues, please help me to fix these error.
>> >>>>
>> >>>>
>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> >>>> abstract method
>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>>> <LongWritable, Text, Text, Text> {
>> >>>> ^
>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> abstract
>> >>>> and does not override abstract method
>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>>> public class XmlRecordReader implements RecordReader {
>> >>>> ^
>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>>> 2 errors
>> >>>>
>> >>>>
>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>>>
>> >>>> Please suggest.
>> >>>>
>> >>>> Thanks in advance.
>> >>>>
>> >>>> Regrads,
>> >>>> Ranjini. R
>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>>> <ra...@gmail.com> wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ---------- Forwarded message ----------
>> >>>>> From: Subroto <ss...@datameer.com>
>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>>>> To: user@hadoop.apache.org
>> >>>>>
>> >>>>>
>> >>>>> Hi Ranjini,
>> >>>>>
>> >>>>> A good example to look into :
>> >>>>> http://www.undercloud.org/?p=408
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Subroto Sanyal
>> >>>>>
>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>> >>>>> using java.
>> >>>>>
>> >>>>> Please provide sample code.
>> >>>>>
>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>> >>>>> should be used.
>> >>>>>
>> >>>>> Thanks in advance.
>> >>>>>
>> >>>>> Ranjini
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I am using hadoop 0.20 version
In that while exceuting the XmlInformat class
I am getting the error as
"Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
interface was excepted,."
Please suggest to fix the error.
Thanks in advance.
Ranjini
On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam
<ra...@gmail.com>wrote:
> hi,
>
> I have fixed the error , the code is running fine, but this code just
> split the part of the tag.
>
> i want to convert into text format so that i can load them into tables of
> hbase and hive.
>
> I have used the DOM Parser but this parser uses File as Object but hdfs
> uses FileSystem.
>
> Eg,
>
> File fXmlFile = new File("D:/elango/test.xml");
>
> System.out.println(g);
> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> Document doc = dBuilder.parse(fXmlFile);
>
>
> This cant be used as hdfs, because hdfs path is accessed through
> FileSystem.
>
> I kindly request u to , Please suggest me to fix the above issue.
>
> Thanks in advance
>
> Ranjini R
>
>
>
>
> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Shekhar Sharma <sh...@gmail.com>
>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>> Cc: ssanyal@datameer.com
>>
>>
>> It does work i have used it long back..
>>
>> BTW if it is not working, write the custom input format and implement
>> your record reader. That would be far more easy than breaking your
>> head with others code.
>>
>> Break your problem in step:
>>
>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> single record for you...May be a record for you would be
>>
>> <person>
>> <fname>x</fname>
>> <lname>y</lname>
>> </person>
>>
>> (2) Implement a record reader that looks out for the starting and
>> ending person tag ( Checkout how RecordReader.java is written)
>>
>> (3) Once you got the contents between starting and ending tag, now you
>> can use a xml parser to parse the contents into an java object and
>> form your own key value pairs ( custom key and custom value)
>>
>>
>> Hope you have enough pointers to write the code.
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>> wrote:
>> > Hi Subroto Sanyal,
>> >
>> > The link provided about xml, it does not work . The Class written
>> > XmlContent is not allowed in the XmlInputFormat.
>> >
>> > I request you to help , whether this scenaio some one has coded, and
>> needed
>> > working code.
>> >
>> > I have written using SAX Parser too, but eventhough the jars are added
>> in
>> > classpath THe error is is coming has NoClasFoung Exception.
>> >
>> > Please provide sample code for the same.
>> >
>> > Thanks in advance,
>> > Ranjini.R
>> >
>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >>
>> >>>> Hi,
>> >>>>
>> >>>> As suggest by the link below , i have used for my program ,
>> >>>>
>> >>>> but i am facing the below issues, please help me to fix these error.
>> >>>>
>> >>>>
>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> >>>> abstract method
>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>>> <LongWritable, Text, Text, Text> {
>> >>>> ^
>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> abstract
>> >>>> and does not override abstract method
>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>>> public class XmlRecordReader implements RecordReader {
>> >>>> ^
>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>>> 2 errors
>> >>>>
>> >>>>
>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>>>
>> >>>> Please suggest.
>> >>>>
>> >>>> Thanks in advance.
>> >>>>
>> >>>> Regrads,
>> >>>> Ranjini. R
>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>>> <ra...@gmail.com> wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ---------- Forwarded message ----------
>> >>>>> From: Subroto <ss...@datameer.com>
>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>>>> To: user@hadoop.apache.org
>> >>>>>
>> >>>>>
>> >>>>> Hi Ranjini,
>> >>>>>
>> >>>>> A good example to look into :
>> >>>>> http://www.undercloud.org/?p=408
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Subroto Sanyal
>> >>>>>
>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>> >>>>> using java.
>> >>>>>
>> >>>>> Please provide sample code.
>> >>>>>
>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>> >>>>> should be used.
>> >>>>>
>> >>>>> Thanks in advance.
>> >>>>>
>> >>>>> Ranjini
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I am using hadoop 0.20 version
In that while exceuting the XmlInformat class
I am getting the error as
"Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
interface was excepted,."
Please suggest to fix the error.
Thanks in advance.
Ranjini
On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam
<ra...@gmail.com>wrote:
> hi,
>
> I have fixed the error , the code is running fine, but this code just
> split the part of the tag.
>
> i want to convert into text format so that i can load them into tables of
> hbase and hive.
>
> I have used the DOM Parser but this parser uses File as Object but hdfs
> uses FileSystem.
>
> Eg,
>
> File fXmlFile = new File("D:/elango/test.xml");
>
> System.out.println(g);
> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> Document doc = dBuilder.parse(fXmlFile);
>
>
> This cant be used as hdfs, because hdfs path is accessed through
> FileSystem.
>
> I kindly request u to , Please suggest me to fix the above issue.
>
> Thanks in advance
>
> Ranjini R
>
>
>
>
> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Shekhar Sharma <sh...@gmail.com>
>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>> Cc: ssanyal@datameer.com
>>
>>
>> It does work i have used it long back..
>>
>> BTW if it is not working, write the custom input format and implement
>> your record reader. That would be far more easy than breaking your
>> head with others code.
>>
>> Break your problem in step:
>>
>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> single record for you...May be a record for you would be
>>
>> <person>
>> <fname>x</fname>
>> <lname>y</lname>
>> </person>
>>
>> (2) Implement a record reader that looks out for the starting and
>> ending person tag ( Checkout how RecordReader.java is written)
>>
>> (3) Once you got the contents between starting and ending tag, now you
>> can use a xml parser to parse the contents into an java object and
>> form your own key value pairs ( custom key and custom value)
>>
>>
>> Hope you have enough pointers to write the code.
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>> wrote:
>> > Hi Subroto Sanyal,
>> >
>> > The link provided about xml, it does not work . The Class written
>> > XmlContent is not allowed in the XmlInputFormat.
>> >
>> > I request you to help , whether this scenaio some one has coded, and
>> needed
>> > working code.
>> >
>> > I have written using SAX Parser too, but eventhough the jars are added
>> in
>> > classpath THe error is is coming has NoClasFoung Exception.
>> >
>> > Please provide sample code for the same.
>> >
>> > Thanks in advance,
>> > Ranjini.R
>> >
>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >>
>> >>>> Hi,
>> >>>>
>> >>>> As suggest by the link below , i have used for my program ,
>> >>>>
>> >>>> but i am facing the below issues, please help me to fix these error.
>> >>>>
>> >>>>
>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> >>>> abstract method
>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>>> <LongWritable, Text, Text, Text> {
>> >>>> ^
>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> abstract
>> >>>> and does not override abstract method
>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>>> public class XmlRecordReader implements RecordReader {
>> >>>> ^
>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>>> 2 errors
>> >>>>
>> >>>>
>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>>>
>> >>>> Please suggest.
>> >>>>
>> >>>> Thanks in advance.
>> >>>>
>> >>>> Regrads,
>> >>>> Ranjini. R
>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>>> <ra...@gmail.com> wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ---------- Forwarded message ----------
>> >>>>> From: Subroto <ss...@datameer.com>
>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>>>> To: user@hadoop.apache.org
>> >>>>>
>> >>>>>
>> >>>>> Hi Ranjini,
>> >>>>>
>> >>>>> A good example to look into :
>> >>>>> http://www.undercloud.org/?p=408
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Subroto Sanyal
>> >>>>>
>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>> >>>>> using java.
>> >>>>>
>> >>>>> Please provide sample code.
>> >>>>>
>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>> >>>>> should be used.
>> >>>>>
>> >>>>> Thanks in advance.
>> >>>>>
>> >>>>> Ranjini
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi,
I am using hadoop 0.20 version
In that while exceuting the XmlInformat class
I am getting the error as
"Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but
interface was excepted,."
Please suggest to fix the error.
Thanks in advance.
Ranjini
On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam
<ra...@gmail.com>wrote:
> hi,
>
> I have fixed the error , the code is running fine, but this code just
> split the part of the tag.
>
> i want to convert into text format so that i can load them into tables of
> hbase and hive.
>
> I have used the DOM Parser but this parser uses File as Object but hdfs
> uses FileSystem.
>
> Eg,
>
> File fXmlFile = new File("D:/elango/test.xml");
>
> System.out.println(g);
> DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
> Document doc = dBuilder.parse(fXmlFile);
>
>
> This cant be used as hdfs, because hdfs path is accessed through
> FileSystem.
>
> I kindly request u to , Please suggest me to fix the above issue.
>
> Thanks in advance
>
> Ranjini R
>
>
>
>
> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Shekhar Sharma <sh...@gmail.com>
>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>> Cc: ssanyal@datameer.com
>>
>>
>> It does work i have used it long back..
>>
>> BTW if it is not working, write the custom input format and implement
>> your record reader. That would be far more easy than breaking your
>> head with others code.
>>
>> Break your problem in step:
>>
>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> single record for you...May be a record for you would be
>>
>> <person>
>> <fname>x</fname>
>> <lname>y</lname>
>> </person>
>>
>> (2) Implement a record reader that looks out for the starting and
>> ending person tag ( Checkout how RecordReader.java is written)
>>
>> (3) Once you got the contents between starting and ending tag, now you
>> can use a xml parser to parse the contents into an java object and
>> form your own key value pairs ( custom key and custom value)
>>
>>
>> Hope you have enough pointers to write the code.
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
>> wrote:
>> > Hi Subroto Sanyal,
>> >
>> > The link provided about xml, it does not work . The Class written
>> > XmlContent is not allowed in the XmlInputFormat.
>> >
>> > I request you to help , whether this scenaio some one has coded, and
>> needed
>> > working code.
>> >
>> > I have written using SAX Parser too, but eventhough the jars are added
>> in
>> > classpath THe error is is coming has NoClasFoung Exception.
>> >
>> > Please provide sample code for the same.
>> >
>> > Thanks in advance,
>> > Ranjini.R
>> >
>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >>
>> >>>> Hi,
>> >>>>
>> >>>> As suggest by the link below , i have used for my program ,
>> >>>>
>> >>>> but i am facing the below issues, please help me to fix these error.
>> >>>>
>> >>>>
>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> >>>> abstract method
>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>>> public static class Map extends MapReduceBase implements Mapper
>> >>>> <LongWritable, Text, Text, Text> {
>> >>>> ^
>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
>> abstract
>> >>>> and does not override abstract method
>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>>> public class XmlRecordReader implements RecordReader {
>> >>>> ^
>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>>> 2 errors
>> >>>>
>> >>>>
>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>>>
>> >>>> Please suggest.
>> >>>>
>> >>>> Thanks in advance.
>> >>>>
>> >>>> Regrads,
>> >>>> Ranjini. R
>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>>> <ra...@gmail.com> wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ---------- Forwarded message ----------
>> >>>>> From: Subroto <ss...@datameer.com>
>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>>>> To: user@hadoop.apache.org
>> >>>>>
>> >>>>>
>> >>>>> Hi Ranjini,
>> >>>>>
>> >>>>> A good example to look into :
>> >>>>> http://www.undercloud.org/?p=408
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Subroto Sanyal
>> >>>>>
>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> How to read xml file via mapreduce and load them in hbase and hive
>> >>>>> using java.
>> >>>>>
>> >>>>> Please provide sample code.
>> >>>>>
>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>> >>>>> should be used.
>> >>>>>
>> >>>>> Thanks in advance.
>> >>>>>
>> >>>>> Ranjini
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
hi,
I have fixed the error , the code is running fine, but this code just split
the part of the tag.
i want to convert into text format so that i can load them into tables of
hbase and hive.
I have used the DOM Parser but this parser uses File as Object but hdfs
uses FileSystem.
Eg,
File fXmlFile = new File("D:/elango/test.xml");
System.out.println(g);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(fXmlFile);
This cant be used as hdfs, because hdfs path is accessed through
FileSystem.
I kindly request u to , Please suggest me to fix the above issue.
Thanks in advance
Ranjini R
On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
<ra...@gmail.com>wrote:
>
>
> ---------- Forwarded message ----------
> From: Shekhar Sharma <sh...@gmail.com>
> Date: Mon, Dec 9, 2013 at 10:23 PM
> Subject: Re: Hadoop-MapReduce
> To: user@hadoop.apache.org
> Cc: ssanyal@datameer.com
>
>
> It does work i have used it long back..
>
> BTW if it is not working, write the custom input format and implement
> your record reader. That would be far more easy than breaking your
> head with others code.
>
> Break your problem in step:
>
> (1) First the XML data is multiline...Meaning multiple lines makes a
> single record for you...May be a record for you would be
>
> <person>
> <fname>x</fname>
> <lname>y</lname>
> </person>
>
> (2) Implement a record reader that looks out for the starting and
> ending person tag ( Checkout how RecordReader.java is written)
>
> (3) Once you got the contents between starting and ending tag, now you
> can use a xml parser to parse the contents into an java object and
> form your own key value pairs ( custom key and custom value)
>
>
> Hope you have enough pointers to write the code.
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
> > Hi Subroto Sanyal,
> >
> > The link provided about xml, it does not work . The Class written
> > XmlContent is not allowed in the XmlInputFormat.
> >
> > I request you to help , whether this scenaio some one has coded, and
> needed
> > working code.
> >
> > I have written using SAX Parser too, but eventhough the jars are added in
> > classpath THe error is is coming has NoClasFoung Exception.
> >
> > Please provide sample code for the same.
> >
> > Thanks in advance,
> > Ranjini.R
> >
> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >>
> >>>> Hi,
> >>>>
> >>>> As suggest by the link below , i have used for my program ,
> >>>>
> >>>> but i am facing the below issues, please help me to fix these error.
> >>>>
> >>>>
> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> >>>> abstract method
> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>>> in org.apache.hadoop.mapred.Mapper
> >>>> public static class Map extends MapReduceBase implements Mapper
> >>>> <LongWritable, Text, Text, Text> {
> >>>> ^
> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> abstract
> >>>> and does not override abstract method
> >>>> next(java.lang.Object,java.lang.Object) in
> >>>> org.apache.hadoop.mapred.RecordReader
> >>>> public class XmlRecordReader implements RecordReader {
> >>>> ^
> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>>> 2 errors
> >>>>
> >>>>
> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>>>
> >>>> Please suggest.
> >>>>
> >>>> Thanks in advance.
> >>>>
> >>>> Regrads,
> >>>> Ranjini. R
> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>>> <ra...@gmail.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> ---------- Forwarded message ----------
> >>>>> From: Subroto <ss...@datameer.com>
> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>>>> Subject: Re: Hadoop-MapReduce
> >>>>> To: user@hadoop.apache.org
> >>>>>
> >>>>>
> >>>>> Hi Ranjini,
> >>>>>
> >>>>> A good example to look into :
> >>>>> http://www.undercloud.org/?p=408
> >>>>>
> >>>>> Cheers,
> >>>>> Subroto Sanyal
> >>>>>
> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> How to read xml file via mapreduce and load them in hbase and hive
> >>>>> using java.
> >>>>>
> >>>>> Please provide sample code.
> >>>>>
> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>>>> should be used.
> >>>>>
> >>>>> Thanks in advance.
> >>>>>
> >>>>> Ranjini
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
hi,
I have fixed the error , the code is running fine, but this code just split
the part of the tag.
i want to convert into text format so that i can load them into tables of
hbase and hive.
I have used the DOM Parser but this parser uses File as Object but hdfs
uses FileSystem.
Eg,
File fXmlFile = new File("D:/elango/test.xml");
System.out.println(g);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(fXmlFile);
This cant be used as hdfs, because hdfs path is accessed through
FileSystem.
I kindly request u to , Please suggest me to fix the above issue.
Thanks in advance
Ranjini R
On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
<ra...@gmail.com>wrote:
>
>
> ---------- Forwarded message ----------
> From: Shekhar Sharma <sh...@gmail.com>
> Date: Mon, Dec 9, 2013 at 10:23 PM
> Subject: Re: Hadoop-MapReduce
> To: user@hadoop.apache.org
> Cc: ssanyal@datameer.com
>
>
> It does work i have used it long back..
>
> BTW if it is not working, write the custom input format and implement
> your record reader. That would be far more easy than breaking your
> head with others code.
>
> Break your problem in step:
>
> (1) First the XML data is multiline...Meaning multiple lines makes a
> single record for you...May be a record for you would be
>
> <person>
> <fname>x</fname>
> <lname>y</lname>
> </person>
>
> (2) Implement a record reader that looks out for the starting and
> ending person tag ( Checkout how RecordReader.java is written)
>
> (3) Once you got the contents between starting and ending tag, now you
> can use a xml parser to parse the contents into an java object and
> form your own key value pairs ( custom key and custom value)
>
>
> Hope you have enough pointers to write the code.
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
> > Hi Subroto Sanyal,
> >
> > The link provided about xml, it does not work . The Class written
> > XmlContent is not allowed in the XmlInputFormat.
> >
> > I request you to help , whether this scenaio some one has coded, and
> needed
> > working code.
> >
> > I have written using SAX Parser too, but eventhough the jars are added in
> > classpath THe error is is coming has NoClasFoung Exception.
> >
> > Please provide sample code for the same.
> >
> > Thanks in advance,
> > Ranjini.R
> >
> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >>
> >>>> Hi,
> >>>>
> >>>> As suggest by the link below , i have used for my program ,
> >>>>
> >>>> but i am facing the below issues, please help me to fix these error.
> >>>>
> >>>>
> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> >>>> abstract method
> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>>> in org.apache.hadoop.mapred.Mapper
> >>>> public static class Map extends MapReduceBase implements Mapper
> >>>> <LongWritable, Text, Text, Text> {
> >>>> ^
> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> abstract
> >>>> and does not override abstract method
> >>>> next(java.lang.Object,java.lang.Object) in
> >>>> org.apache.hadoop.mapred.RecordReader
> >>>> public class XmlRecordReader implements RecordReader {
> >>>> ^
> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>>> 2 errors
> >>>>
> >>>>
> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>>>
> >>>> Please suggest.
> >>>>
> >>>> Thanks in advance.
> >>>>
> >>>> Regrads,
> >>>> Ranjini. R
> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>>> <ra...@gmail.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> ---------- Forwarded message ----------
> >>>>> From: Subroto <ss...@datameer.com>
> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>>>> Subject: Re: Hadoop-MapReduce
> >>>>> To: user@hadoop.apache.org
> >>>>>
> >>>>>
> >>>>> Hi Ranjini,
> >>>>>
> >>>>> A good example to look into :
> >>>>> http://www.undercloud.org/?p=408
> >>>>>
> >>>>> Cheers,
> >>>>> Subroto Sanyal
> >>>>>
> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> How to read xml file via mapreduce and load them in hbase and hive
> >>>>> using java.
> >>>>>
> >>>>> Please provide sample code.
> >>>>>
> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>>>> should be used.
> >>>>>
> >>>>> Thanks in advance.
> >>>>>
> >>>>> Ranjini
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
hi,
I have fixed the error , the code is running fine, but this code just split
the part of the tag.
i want to convert into text format so that i can load them into tables of
hbase and hive.
I have used the DOM Parser but this parser uses File as Object but hdfs
uses FileSystem.
Eg,
File fXmlFile = new File("D:/elango/test.xml");
System.out.println(g);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(fXmlFile);
This cant be used as hdfs, because hdfs path is accessed through
FileSystem.
I kindly request u to , Please suggest me to fix the above issue.
Thanks in advance
Ranjini R
On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
<ra...@gmail.com>wrote:
>
>
> ---------- Forwarded message ----------
> From: Shekhar Sharma <sh...@gmail.com>
> Date: Mon, Dec 9, 2013 at 10:23 PM
> Subject: Re: Hadoop-MapReduce
> To: user@hadoop.apache.org
> Cc: ssanyal@datameer.com
>
>
> It does work i have used it long back..
>
> BTW if it is not working, write the custom input format and implement
> your record reader. That would be far more easy than breaking your
> head with others code.
>
> Break your problem in step:
>
> (1) First the XML data is multiline...Meaning multiple lines makes a
> single record for you...May be a record for you would be
>
> <person>
> <fname>x</fname>
> <lname>y</lname>
> </person>
>
> (2) Implement a record reader that looks out for the starting and
> ending person tag ( Checkout how RecordReader.java is written)
>
> (3) Once you got the contents between starting and ending tag, now you
> can use a xml parser to parse the contents into an java object and
> form your own key value pairs ( custom key and custom value)
>
>
> Hope you have enough pointers to write the code.
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
> > Hi Subroto Sanyal,
> >
> > The link provided about xml, it does not work . The Class written
> > XmlContent is not allowed in the XmlInputFormat.
> >
> > I request you to help , whether this scenaio some one has coded, and
> needed
> > working code.
> >
> > I have written using SAX Parser too, but eventhough the jars are added in
> > classpath THe error is is coming has NoClasFoung Exception.
> >
> > Please provide sample code for the same.
> >
> > Thanks in advance,
> > Ranjini.R
> >
> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >>
> >>>> Hi,
> >>>>
> >>>> As suggest by the link below , i have used for my program ,
> >>>>
> >>>> but i am facing the below issues, please help me to fix these error.
> >>>>
> >>>>
> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> >>>> abstract method
> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>>> in org.apache.hadoop.mapred.Mapper
> >>>> public static class Map extends MapReduceBase implements Mapper
> >>>> <LongWritable, Text, Text, Text> {
> >>>> ^
> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> abstract
> >>>> and does not override abstract method
> >>>> next(java.lang.Object,java.lang.Object) in
> >>>> org.apache.hadoop.mapred.RecordReader
> >>>> public class XmlRecordReader implements RecordReader {
> >>>> ^
> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>>> 2 errors
> >>>>
> >>>>
> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>>>
> >>>> Please suggest.
> >>>>
> >>>> Thanks in advance.
> >>>>
> >>>> Regrads,
> >>>> Ranjini. R
> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>>> <ra...@gmail.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> ---------- Forwarded message ----------
> >>>>> From: Subroto <ss...@datameer.com>
> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>>>> Subject: Re: Hadoop-MapReduce
> >>>>> To: user@hadoop.apache.org
> >>>>>
> >>>>>
> >>>>> Hi Ranjini,
> >>>>>
> >>>>> A good example to look into :
> >>>>> http://www.undercloud.org/?p=408
> >>>>>
> >>>>> Cheers,
> >>>>> Subroto Sanyal
> >>>>>
> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> How to read xml file via mapreduce and load them in hbase and hive
> >>>>> using java.
> >>>>>
> >>>>> Please provide sample code.
> >>>>>
> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>>>> should be used.
> >>>>>
> >>>>> Thanks in advance.
> >>>>>
> >>>>> Ranjini
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
hi,
I have fixed the error , the code is running fine, but this code just split
the part of the tag.
i want to convert into text format so that i can load them into tables of
hbase and hive.
I have used the DOM Parser but this parser uses File as Object but hdfs
uses FileSystem.
Eg,
File fXmlFile = new File("D:/elango/test.xml");
System.out.println(g);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(fXmlFile);
This cant be used as hdfs, because hdfs path is accessed through
FileSystem.
I kindly request u to , Please suggest me to fix the above issue.
Thanks in advance
Ranjini R
On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
<ra...@gmail.com>wrote:
>
>
> ---------- Forwarded message ----------
> From: Shekhar Sharma <sh...@gmail.com>
> Date: Mon, Dec 9, 2013 at 10:23 PM
> Subject: Re: Hadoop-MapReduce
> To: user@hadoop.apache.org
> Cc: ssanyal@datameer.com
>
>
> It does work i have used it long back..
>
> BTW if it is not working, write the custom input format and implement
> your record reader. That would be far more easy than breaking your
> head with others code.
>
> Break your problem in step:
>
> (1) First the XML data is multiline...Meaning multiple lines makes a
> single record for you...May be a record for you would be
>
> <person>
> <fname>x</fname>
> <lname>y</lname>
> </person>
>
> (2) Implement a record reader that looks out for the starting and
> ending person tag ( Checkout how RecordReader.java is written)
>
> (3) Once you got the contents between starting and ending tag, now you
> can use a xml parser to parse the contents into an java object and
> form your own key value pairs ( custom key and custom value)
>
>
> Hope you have enough pointers to write the code.
>
>
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
> > Hi Subroto Sanyal,
> >
> > The link provided about xml, it does not work . The Class written
> > XmlContent is not allowed in the XmlInputFormat.
> >
> > I request you to help , whether this scenaio some one has coded, and
> needed
> > working code.
> >
> > I have written using SAX Parser too, but eventhough the jars are added in
> > classpath THe error is is coming has NoClasFoung Exception.
> >
> > Please provide sample code for the same.
> >
> > Thanks in advance,
> > Ranjini.R
> >
> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <
> ranjinibecse@gmail.com>
> > wrote:
> >>
> >>
> >>>> Hi,
> >>>>
> >>>> As suggest by the link below , i have used for my program ,
> >>>>
> >>>> but i am facing the below issues, please help me to fix these error.
> >>>>
> >>>>
> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> >>>> abstract method
> >>>>
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> >>>> in org.apache.hadoop.mapred.Mapper
> >>>> public static class Map extends MapReduceBase implements Mapper
> >>>> <LongWritable, Text, Text, Text> {
> >>>> ^
> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not
> abstract
> >>>> and does not override abstract method
> >>>> next(java.lang.Object,java.lang.Object) in
> >>>> org.apache.hadoop.mapred.RecordReader
> >>>> public class XmlRecordReader implements RecordReader {
> >>>> ^
> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
> >>>> Note: Recompile with -Xlint:unchecked for details.
> >>>> 2 errors
> >>>>
> >>>>
> >>>> i am using hadoop 0.20 version and java 1.6 .
> >>>>
> >>>> Please suggest.
> >>>>
> >>>> Thanks in advance.
> >>>>
> >>>> Regrads,
> >>>> Ranjini. R
> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
> >>>> <ra...@gmail.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> ---------- Forwarded message ----------
> >>>>> From: Subroto <ss...@datameer.com>
> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
> >>>>> Subject: Re: Hadoop-MapReduce
> >>>>> To: user@hadoop.apache.org
> >>>>>
> >>>>>
> >>>>> Hi Ranjini,
> >>>>>
> >>>>> A good example to look into :
> >>>>> http://www.undercloud.org/?p=408
> >>>>>
> >>>>> Cheers,
> >>>>> Subroto Sanyal
> >>>>>
> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> How to read xml file via mapreduce and load them in hbase and hive
> >>>>> using java.
> >>>>>
> >>>>> Please provide sample code.
> >>>>>
> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
> >>>>> should be used.
> >>>>>
> >>>>> Thanks in advance.
> >>>>>
> >>>>> Ranjini
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
It does work i have used it long back..
BTW if it is not working, write the custom input format and implement
your record reader. That would be far more easy than breaking your
head with others code.
Break your problem in step:
(1) First the XML data is multiline...Meaning multiple lines makes a
single record for you...May be a record for you would be
<person>
<fname>x</fname>
<lname>y</lname>
</person>
(2) Implement a record reader that looks out for the starting and
ending person tag ( Checkout how RecordReader.java is written)
(3) Once you got the contents between starting and ending tag, now you
can use a xml parser to parse the contents into an java object and
form your own key value pairs ( custom key and custom value)
Hope you have enough pointers to write the code.
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
It does work i have used it long back..
BTW if it is not working, write the custom input format and implement
your record reader. That would be far more easy than breaking your
head with others code.
Break your problem in step:
(1) First the XML data is multiline...Meaning multiple lines makes a
single record for you...May be a record for you would be
<person>
<fname>x</fname>
<lname>y</lname>
</person>
(2) Implement a record reader that looks out for the starting and
ending person tag ( Checkout how RecordReader.java is written)
(3) Once you got the contents between starting and ending tag, now you
can use a xml parser to parse the contents into an java object and
form your own key value pairs ( custom key and custom value)
Hope you have enough pointers to write the code.
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
It does work i have used it long back..
BTW if it is not working, write the custom input format and implement
your record reader. That would be far more easy than breaking your
head with others code.
Break your problem in step:
(1) First the XML data is multiline...Meaning multiple lines makes a
single record for you...May be a record for you would be
<person>
<fname>x</fname>
<lname>y</lname>
</person>
(2) Implement a record reader that looks out for the starting and
ending person tag ( Checkout how RecordReader.java is written)
(3) Once you got the contents between starting and ending tag, now you
can use a xml parser to parse the contents into an java object and
form your own key value pairs ( custom key and custom value)
Hope you have enough pointers to write the code.
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
First Option: Put the jar in $HADOOP_HOME/lib folder
And then run "hadoop classpath" command on your terminal to check
whether the jar has been added
Second OPtion: PUt the jar path in HADOOP_CLASSPATH variable (
hadoop-env.sh file) and restart your cluster..
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
First Option: Put the jar in $HADOOP_HOME/lib folder
And then run "hadoop classpath" command on your terminal to check
whether the jar has been added
Second OPtion: PUt the jar path in HADOOP_CLASSPATH variable (
hadoop-env.sh file) and restart your cluster..
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Shekhar Sharma <sh...@gmail.com>.
It does work i have used it long back..
BTW if it is not working, write the custom input format and implement
your record reader. That would be far more easy than breaking your
head with others code.
Break your problem in step:
(1) First the XML data is multiline...Meaning multiple lines makes a
single record for you...May be a record for you would be
<person>
<fname>x</fname>
<lname>y</lname>
</person>
(2) Implement a record reader that looks out for the starting and
ending person tag ( Checkout how RecordReader.java is written)
(3) Once you got the contents between starting and ending tag, now you
can use a xml parser to parse the contents into an java object and
form your own key value pairs ( custom key and custom value)
Hope you have enough pointers to write the code.
Regards,
Som Shekhar Sharma
+91-8197243810
On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <ra...@gmail.com> wrote:
> Hi Subroto Sanyal,
>
> The link provided about xml, it does not work . The Class written
> XmlContent is not allowed in the XmlInputFormat.
>
> I request you to help , whether this scenaio some one has coded, and needed
> working code.
>
> I have written using SAX Parser too, but eventhough the jars are added in
> classpath THe error is is coming has NoClasFoung Exception.
>
> Please provide sample code for the same.
>
> Thanks in advance,
> Ranjini.R
>
> On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>
> wrote:
>>
>>
>>>> Hi,
>>>>
>>>> As suggest by the link below , i have used for my program ,
>>>>
>>>> but i am facing the below issues, please help me to fix these error.
>>>>
>>>>
>>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>>> abstract method
>>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public static class Map extends MapReduceBase implements Mapper
>>>> <LongWritable, Text, Text, Text> {
>>>> ^
>>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>>> and does not override abstract method
>>>> next(java.lang.Object,java.lang.Object) in
>>>> org.apache.hadoop.mapred.RecordReader
>>>> public class XmlRecordReader implements RecordReader {
>>>> ^
>>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>>> Note: Recompile with -Xlint:unchecked for details.
>>>> 2 errors
>>>>
>>>>
>>>> i am using hadoop 0.20 version and java 1.6 .
>>>>
>>>> Please suggest.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Regrads,
>>>> Ranjini. R
>>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>>> <ra...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Subroto <ss...@datameer.com>
>>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>>> Subject: Re: Hadoop-MapReduce
>>>>> To: user@hadoop.apache.org
>>>>>
>>>>>
>>>>> Hi Ranjini,
>>>>>
>>>>> A good example to look into :
>>>>> http://www.undercloud.org/?p=408
>>>>>
>>>>> Cheers,
>>>>> Subroto Sanyal
>>>>>
>>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>>> using java.
>>>>>
>>>>> Please provide sample code.
>>>>>
>>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>>> should be used.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>> Ranjini
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi Subroto Sanyal,
The link provided about xml, it does not work . The Class written
XmlContent is not allowed in the XmlInputFormat.
I request you to help , whether this scenaio some one has coded, and needed
working code.
I have written using SAX Parser too, but eventhough the jars are added in
classpath THe error is is coming has NoClasFoung Exception.
Please provide sample code for the same.
Thanks in advance,
Ranjini.R
On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
> Hi,
>>>
>>> As suggest by the link below , i have used for my program ,
>>>
>>> but i am facing the below issues, please help me to fix these error.
>>>
>>>
>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>> abstract method
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> in org.apache.hadoop.mapred.Mapper
>>> public static class Map extends MapReduceBase implements Mapper
>>> <LongWritable, Text, Text, Text> {
>>> ^
>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>> and does not override abstract method
>>> next(java.lang.Object,java.lang.Object) in
>>> org.apache.hadoop.mapred.RecordReader
>>> public class XmlRecordReader implements RecordReader {
>>> ^
>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> Note: Recompile with -Xlint:unchecked for details.
>>> 2 errors
>>>
>>>
>>> i am using hadoop 0.20 version and java 1.6 .
>>>
>>> Please suggest.
>>>
>>> Thanks in advance.
>>>
>>> Regrads,
>>> Ranjini. R
>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Subroto <ss...@datameer.com>
>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>> Subject: Re: Hadoop-MapReduce
>>>> To: user@hadoop.apache.org
>>>>
>>>>
>>>> Hi Ranjini,
>>>>
>>>> A good example to look into :
>>>> http://www.undercloud.org/?p=408
>>>>
>>>> Cheers,
>>>> Subroto Sanyal
>>>>
>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>
>>>> Hi,
>>>>
>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>> using java.
>>>>
>>>> Please provide sample code.
>>>>
>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>> should be used.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Ranjini
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi Subroto Sanyal,
The link provided about xml, it does not work . The Class written
XmlContent is not allowed in the XmlInputFormat.
I request you to help , whether this scenaio some one has coded, and needed
working code.
I have written using SAX Parser too, but eventhough the jars are added in
classpath THe error is is coming has NoClasFoung Exception.
Please provide sample code for the same.
Thanks in advance,
Ranjini.R
On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
> Hi,
>>>
>>> As suggest by the link below , i have used for my program ,
>>>
>>> but i am facing the below issues, please help me to fix these error.
>>>
>>>
>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>> abstract method
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> in org.apache.hadoop.mapred.Mapper
>>> public static class Map extends MapReduceBase implements Mapper
>>> <LongWritable, Text, Text, Text> {
>>> ^
>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>> and does not override abstract method
>>> next(java.lang.Object,java.lang.Object) in
>>> org.apache.hadoop.mapred.RecordReader
>>> public class XmlRecordReader implements RecordReader {
>>> ^
>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> Note: Recompile with -Xlint:unchecked for details.
>>> 2 errors
>>>
>>>
>>> i am using hadoop 0.20 version and java 1.6 .
>>>
>>> Please suggest.
>>>
>>> Thanks in advance.
>>>
>>> Regrads,
>>> Ranjini. R
>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Subroto <ss...@datameer.com>
>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>> Subject: Re: Hadoop-MapReduce
>>>> To: user@hadoop.apache.org
>>>>
>>>>
>>>> Hi Ranjini,
>>>>
>>>> A good example to look into :
>>>> http://www.undercloud.org/?p=408
>>>>
>>>> Cheers,
>>>> Subroto Sanyal
>>>>
>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>
>>>> Hi,
>>>>
>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>> using java.
>>>>
>>>> Please provide sample code.
>>>>
>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>> should be used.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Ranjini
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi Subroto Sanyal,
The link provided about xml, it does not work . The Class written
XmlContent is not allowed in the XmlInputFormat.
I request you to help , whether this scenaio some one has coded, and needed
working code.
I have written using SAX Parser too, but eventhough the jars are added in
classpath THe error is is coming has NoClasFoung Exception.
Please provide sample code for the same.
Thanks in advance,
Ranjini.R
On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
> Hi,
>>>
>>> As suggest by the link below , i have used for my program ,
>>>
>>> but i am facing the below issues, please help me to fix these error.
>>>
>>>
>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>> abstract method
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> in org.apache.hadoop.mapred.Mapper
>>> public static class Map extends MapReduceBase implements Mapper
>>> <LongWritable, Text, Text, Text> {
>>> ^
>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>> and does not override abstract method
>>> next(java.lang.Object,java.lang.Object) in
>>> org.apache.hadoop.mapred.RecordReader
>>> public class XmlRecordReader implements RecordReader {
>>> ^
>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> Note: Recompile with -Xlint:unchecked for details.
>>> 2 errors
>>>
>>>
>>> i am using hadoop 0.20 version and java 1.6 .
>>>
>>> Please suggest.
>>>
>>> Thanks in advance.
>>>
>>> Regrads,
>>> Ranjini. R
>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Subroto <ss...@datameer.com>
>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>> Subject: Re: Hadoop-MapReduce
>>>> To: user@hadoop.apache.org
>>>>
>>>>
>>>> Hi Ranjini,
>>>>
>>>> A good example to look into :
>>>> http://www.undercloud.org/?p=408
>>>>
>>>> Cheers,
>>>> Subroto Sanyal
>>>>
>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>
>>>> Hi,
>>>>
>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>> using java.
>>>>
>>>> Please provide sample code.
>>>>
>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>> should be used.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Ranjini
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
Hi Subroto Sanyal,
The link provided about xml, it does not work . The Class written
XmlContent is not allowed in the XmlInputFormat.
I request you to help , whether this scenaio some one has coded, and needed
working code.
I have written using SAX Parser too, but eventhough the jars are added in
classpath THe error is is coming has NoClasFoung Exception.
Please provide sample code for the same.
Thanks in advance,
Ranjini.R
On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam <ra...@gmail.com>wrote:
>
> Hi,
>>>
>>> As suggest by the link below , i have used for my program ,
>>>
>>> but i am facing the below issues, please help me to fix these error.
>>>
>>>
>>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>>> abstract method
>>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> in org.apache.hadoop.mapred.Mapper
>>> public static class Map extends MapReduceBase implements Mapper
>>> <LongWritable, Text, Text, Text> {
>>> ^
>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>>> and does not override abstract method
>>> next(java.lang.Object,java.lang.Object) in
>>> org.apache.hadoop.mapred.RecordReader
>>> public class XmlRecordReader implements RecordReader {
>>> ^
>>> Note: XmlReader.java uses unchecked or unsafe operations.
>>> Note: Recompile with -Xlint:unchecked for details.
>>> 2 errors
>>>
>>>
>>> i am using hadoop 0.20 version and java 1.6 .
>>>
>>> Please suggest.
>>>
>>> Thanks in advance.
>>>
>>> Regrads,
>>> Ranjini. R
>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>>> ranjinibecse@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Subroto <ss...@datameer.com>
>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>>> Subject: Re: Hadoop-MapReduce
>>>> To: user@hadoop.apache.org
>>>>
>>>>
>>>> Hi Ranjini,
>>>>
>>>> A good example to look into :
>>>> http://www.undercloud.org/?p=408
>>>>
>>>> Cheers,
>>>> Subroto Sanyal
>>>>
>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>>
>>>> Hi,
>>>>
>>>> How to read xml file via mapreduce and load them in hbase and hive
>>>> using java.
>>>>
>>>> Please provide sample code.
>>>>
>>>> I am using hadoop 0.20 version and java 1.6. Which parser version
>>>> should be used.
>>>>
>>>> Thanks in advance.
>>>>
>>>> Ranjini
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
> Hi,
>>
>> As suggest by the link below , i have used for my program ,
>>
>> but i am facing the below issues, please help me to fix these error.
>>
>>
>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> abstract method
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> in org.apache.hadoop.mapred.Mapper
>> public static class Map extends MapReduceBase implements Mapper
>> <LongWritable, Text, Text, Text> {
>> ^
>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>> and does not override abstract method
>> next(java.lang.Object,java.lang.Object) in
>> org.apache.hadoop.mapred.RecordReader
>> public class XmlRecordReader implements RecordReader {
>> ^
>> Note: XmlReader.java uses unchecked or unsafe operations.
>> Note: Recompile with -Xlint:unchecked for details.
>> 2 errors
>>
>>
>> i am using hadoop 0.20 version and java 1.6 .
>>
>> Please suggest.
>>
>> Thanks in advance.
>>
>> Regrads,
>> Ranjini. R
>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>> ranjinibecse@gmail.com> wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Subroto <ss...@datameer.com>
>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>>
>>>
>>> Hi Ranjini,
>>>
>>> A good example to look into :
>>> http://www.undercloud.org/?p=408
>>>
>>> Cheers,
>>> Subroto Sanyal
>>>
>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>
>>> Hi,
>>>
>>> How to read xml file via mapreduce and load them in hbase and hive using
>>> java.
>>>
>>> Please provide sample code.
>>>
>>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>>> be used.
>>>
>>> Thanks in advance.
>>>
>>> Ranjini
>>>
>>>
>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
> Hi,
>>
>> As suggest by the link below , i have used for my program ,
>>
>> but i am facing the below issues, please help me to fix these error.
>>
>>
>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> abstract method
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> in org.apache.hadoop.mapred.Mapper
>> public static class Map extends MapReduceBase implements Mapper
>> <LongWritable, Text, Text, Text> {
>> ^
>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>> and does not override abstract method
>> next(java.lang.Object,java.lang.Object) in
>> org.apache.hadoop.mapred.RecordReader
>> public class XmlRecordReader implements RecordReader {
>> ^
>> Note: XmlReader.java uses unchecked or unsafe operations.
>> Note: Recompile with -Xlint:unchecked for details.
>> 2 errors
>>
>>
>> i am using hadoop 0.20 version and java 1.6 .
>>
>> Please suggest.
>>
>> Thanks in advance.
>>
>> Regrads,
>> Ranjini. R
>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>> ranjinibecse@gmail.com> wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Subroto <ss...@datameer.com>
>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>>
>>>
>>> Hi Ranjini,
>>>
>>> A good example to look into :
>>> http://www.undercloud.org/?p=408
>>>
>>> Cheers,
>>> Subroto Sanyal
>>>
>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>
>>> Hi,
>>>
>>> How to read xml file via mapreduce and load them in hbase and hive using
>>> java.
>>>
>>> Please provide sample code.
>>>
>>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>>> be used.
>>>
>>> Thanks in advance.
>>>
>>> Ranjini
>>>
>>>
>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
> Hi,
>>
>> As suggest by the link below , i have used for my program ,
>>
>> but i am facing the below issues, please help me to fix these error.
>>
>>
>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> abstract method
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> in org.apache.hadoop.mapred.Mapper
>> public static class Map extends MapReduceBase implements Mapper
>> <LongWritable, Text, Text, Text> {
>> ^
>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>> and does not override abstract method
>> next(java.lang.Object,java.lang.Object) in
>> org.apache.hadoop.mapred.RecordReader
>> public class XmlRecordReader implements RecordReader {
>> ^
>> Note: XmlReader.java uses unchecked or unsafe operations.
>> Note: Recompile with -Xlint:unchecked for details.
>> 2 errors
>>
>>
>> i am using hadoop 0.20 version and java 1.6 .
>>
>> Please suggest.
>>
>> Thanks in advance.
>>
>> Regrads,
>> Ranjini. R
>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>> ranjinibecse@gmail.com> wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Subroto <ss...@datameer.com>
>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>>
>>>
>>> Hi Ranjini,
>>>
>>> A good example to look into :
>>> http://www.undercloud.org/?p=408
>>>
>>> Cheers,
>>> Subroto Sanyal
>>>
>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>
>>> Hi,
>>>
>>> How to read xml file via mapreduce and load them in hbase and hive using
>>> java.
>>>
>>> Please provide sample code.
>>>
>>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>>> be used.
>>>
>>> Thanks in advance.
>>>
>>> Ranjini
>>>
>>>
>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
> Hi,
>>
>> As suggest by the link below , i have used for my program ,
>>
>> but i am facing the below issues, please help me to fix these error.
>>
>>
>> XmlReader.java:8: XmlReader.Map is not abstract and does not override
>> abstract method
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> in org.apache.hadoop.mapred.Mapper
>> public static class Map extends MapReduceBase implements Mapper
>> <LongWritable, Text, Text, Text> {
>> ^
>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
>> and does not override abstract method
>> next(java.lang.Object,java.lang.Object) in
>> org.apache.hadoop.mapred.RecordReader
>> public class XmlRecordReader implements RecordReader {
>> ^
>> Note: XmlReader.java uses unchecked or unsafe operations.
>> Note: Recompile with -Xlint:unchecked for details.
>> 2 errors
>>
>>
>> i am using hadoop 0.20 version and java 1.6 .
>>
>> Please suggest.
>>
>> Thanks in advance.
>>
>> Regrads,
>> Ranjini. R
>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <
>> ranjinibecse@gmail.com> wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Subroto <ss...@datameer.com>
>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org
>>>
>>>
>>> Hi Ranjini,
>>>
>>> A good example to look into :
>>> http://www.undercloud.org/?p=408
>>>
>>> Cheers,
>>> Subroto Sanyal
>>>
>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>>
>>> Hi,
>>>
>>> How to read xml file via mapreduce and load them in hbase and hive using
>>> java.
>>>
>>> Please provide sample code.
>>>
>>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>>> be used.
>>>
>>> Thanks in advance.
>>>
>>> Ranjini
>>>
>>>
>>>
>>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
>
> Hi,
>
> As suggest by the link below , i have used for my program ,
>
> but i am facing the below issues, please help me to fix these error.
>
>
> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> abstract method
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> in org.apache.hadoop.mapred.Mapper
> public static class Map extends MapReduceBase implements Mapper
> <LongWritable, Text, Text, Text> {
> ^
> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
> and does not override abstract method
> next(java.lang.Object,java.lang.Object) in
> org.apache.hadoop.mapred.RecordReader
> public class XmlRecordReader implements RecordReader {
> ^
> Note: XmlReader.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 2 errors
>
>
> i am using hadoop 0.20 version and java 1.6 .
>
> Please suggest.
>
> Thanks in advance.
>
> Regrads,
> Ranjini. R
> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Subroto <ss...@datameer.com>
>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>>
>>
>> Hi Ranjini,
>>
>> A good example to look into :
>> http://www.undercloud.org/?p=408
>>
>> Cheers,
>> Subroto Sanyal
>>
>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>
>> Hi,
>>
>> How to read xml file via mapreduce and load them in hbase and hive using
>> java.
>>
>> Please provide sample code.
>>
>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>> be used.
>>
>> Thanks in advance.
>>
>> Ranjini
>>
>>
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
>
> Hi,
>
> As suggest by the link below , i have used for my program ,
>
> but i am facing the below issues, please help me to fix these error.
>
>
> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> abstract method
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> in org.apache.hadoop.mapred.Mapper
> public static class Map extends MapReduceBase implements Mapper
> <LongWritable, Text, Text, Text> {
> ^
> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
> and does not override abstract method
> next(java.lang.Object,java.lang.Object) in
> org.apache.hadoop.mapred.RecordReader
> public class XmlRecordReader implements RecordReader {
> ^
> Note: XmlReader.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 2 errors
>
>
> i am using hadoop 0.20 version and java 1.6 .
>
> Please suggest.
>
> Thanks in advance.
>
> Regrads,
> Ranjini. R
> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Subroto <ss...@datameer.com>
>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>>
>>
>> Hi Ranjini,
>>
>> A good example to look into :
>> http://www.undercloud.org/?p=408
>>
>> Cheers,
>> Subroto Sanyal
>>
>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>
>> Hi,
>>
>> How to read xml file via mapreduce and load them in hbase and hive using
>> java.
>>
>> Please provide sample code.
>>
>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>> be used.
>>
>> Thanks in advance.
>>
>> Ranjini
>>
>>
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
>
> Hi,
>
> As suggest by the link below , i have used for my program ,
>
> but i am facing the below issues, please help me to fix these error.
>
>
> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> abstract method
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> in org.apache.hadoop.mapred.Mapper
> public static class Map extends MapReduceBase implements Mapper
> <LongWritable, Text, Text, Text> {
> ^
> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
> and does not override abstract method
> next(java.lang.Object,java.lang.Object) in
> org.apache.hadoop.mapred.RecordReader
> public class XmlRecordReader implements RecordReader {
> ^
> Note: XmlReader.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 2 errors
>
>
> i am using hadoop 0.20 version and java 1.6 .
>
> Please suggest.
>
> Thanks in advance.
>
> Regrads,
> Ranjini. R
> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Subroto <ss...@datameer.com>
>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>>
>>
>> Hi Ranjini,
>>
>> A good example to look into :
>> http://www.undercloud.org/?p=408
>>
>> Cheers,
>> Subroto Sanyal
>>
>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>
>> Hi,
>>
>> How to read xml file via mapreduce and load them in hbase and hive using
>> java.
>>
>> Please provide sample code.
>>
>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>> be used.
>>
>> Thanks in advance.
>>
>> Ranjini
>>
>>
>>
>>
>
Re: Hadoop-MapReduce
Posted by Ranjini Rathinam <ra...@gmail.com>.
>
> Hi,
>
> As suggest by the link below , i have used for my program ,
>
> but i am facing the below issues, please help me to fix these error.
>
>
> XmlReader.java:8: XmlReader.Map is not abstract and does not override
> abstract method
> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
> in org.apache.hadoop.mapred.Mapper
> public static class Map extends MapReduceBase implements Mapper
> <LongWritable, Text, Text, Text> {
> ^
> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not abstract
> and does not override abstract method
> next(java.lang.Object,java.lang.Object) in
> org.apache.hadoop.mapred.RecordReader
> public class XmlRecordReader implements RecordReader {
> ^
> Note: XmlReader.java uses unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 2 errors
>
>
> i am using hadoop 0.20 version and java 1.6 .
>
> Please suggest.
>
> Thanks in advance.
>
> Regrads,
> Ranjini. R
> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam <ranjinibecse@gmail.com
> > wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Subroto <ss...@datameer.com>
>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> Subject: Re: Hadoop-MapReduce
>> To: user@hadoop.apache.org
>>
>>
>> Hi Ranjini,
>>
>> A good example to look into :
>> http://www.undercloud.org/?p=408
>>
>> Cheers,
>> Subroto Sanyal
>>
>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>>
>> Hi,
>>
>> How to read xml file via mapreduce and load them in hbase and hive using
>> java.
>>
>> Please provide sample code.
>>
>> I am using hadoop 0.20 version and java 1.6. Which parser version should
>> be used.
>>
>> Thanks in advance.
>>
>> Ranjini
>>
>>
>>
>>
>
Re: Hadoop-MapReduce
Posted by Subroto <ss...@datameer.com>.
Hi Ranjini,
A good example to look into :
http://www.undercloud.org/?p=408
Cheers,
Subroto Sanyal
On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> Hi,
>
> How to read xml file via mapreduce and load them in hbase and hive using java.
>
> Please provide sample code.
>
> I am using hadoop 0.20 version and java 1.6. Which parser version should be used.
>
> Thanks in advance.
>
> Ranjini
Re: Hadoop-MapReduce
Posted by Subroto <ss...@datameer.com>.
Hi Ranjini,
A good example to look into :
http://www.undercloud.org/?p=408
Cheers,
Subroto Sanyal
On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> Hi,
>
> How to read xml file via mapreduce and load them in hbase and hive using java.
>
> Please provide sample code.
>
> I am using hadoop 0.20 version and java 1.6. Which parser version should be used.
>
> Thanks in advance.
>
> Ranjini
Re: Hadoop-MapReduce
Posted by Subroto <ss...@datameer.com>.
Hi Ranjini,
A good example to look into :
http://www.undercloud.org/?p=408
Cheers,
Subroto Sanyal
On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> Hi,
>
> How to read xml file via mapreduce and load them in hbase and hive using java.
>
> Please provide sample code.
>
> I am using hadoop 0.20 version and java 1.6. Which parser version should be used.
>
> Thanks in advance.
>
> Ranjini
Re: Hadoop-MapReduce
Posted by Subroto <ss...@datameer.com>.
Hi Ranjini,
A good example to look into :
http://www.undercloud.org/?p=408
Cheers,
Subroto Sanyal
On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
> Hi,
>
> How to read xml file via mapreduce and load them in hbase and hive using java.
>
> Please provide sample code.
>
> I am using hadoop 0.20 version and java 1.6. Which parser version should be used.
>
> Thanks in advance.
>
> Ranjini