You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@chukwa.apache.org by NehaS Singh <Ne...@lntinfotech.com> on 2012/12/28 11:56:21 UTC

Writing logs collected by chukwa to hbase

Hi,
                I am using Chukwa to collect log4j logs which I can see in HDFS. But I am not able to write it to HBASE. Can anyone help ?

Regards,
Neha Singh


________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"

Re: Writing logs collected by chukwa to hbase

Posted by Eric Yang <er...@gmail.com>.

Hi Neha,

I don't think wild card is supported.

regards,
Eric

On Sun, Jan 6, 2013 at 10:18 PM, NehaS Singh <Ne...@lntinfotech.com>wrote:

>  Hi,****
>
> ** **
>
> Whenever I run the command  $CHUKWA_HOME/bin/chukwa dumpArchive
> 'datatype=.*' 'hdfs://host:8020/chukwa/archive/*.arc' the following output
> is displayed ****
>
> “*total of 0 files to search*”****
>
> ** **
>
> Regards,****
>
> Neha Singh****
>
> ** **
>
> *From:* Eric Yang [mailto:eric818@gmail.com]
> *Sent:* Monday, January 07, 2013 2:48 AM
>
> *To:* chukwa-user@incubator.apache.org
> *Subject:* Re: Writing logs collected by chukwa to hbase****
>
>  ** **
>
> There are a few things that you need to do:****
>
> ** **
>
> 1. Configure the collector to use HBaseWriter.****
>
> ** **
>
>   http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html****
>
> ** **
>
> Make sure that hadoop and hbase configuration are in the classpath of the
> collector.****
>
> ** **
>
> 2. Write a demux parser for your log file to store on HBase, and this runs
> in chukwa collector.****
>
> ** **
>
>   http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html****
>
>
> 3. What log4j adaptors and appenders are you using to write the log files?
> ****
>
>   I would recommend to use socket appender from Log4j and use socket
> adaptor to listen for the incoming data stream.  This will ensure the
> records are delimited properly.****
>
> ** **
>
> Hope this helps.****
>
> ** **
>
> regards,****
>
> Eric****
>
> ** **
>
> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <Ne...@lntinfotech.com>
> wrote:****
>
> I am using chukwa 0.5. My probleam is want to store the logs from log4j or
> other party which is not hadoop.Also i want to  write the logs to hbase.
> But i am not ale to do so. I have configuerd log4j.properties file so i am
> able to see th logs in hdfs.
> what are the configuration or steps using which i can write the logs to
> hbase.
> Please help.
> ________________________________________
> From: Eric Yang [eric818@gmail.com]
> Sent: Saturday, January 05, 2013 11:55 PM
> To: chukwa-user@incubator.apache.org
> Subject: Re: Writing logs collected by chukwa to hbase****
>
>
> What version of Chukwa are you running?
>
> regards,
> Eric****
>
> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <NehaS.Singh@lntinfotech.com
> <ma...@lntinfotech.com>> wrote:
> Hi,
>                 I am using Chukwa to collect log4j logs which I can see in
> HDFS. But I am not able to write it to HBASE. Can anyone help ?
>
> Regards,
> Neha Singh
>
>
> ________________________________
> The contents of this e-mail and any attachment(s) may contain confidential
> or privileged information for the intended recipient(s). Unintended
> recipients are prohibited from taking action on the basis of information in
> this e-mail and using or disseminating the information, and must notify the
> sender and delete it from their system. L&T Infotech will not accept
> responsibility or liability for the accuracy or completeness of, or the
> presence of any virus or disabling code in this e-mail"****
>
> ** **
>