You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by Mohammad Tariq <do...@gmail.com> on 2011/12/18 15:35:01 UTC
Data collection using 0.5.0
Hello list,
Can I use the steps, I had used to collect data to the HDFS, for
Hbase as well??Or are there any changes now?? It's been quite sometime
since I had configured chukwa-0.5.0, but I couldn't proceed further
due to some tight schedule and now I want to go ahead.Many thanks.
Regards,
Mohammad Tariq
Re: Data collection using 0.5.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hello AD,
I have these properties set, but it seems there is still some
problem as I can't see data moving.I have just posted a mail regarding
this.Please have a look and let me know where I am going wrong.
Regards,
Mohammad Tariq
On Mon, Dec 19, 2011 at 4:03 AM, AD <st...@gmail.com> wrote:
> You *should* be able to use the pipeline writer
>
> <property>
> <name>chukwaCollector.writerClass</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
> </property>
>
> and then chain your writers
>
> <property>
> <name>chukwaCollector.pipeline</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.hbase.HbaseWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value> </property>
>
> On Sun, Dec 18, 2011 at 3:13 PM, Mohammad Tariq <do...@gmail.com> wrote:
>>
>> Hi Eric,
>>
>> Thanks again for your reply.. I was just going through the updated
>> docs..I have a small question for you, is it possible to collect data
>> to HDFS and Hbase simultaneously??Thanks once again.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>>
>> On Mon, Dec 19, 2011 at 1:37 AM, Eric Yang <er...@gmail.com> wrote:
>> > For collecting data to HBase, you will need to make sure that you have
>> > a demux parser
>> > that is compatible with HBase. TSProcessor doe not work with HBase
>> > because it does
>> > not have a targeted HBase table annotation embedded in TSProcessor.
>> >
>> > See the programming guide to stream your data to HBase:
>> >
>> > http://people.apache.org/~eyang/chukwa-0.5.0-docs/programming.html
>> >
>> > For enabling HBase as the data sink, see the configuration guide for
>> > collector:
>> >
>> > http://people.apache.org/~eyang/chukwa-0.5.0-docs/collector.html
>> >
>> > Hope this helps.
>> >
>> > regards,
>> > Eric
>> >
>> > On Sun, Dec 18, 2011 at 6:35 AM, Mohammad Tariq <do...@gmail.com>
>> > wrote:
>> >> Hello list,
>> >>
>> >> Can I use the steps, I had used to collect data to the HDFS, for
>> >> Hbase as well??Or are there any changes now?? It's been quite sometime
>> >> since I had configured chukwa-0.5.0, but I couldn't proceed further
>> >> due to some tight schedule and now I want to go ahead.Many thanks.
>> >>
>> >> Regards,
>> >> Mohammad Tariq
>
>
Re: Data collection using 0.5.0
Posted by AD <st...@gmail.com>.
You *should* be able to use the pipeline writer
<property>
<name>chukwaCollector.writerClass</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
</property>
and then chain your writers
<property>
<name>chukwaCollector.pipeline</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.hbase.HbaseWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
</property>
On Sun, Dec 18, 2011 at 3:13 PM, Mohammad Tariq <do...@gmail.com> wrote:
> Hi Eric,
>
> Thanks again for your reply.. I was just going through the updated
> docs..I have a small question for you, is it possible to collect data
> to HDFS and Hbase simultaneously??Thanks once again.
>
> Regards,
> Mohammad Tariq
>
>
>
> On Mon, Dec 19, 2011 at 1:37 AM, Eric Yang <er...@gmail.com> wrote:
> > For collecting data to HBase, you will need to make sure that you have
> > a demux parser
> > that is compatible with HBase. TSProcessor doe not work with HBase
> > because it does
> > not have a targeted HBase table annotation embedded in TSProcessor.
> >
> > See the programming guide to stream your data to HBase:
> >
> > http://people.apache.org/~eyang/chukwa-0.5.0-docs/programming.html
> >
> > For enabling HBase as the data sink, see the configuration guide for
> collector:
> >
> > http://people.apache.org/~eyang/chukwa-0.5.0-docs/collector.html
> >
> > Hope this helps.
> >
> > regards,
> > Eric
> >
> > On Sun, Dec 18, 2011 at 6:35 AM, Mohammad Tariq <do...@gmail.com>
> wrote:
> >> Hello list,
> >>
> >> Can I use the steps, I had used to collect data to the HDFS, for
> >> Hbase as well??Or are there any changes now?? It's been quite sometime
> >> since I had configured chukwa-0.5.0, but I couldn't proceed further
> >> due to some tight schedule and now I want to go ahead.Many thanks.
> >>
> >> Regards,
> >> Mohammad Tariq
>
Re: Data collection using 0.5.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hi Eric,
Thanks again for your reply.. I was just going through the updated
docs..I have a small question for you, is it possible to collect data
to HDFS and Hbase simultaneously??Thanks once again.
Regards,
Mohammad Tariq
On Mon, Dec 19, 2011 at 1:37 AM, Eric Yang <er...@gmail.com> wrote:
> For collecting data to HBase, you will need to make sure that you have
> a demux parser
> that is compatible with HBase. TSProcessor doe not work with HBase
> because it does
> not have a targeted HBase table annotation embedded in TSProcessor.
>
> See the programming guide to stream your data to HBase:
>
> http://people.apache.org/~eyang/chukwa-0.5.0-docs/programming.html
>
> For enabling HBase as the data sink, see the configuration guide for collector:
>
> http://people.apache.org/~eyang/chukwa-0.5.0-docs/collector.html
>
> Hope this helps.
>
> regards,
> Eric
>
> On Sun, Dec 18, 2011 at 6:35 AM, Mohammad Tariq <do...@gmail.com> wrote:
>> Hello list,
>>
>> Can I use the steps, I had used to collect data to the HDFS, for
>> Hbase as well??Or are there any changes now?? It's been quite sometime
>> since I had configured chukwa-0.5.0, but I couldn't proceed further
>> due to some tight schedule and now I want to go ahead.Many thanks.
>>
>> Regards,
>> Mohammad Tariq
Re: Data collection using 0.5.0
Posted by Eric Yang <er...@gmail.com>.
For collecting data to HBase, you will need to make sure that you have
a demux parser
that is compatible with HBase. TSProcessor doe not work with HBase
because it does
not have a targeted HBase table annotation embedded in TSProcessor.
See the programming guide to stream your data to HBase:
http://people.apache.org/~eyang/chukwa-0.5.0-docs/programming.html
For enabling HBase as the data sink, see the configuration guide for collector:
http://people.apache.org/~eyang/chukwa-0.5.0-docs/collector.html
Hope this helps.
regards,
Eric
On Sun, Dec 18, 2011 at 6:35 AM, Mohammad Tariq <do...@gmail.com> wrote:
> Hello list,
>
> Can I use the steps, I had used to collect data to the HDFS, for
> Hbase as well??Or are there any changes now?? It's been quite sometime
> since I had configured chukwa-0.5.0, but I couldn't proceed further
> due to some tight schedule and now I want to go ahead.Many thanks.
>
> Regards,
> Mohammad Tariq