You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by IvyTang <iv...@gmail.com> on 2012/03/15 04:33:14 UTC
The collector data sink file path
the chukwaCollector.localOutputDir in chukwa-collector-conf.xml is set to
be /tmp/chukwa/dataSink/.
when i want to see the sink file , i found there is no such directory.
but the log is being transfered to hadoop.
Where the data sink file?
--
Best regards,
Ivy Tang
Re: The collector data sink file path
Posted by Eric Yang <er...@gmail.com>.
Hi Ivy,
This is because the configuration is using SeqFileWriter instead of LocalWriter.
You should change your configuration to:
<property>
<name>chukwaCollector.pipeline</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.LocalWriter</value>
</property>
to use write data locally. By using SeqFileWriter, the data is
written to hdfs://hostname:port/chukwa/logs.
regards,
Eric
On Wed, Mar 14, 2012 at 9:05 PM, IvyTang <iv...@gmail.com> wrote:
> Sorry , got it wrong.
>
> My question is:
>
>
> The the wiki says
>
> Configuring and starting the Collector
>
> Edit etc/chukwa/chukwa-collector-conf.xml and comment out the default
> properties for chukwaCollector.writerClass, and chukwaCollector.pipeline.
> Uncomment block for HBaseWriter parameters, and save.
>
> But my conf is
> <property>
> <name>chukwaCollector.writerClass</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
> </property>
>
> <property>
> <name>chukwaCollector.pipeline</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
> </property>
>
> however , the hadoop indeed received the log .
>
> 2. The chukwaCollector.localOutputDir and chukwaCollector.writerClass has
> been comment.
> So where is the local data sink file ?
>
>
> On Thu, Mar 15, 2012 at 11:33 AM, IvyTang <iv...@gmail.com> wrote:
>>
>> the chukwaCollector.localOutputDir in chukwa-collector-conf.xml is set to
>> be /tmp/chukwa/dataSink/.
>>
>> when i want to see the sink file , i found there is no such directory.
>>
>> but the log is being transfered to hadoop.
>>
>> Where the data sink file?
>>
>> --
>> Best regards,
>>
>> Ivy Tang
>>
>>
>>
>
>
>
> --
> Best regards,
>
> Ivy Tang
>
>
>
Re: The collector data sink file path
Posted by IvyTang <iv...@gmail.com>.
Sorry , got it wrong.
My question is:
The the wiki says
Configuring and starting the Collector
1. Edit etc/chukwa/chukwa-collector-conf.xml and comment out the default
properties for chukwaCollector.writerClass, and chukwaCollector.pipeline.
Uncomment block for HBaseWriter parameters, and save.
But my conf is
<property>
<name>chukwaCollector.writerClass</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
</property>
<property>
<name>chukwaCollector.pipeline</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
</property>
however , the hadoop indeed received the log .
2. The chukwaCollector.localOutputDir and chukwaCollector.writerClass has
been comment.
So where is the local data sink file ?
On Thu, Mar 15, 2012 at 11:33 AM, IvyTang <iv...@gmail.com> wrote:
> the chukwaCollector.localOutputDir in chukwa-collector-conf.xml is set to
> be /tmp/chukwa/dataSink/.
>
> when i want to see the sink file , i found there is no such directory.
>
> but the log is being transfered to hadoop.
>
> Where the data sink file?
>
> --
> Best regards,
>
> Ivy Tang
>
>
>
>
--
Best regards,
Ivy Tang