You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by DKN <de...@in.ibm.com> on 2011/06/03 14:21:57 UTC

How to obtain the chunk time stamp while processing the chunk using a Socket Program

I implemented a simple Socket program to listen to the collector on 9094
port, so that I can process that information further (using the
SocketTeeWriter).

I used this type of a command to tap into my specific data type :

HEADER datatype=SysOut*\n

I was able to get the chunk as RAW data. But it didn't have any information
about the time stamp which would be used while persisting in HDFS/HBase. Is
it possible to send the time stamp information of the chunk in the header ?
{ I am basically wondering about data consistency if I persist this
information else where apart from HDFS .. }

Cheers, DKN

--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-obtain-the-chunk-time-stamp-while-processing-the-chunk-using-a-Socket-Program-tp3019416p3019416.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.

Re: How to obtain the chunk time stamp while processing the chunk using a Socket Program

Posted by Eric Yang <ey...@yahoo-inc.com>.
In Chukwa Chunk data structure, we have a meta data field called tags, which allows Chukwa agent/adaptor to inject data into this field.

We currently put cluster="" into the tag field in Chukwa Agent.  It is possible to add a tag for Chukwa adaptor to send over the timestamp.

Regards,
Eric

On 6/3/11 5:21 AM, "DKN" <de...@in.ibm.com> wrote:

I implemented a simple Socket program to listen to the collector on 9094
port, so that I can process that information further (using the
SocketTeeWriter).

I used this type of a command to tap into my specific data type :

HEADER datatype=SysOut*\n

I was able to get the chunk as RAW data. But it didn't have any information
about the time stamp which would be used while persisting in HDFS/HBase. Is
it possible to send the time stamp information of the chunk in the header ?
{ I am basically wondering about data consistency if I persist this
information else where apart from HDFS .. }

Cheers, DKN

--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-obtain-the-chunk-time-stamp-while-processing-the-chunk-using-a-Socket-Program-tp3019416p3019416.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.