You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by DKN <de...@in.ibm.com> on 2011/06/03 14:21:57 UTC
How to obtain the chunk time stamp while processing the chunk using
a Socket Program
I implemented a simple Socket program to listen to the collector on 9094
port, so that I can process that information further (using the
SocketTeeWriter).
I used this type of a command to tap into my specific data type :
HEADER datatype=SysOut*\n
I was able to get the chunk as RAW data. But it didn't have any information
about the time stamp which would be used while persisting in HDFS/HBase. Is
it possible to send the time stamp information of the chunk in the header ?
{ I am basically wondering about data consistency if I persist this
information else where apart from HDFS .. }
Cheers, DKN
--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-obtain-the-chunk-time-stamp-while-processing-the-chunk-using-a-Socket-Program-tp3019416p3019416.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.
Re: How to obtain the chunk time stamp while processing the chunk
using a Socket Program
Posted by Eric Yang <ey...@yahoo-inc.com>.
In Chukwa Chunk data structure, we have a meta data field called tags, which allows Chukwa agent/adaptor to inject data into this field.
We currently put cluster="" into the tag field in Chukwa Agent. It is possible to add a tag for Chukwa adaptor to send over the timestamp.
Regards,
Eric
On 6/3/11 5:21 AM, "DKN" <de...@in.ibm.com> wrote:
I implemented a simple Socket program to listen to the collector on 9094
port, so that I can process that information further (using the
SocketTeeWriter).
I used this type of a command to tap into my specific data type :
HEADER datatype=SysOut*\n
I was able to get the chunk as RAW data. But it didn't have any information
about the time stamp which would be used while persisting in HDFS/HBase. Is
it possible to send the time stamp information of the chunk in the header ?
{ I am basically wondering about data consistency if I persist this
information else where apart from HDFS .. }
Cheers, DKN
--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-obtain-the-chunk-time-stamp-while-processing-the-chunk-using-a-Socket-Program-tp3019416p3019416.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.