[GitHub] [incubator-pinot] kishoreg commented on issue #7004: Preserver Kafka Message Metadata in Pinot Tables

[GitBox <gi...@apache.org>]

kishoreg commented on issue #7004:
URL: https://github.com/apache/incubator-pinot/issues/7004#issuecomment-884302410


   @mcvsubbu , this is a very powerful feature. This is equivalent to cursor API in databases. For e.g. we can use this in ThirdEye to analyze new events that come in since the last time we ran anomaly detection.
   
   I am +1000 on this feature. Another side benefit of this is in debugging kafka lag. Kafka provides an offset API and we can compare the two to compute the lag.
   
   btw, I am fine with this being an actual column instead of virtual. we can solve this by having a udf pull the value from header.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org