[GitHub] [druid] gianm commented on issue #6513: Ability to auto populate timestamp on event ingestion.

[GitBox <gi...@apache.org>]

gianm commented on issue #6513:
URL: https://github.com/apache/druid/issues/6513#issuecomment-1029255137


   Unfortunately, there's still this issue with determinism. It's important that two servers reading the same stream will evaluate expressions the same way, which means we can't have expressions like `now()` or `random()` that are evaluated on a per-row basis. As to letting the user make the decision -- imo, we should avoid features like that, because even though they can be useful, they are foot-guns that make it easy to accidentally mess up your state.
   
   We did recently add #11630, which makes it possible to use the Kafka-assigned timestamp when reading from Kafka streams. I think that would help in a lot of these cases -- not all -- but hopefully a good amount. It's also totally safe, which is a plus!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org