[GitHub] [pinot] ybrankov commented on issue #10571: Pinot Dedup requires a lot of memory

["ybrankov (via GitHub)" <gi...@apache.org>]

ybrankov commented on issue #10571:
URL: https://github.com/apache/pinot/issues/10571#issuecomment-1499522710

   I identified deduplication to be very frequently necessary in use cases with kafka ingestion. Even if the produced messages are unique, it is still a challenge to build an exactly-once kafka delivery in order to avoid duplication. This is where I found pinot upsert (deduplication) to come very useful.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org