[GitHub] [pinot] ankitsultana commented on issue #10494: Partial Upsert Tables Read a Lot of MMAP'ed Data

["ankitsultana (via GitHub)" <gi...@apache.org>]

ankitsultana commented on issue #10494:
URL: https://github.com/apache/pinot/issues/10494#issuecomment-1487641599

   > For some context, do you observe this issue when some columns are raw (no dictionary) and also compressed? It will be the worst case when values are compressed but we just need to read one value out
   
   In our case we have ~100 columns. All are dictionary encoded. We also have a bit high ingestion rate (messages per second) for this use-case, so the number of columns multiplied with the ingestion rate worsen the issue.
   
   > @ankitsultana - Just curious, if you have any stats that can indicate how bad the situation is and also how frequently this happens ?
   
   We don't have any general stats to share. Most likely you'll run into this when your table is larger than mmap memory, and the (messages per second x number of columns) are "high".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org