[GitHub] [druid] maytasm commented on pull request #11078: Introduce a new configuration that skip storing audit payload if payload size exceed limit and skip storing null fields for audit payload

[GitBox <gi...@apache.org>]

maytasm commented on pull request #11078:
URL: https://github.com/apache/druid/pull/11078#issuecomment-815461883


   > I have a question. Users have no idea of the size of payload, are there any suggestions that tell users how to set this configuration ? Can we skip storing payload based on value of `type` ?
   
   In another PR, I am adding an auto cleanup based on size limit for the druid_audit metadata table. This work similar to druid.indexer.logs.kill.* which periodically clean up task entries in the druid_tasks table. Basically, user will be able to configured the max number of entries they want to store in the druid_audit metadata table, and Druid will automatically remove from oldest entry (FIFO) when the table entries exceed this size limit. Using both of these configuration, Druid operator will be able to control the size of the audit metadata table. For example, if you want to limit your audit table metadata to 20GB (that is how much storage you have on your machine, etc.), and say you want keep `x` number of last audit entries, you can then set `maxPayloadSizeBytes` to `20GB / x`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org