[GitHub] [spark] HeartSaVioR commented on issue #24627: [SPARK-27748][SS] Kafka consumer/producer password/token redaction.

[GitBox <gi...@apache.org>]

HeartSaVioR commented on issue #24627: [SPARK-27748][SS] Kafka consumer/producer password/token redaction.
URL: https://github.com/apache/spark/pull/24627#issuecomment-494267955
 
 
   I know this patch just apply redaction to all the places which print configuration as of now, but I'm feeling that Kafka module is too verbose on configuration, especially KafkaConfigUpdater. Assuming debug log level is turned on, you would majorly want to track which consumers are being acquired or evicted, or closed. Logging in KafkaConfigUpdater doesn't help either.
   
   Let's revisit the purpose of logging. I guess we are logging to find when consumer/producer is created, and when it's evicted/closed. (either tracking each instance, or counting for each key). Though we use whole configuration as a key of cache, we (as human) cannot check every key-value pairs to find same instance. Some pairs play as a differentiator and we (again, as human) just assume other pairs are not different.
   
   So IMHO we don't even need to print out all configurations. Especially, if we really want to track each instance, explicit ID (as #19096 introduces) would work better and much helpful to debug.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org