[jira] [Created] (KAFKA-395) kafka.tools.MirrorMaker black/white list improvement

["Dave DeMaagd (JIRA)" <ji...@apache.org>]

Dave DeMaagd created KAFKA-395:
----------------------------------

             Summary: kafka.tools.MirrorMaker black/white list improvement
                 Key: KAFKA-395
                 URL: https://issues.apache.org/jira/browse/KAFKA-395
             Project: Kafka
          Issue Type: Improvement
          Components: config
            Reporter: Dave DeMaagd
            Priority: Minor


Current black/white list topics are specified directly on the command line, while functional, this has two drawbacks:
1) Changes become unwieldy if there are a large number of running instances - potentially many instances to restart, which can have implications for data stream lag
2) Maintaining the list itself can become increasingly complex if there are a large number of elements in the list (particularly if they are complex expressions)

Suggest extending the way that black/white lists can be fed to the mirror maker application, in particular, being able to specify the black/white list as a file (or possibly a generic URI).  Thinking that this could be accomplished either by adding '--whitelistfile' and '--blacklistfile' command line parameters, or modifying the existing '--blacklist' and '--whitelist' parameters to include a 'is this a valid file?' test and decide how to handle it based on that (if it is a file, read it, if not, use current behavior). 

Follow up suggestion would be to have the mirror maker process check for updates to the list file, and on change, validate and reload it, and run from that point with the new list information. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira