[jira] [Created] (KAFKA-9814) Allow AdminClient.listTopics to list based on expression

["Viktor Somogyi-Vass (Jira)" <ji...@apache.org>]

Viktor Somogyi-Vass created KAFKA-9814:
------------------------------------------

             Summary: Allow AdminClient.listTopics to list based on expression
                 Key: KAFKA-9814
                 URL: https://issues.apache.org/jira/browse/KAFKA-9814
             Project: Kafka
          Issue Type: Improvement
    Affects Versions: 2.5.0
            Reporter: Viktor Somogyi-Vass
            Assignee: Viktor Somogyi-Vass


Currently listTopics would list all the topics in the cluster which often isn't desired due to the large number of topics and because the authorizer would check access for every topic which could result in various bloated access/audit logs that would make them hard to read and also in case of many topics it isn't really effective to return all of them due to the response's size.

To take an example let's have a user who issues a naive command such as:
{noformat}
kafka-topics --list --topic topic6 --bootstrap-server mycompany.com:9092 --command-config client.properties
{noformat}
In this case the admin client would issue a metadata request for all the topics. When this request gets to the broker then it will try to check permissions to every topic in the cluster and then return those which are allowed for "describe" to the client. Then the command when it gets the answer will simply filter out {{topic6}} from the response.

Looking at this problem we can see that if we allow the broker to do the filtering then we can:
* do less permission checks
* return less data to the clients

Regarding the solution I see two path but both of them of course subject community discussion:
# modify the MetadataRequest and put some kind of regex in it 
# create a new list API that would contain this filter pattern
Perhaps the list API would make more sense as the MetadataRequest is currently used because it just fits but there is no real reason to extend it with this as the functionality won't be used elsewhere.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)