You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/05/05 21:48:34 UTC
[GitHub] [incubator-pinot] mqliang opened a new issue #6882: Use middleware mechanism to do preprocessing
mqliang opened a new issue #6882:
URL: https://github.com/apache/incubator-pinot/issues/6882
Copy from discussion at https://github.com/apache/incubator-pinot/pull/6811#issuecomment-827016380
I observe there are bunch of pre-processing logic for a request:
* authz/authn
* admission control (e.g. disallow user to create new segments if existing number of segments exceed a threshold. We don't have this logic yet, but it's useful and may be added in future)
* request validation (validate table schema, validate queries etc)
* request rewriting( e.g. this [PR](https://github.com/apache/incubator-pinot/pull/6811) rewrite request by normalizing LHS and RHS for numerical types)
* log and audit.
Now all those logic are put into BrokerRequestHandler, which is a little messy and hard to maintain. Ideally, we should use HTTP middleware to decouple those pre-processing/post-processing logic from the actual query processing logic. Most web framework provide such a HTTP middleware mechanism out of box, the HTTP library JAX-RS we use also provide similar mechanism: [Filters and Interceptors](https://dennis-xlc.gitbooks.io/restful-java-with-jax-rs-2-0-2rd-edition/content/en/part1/chapter12/filters_and_interceptors.html)
cc @Jackie-Jiang @siddharthteotia @snleee @mcvsubbu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6882: Use middleware mechanism to do preprocessing
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6882:
URL: https://github.com/apache/incubator-pinot/issues/6882#issuecomment-833949956
Authz/authn can be done in an HTTP middleware, the query specific check/rewrite does not fit well in the middleware IMO. AFAIK LinkedIn does not use the broker rest endpoint and has their own restli endpoint, and that is the reason why authz/authn is implemented within the request handler in the first place
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org