You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/03/10 00:53:40 UTC
[GitHub] [incubator-druid] clintropolis opened a new pull request #7222:
refactor lookups to be more chill to router
clintropolis opened a new pull request #7222: refactor lookups to be more chill to router
URL: https://github.com/apache/incubator-druid/pull/7222
This PR introduces `RouterLookupModule` which allows the router to perform query serde but without having to load lookups as happens using the stock `LookupModule`. To achieve this, an interface has been extracted from `LookupReferencesManager`, the very java named `LookupExtractorFactoryContainerProvider`, which is now used everywhere `LookupReferencesManager` was previously used, _except_ for `LookupListeningResource`. I think this is a bit nicer, because now all query processing stuff no longer has obvious access to methods to load and drop lookups, only the method to get the `LookupExtractorFactoryContainer` needed to do the transforms.
An integration test has been added to the wiki tests, which loads a simple json lookup, and tests queries against brokers and routers. This also means there is now an integration test that is testing lookups!
The router will proxy requests to the lookup introspection API it was previously serving, so if the router was being queried to introspect lookups things should still function, it will just be proxied to a default broker now.
I suspect it wasn't _too much_ of an issue, because that API didn't appear functional because didn't have auth checks performed, and would throw an exception of the form
```
2019-03-10T00:15:20,298 WARN [qtp2080864181-118] org.eclipse.jetty.server.HttpChannel - /druid/v1/lookups/introspect/simple-wiki
org.apache.druid.java.util.common.ISE: Request did not have an authorization check performed.
at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.handleAuthorizationCheckError(PreResponseAuthorizationCheckFilter.java:156) ~[classes/:?]
at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:89) ~[classes/:?]
...
```
So I've included a fix for that as well.
I also made some minor adjustments to the integration tests configurations, switching gc from cms to g1, and lowering some thread pool sizes to maybe result in a slightly lighter footprint.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org