Druid module separation (And a related lookups question)

[Eyal Yurman <ey...@verizonmedia.com.INVALID>]

Hi,

Can someone help me understand the logic for the separation of core, server
and processing modules, and why?

More specificly, I'm trying to understand the split of the package
`org.apache.druid.query.lookup` between the two latter modules.

Eyal.

Re: Druid module separation (And a related lookups question)

[Roman Leventov <le...@gmail.com>]

Hadoop and Spark batch indexing are the only contexts (I think) where only
'processing' is needed, but not 'server'.

The separation between 'core' and 'processing' is a historical artifact
that should be fixed, see
https://github.com/apache/incubator-druid/issues/4312#issuecomment-356127914

Also, when it is done, it would be nice to check whether each utility class
that is currently in 'core' is used in 'processing' (or new 'core', if
'processing' is renamed to 'core'), or only in 'server' and elsewhere. In
the latter case, the utility class should be moved to 'server'.

On Wed, 20 Mar 2019 at 18:59, Charles Allen <ch...@snap.com.invalid>
wrote:

> First, note that Server depends on Processing.
>
> Most of the things in Core are used everywhere and are underlying
> utilities. This has taken on various names like "common" and "util" in the
> past.
>
> Most of the things in Processing are there to facilitate the execution of a
> query.
>
> Most of the things in Server are there to facilitate the running of the
> service.
>
> There are exceptions to this, such
> as org.apache.druid.query.dimension.LookupDimensionSpec being in Server.
> The reason is that it currently requires reference to a thing in Server to
> bind to. This should probably be undone (via an interface) at some point
> but has not been a priority.
>
> Most of the other look up stuff in Server is about the metadata management
> and loading if I recall.
>
>
>
> On Wed, Mar 20, 2019 at 2:51 PM Eyal Yurman
> <ey...@verizonmedia.com.invalid> wrote:
>
> > Hi,
> >
> > Can someone help me understand the logic for the separation of core,
> server
> > and processing modules, and why?
> >
> > More specificly, I'm trying to understand the split of the package
> > `org.apache.druid.query.lookup` between the two latter modules.
> >
> > Eyal.
> >
>

Re: Druid module separation (And a related lookups question)

[Charles Allen <ch...@snap.com.INVALID>]

First, note that Server depends on Processing.

Most of the things in Core are used everywhere and are underlying
utilities. This has taken on various names like "common" and "util" in the
past.

Most of the things in Processing are there to facilitate the execution of a
query.

Most of the things in Server are there to facilitate the running of the
service.

There are exceptions to this, such
as org.apache.druid.query.dimension.LookupDimensionSpec being in Server.
The reason is that it currently requires reference to a thing in Server to
bind to. This should probably be undone (via an interface) at some point
but has not been a priority.

Most of the other look up stuff in Server is about the metadata management
and loading if I recall.



On Wed, Mar 20, 2019 at 2:51 PM Eyal Yurman
<ey...@verizonmedia.com.invalid> wrote:

> Hi,
>
> Can someone help me understand the logic for the separation of core, server
> and processing modules, and why?
>
> More specificly, I'm trying to understand the split of the package
> `org.apache.druid.query.lookup` between the two latter modules.
>
> Eyal.
>