[Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Zuyu Zhang <zu...@apache.org>]

Hi Harshad,

I am working on the distributed query execution engine, and would like to
have a discussion regarding how both PolicyEnforcer and QueryManager fit
into the distributed case.

I was wondering if we could remove using StorageManager in PolicyEnforcer.
In other words, could Foreman creates QueryManagers directly, instead of
PolicyEnforcer::admitQuery?

The main concern is that like in Foreman, PolicyEnforcer will be created in
ForemanDistributed, which, however, does not have access to StorageManager
in the distributed case.

Any thoughts?

Cheers,
Zuyu

Re: [Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Zuyu Zhang <zu...@apache.org>]

In the distributed version, ForemanDistributed would pass a nullptr of
StorageManager to PolicyEnforcer. And QueryManager in the distributed
version would not use StorageManager at all, as QueryContext would be
reconstructed in Worker site.

Actually, we may use the QUICKSTEP_DISTRIBUTED micro in QueryManager so
that we may avoid the class hierarchy as a whole.
​

Re: [Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Harshad Deshmukh <ha...@cs.wisc.edu>]

Hi Zuyu,

I am not sure how this hierarchy will solve the StorageManager issue we 
were discussing. Can you please explain?

On a slightly unrelated and less important note, I feel the suffix 
"lite" for base classes may not be the best choice. I know that protobuf 
uses this suffix but their context is different (e.g. MessageLite stands 
for light weight messages).

On 06/23/2016 03:49 PM, Zuyu Zhang wrote:
> Hi Harshad,
>
> After a second though, I have the following proposal:
>
> We add a new base class called QueryManagerLite, from which the 
> current QueryManager and the future QueryManagerDistributed would 
> derive. Thus, we could keep the current QueryManager intact, including 
> holding QueryContext.
>
> With that being said, in the distributed version, we could reuse 
> PolicyEnforcer, but both ForemanDistributed and 
> QueryManagerDistributed are unique implementations.
>
> Any thoughts?
>
> Cheers,
> Zuyu

-- 
Thanks,
Harshad

Re: [Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Zuyu Zhang <zu...@apache.org>]

Hi Harshad,

After a second though, I have the following proposal:

We add a new base class called QueryManagerLite, from which the current
QueryManager and the future QueryManagerDistributed would derive. Thus, we
could keep the current QueryManager intact, including holding QueryContext.

With that being said, in the distributed version, we could reuse
PolicyEnforcer, but both ForemanDistributed and QueryManagerDistributed are
unique implementations.

Any thoughts?

Cheers,
Zuyu

Re: [Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Zuyu Zhang <zu...@apache.org>]

The main issue is that QueryManager -> QueryContext. I don't think late
binding would work, since in the distributed case, QueryContext will be
reconstructed by Worker's Shiftboss.

So in the single node case, if Foreman could create QueryContext w/
StorageManager, it would be fine.

Zuyu

Re: [Design Discussion] StorageManager in PolicyEnforcer in Distributed Case

[Harshad Deshmukh <ha...@cs.wisc.edu>]

Hi Zuyu,

I understand your concern on non-availability of storage manager in 
distributed case.

I am not sure I fully understand the following proposal:
 > Could Foreman creates QueryManagers directly, instead of 
PolicyEnforcer::admitQuery?
Can you elaborate?

Right now, storage manager is propagated from Foreman -> Policy Enforcer 
-> Query Manager -> Query Context. Developing some kind of late binding 
mechanism of storage manager can be one possible solution.

On 06/20/2016 12:42 PM, Zuyu Zhang wrote:
> Hi Harshad,
>
> I am working on the distributed query execution engine, and would like 
> to have a discussion regarding how both PolicyEnforcer and 
> QueryManager fit into the distributed case.
>
> I was wondering if we could remove using StorageManager in 
> PolicyEnforcer. In other words, could Foreman creates QueryManagers 
> directly, instead of PolicyEnforcer::admitQuery?
>
> The main concern is that like in Foreman, PolicyEnforcer will be 
> created in ForemanDistributed, which, however, does not have access to 
> StorageManager in the distributed case.
>
> Any thoughts?
>
> Cheers,
> Zuyu

-- 
Thanks,
Harshad