You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by "Vikas Singh (Tech - VS)" <vi...@flipkart.com> on 2015/05/13 14:13:36 UTC
Re: Authorization Filter in lens
Hi Amareshwari
Thanks for the reply.
Just to set the context, According to the above patch the rewrite flow will
be-
query --> rewriteToCubeQL --> rewriteToHQL
Here rewriteToHQL does the cube rewrites which are similar to previous
functionality. So the hook is added before any Cube rewrite is happening
(before any actual fact table, storage table, candidate table resolution
happens). In our use case we need rewrite to happen after all these
resolutions happen. So having a rewriter at HQL Driver level, analogous to
JDBC Driver will give flexibility to do rewrite at the CubeLevel as well as
at the driver level (based on actual storage table, candidate tables).
So I propose adding a rewriter to HiveDriver as well. Please share your
thoughts.
Regards
Vikas Singh
On Thu, Apr 30, 2015 at 11:30 AM, amareshwarisr . <am...@gmail.com>
wrote:
> Hello Vikas,
>
> Sorry missed replying this mail.
>
> From the requirement put up, i see query needs to be rewritten with where
> clause appended. For plugging in any new rewriter, there is open issue
> LENS-24 (https://issues.apache.org/jira/browse/LENS-24) and the changes
> put
> up at https://reviews.apache.org/r/29110/. The changes need a rebase with
> master. One the pluggable rewriter is available, plugging in AuthRewriter
> should be simple.
>
> Thanks
> Amareshwari
>
>
> On Thu, Apr 23, 2015 at 11:16 AM, Vikas Singh (Tech - VS) <
> vikas.singh@flipkart.com> wrote:
>
> > Problem Statement: Implementing Authorisation filter in lens.
> >
> > The way we do Row level authorisation is -
> > For row level security, we have auth_db which stores map of significant
> > columns(In which we are interested to do row level security) and allowed
> > values per user. During run time we get all the Facts associated in
> query,
> > get their column set and do a intersection with set returned from
> auth_db.
> > Resulting set is appended as "IN Clause" or column!=value to the query
> > accordingly.
> >
> > My guess is the best place to do it will be driver ReWriter level as
> it
> > will handle CubeQl and bypassed HQL seamlessly. I was wondering if there
> is
> > any better place to do this, i.e. any hooks available.
> >
> > Regards
> > Vikas Singh
> >
>
Re: Authorization Filter in lens
Posted by "amareshwarisr ." <am...@gmail.com>.
As LENS-24 is not in active development, unless a contributor picks it and
updates, we cannot say when the changes will be available.
On Mon, May 18, 2015 at 12:44 PM, Vikas Singh (Tech - VS) <
vikas.singh@flipkart.com> wrote:
> Hi Amareshwari
>
> Thanks for the reply, I will create the Jira, Also I was wondering when the
> above changes will be merged/rebased to the main codebase.
>
> Regards
> Vikas Singh
>
> On Fri, May 15, 2015 at 9:18 AM, amareshwarisr . <am...@gmail.com>
> wrote:
>
> > Adding rewriter in HiveDriver should not be an issue. Will be different
> > jira than LENS-24.
> >
> > Thanks
> > Amareshwari
> >
> > On Wed, May 13, 2015 at 5:43 PM, Vikas Singh (Tech - VS) <
> > vikas.singh@flipkart.com> wrote:
> >
> > > Hi Amareshwari
> > >
> > > Thanks for the reply.
> > > Just to set the context, According to the above patch the rewrite flow
> > will
> > > be-
> > >
> > > query --> rewriteToCubeQL --> rewriteToHQL
> > > Here rewriteToHQL does the cube rewrites which are similar to
> > previous
> > > functionality. So the hook is added before any Cube rewrite is
> happening
> > > (before any actual fact table, storage table, candidate table
> resolution
> > > happens). In our use case we need rewrite to happen after all these
> > > resolutions happen. So having a rewriter at HQL Driver level, analogous
> > to
> > > JDBC Driver will give flexibility to do rewrite at the CubeLevel as
> well
> > as
> > > at the driver level (based on actual storage table, candidate tables).
> > >
> > > So I propose adding a rewriter to HiveDriver as well. Please share
> > your
> > > thoughts.
> > >
> > > Regards
> > > Vikas Singh
> > >
> > > On Thu, Apr 30, 2015 at 11:30 AM, amareshwarisr . <
> amareshwari@gmail.com
> > >
> > > wrote:
> > >
> > > > Hello Vikas,
> > > >
> > > > Sorry missed replying this mail.
> > > >
> > > > From the requirement put up, i see query needs to be rewritten with
> > where
> > > > clause appended. For plugging in any new rewriter, there is open
> issue
> > > > LENS-24 (https://issues.apache.org/jira/browse/LENS-24) and the
> > changes
> > > > put
> > > > up at https://reviews.apache.org/r/29110/. The changes need a rebase
> > > with
> > > > master. One the pluggable rewriter is available, plugging in
> > AuthRewriter
> > > > should be simple.
> > > >
> > > > Thanks
> > > > Amareshwari
> > > >
> > > >
> > > > On Thu, Apr 23, 2015 at 11:16 AM, Vikas Singh (Tech - VS) <
> > > > vikas.singh@flipkart.com> wrote:
> > > >
> > > > > Problem Statement: Implementing Authorisation filter in lens.
> > > > >
> > > > > The way we do Row level authorisation is -
> > > > > For row level security, we have auth_db which stores map of
> > significant
> > > > > columns(In which we are interested to do row level security) and
> > > allowed
> > > > > values per user. During run time we get all the Facts associated in
> > > > query,
> > > > > get their column set and do a intersection with set returned from
> > > > auth_db.
> > > > > Resulting set is appended as "IN Clause" or column!=value to the
> > query
> > > > > accordingly.
> > > > >
> > > > > My guess is the best place to do it will be driver ReWriter
> level
> > > as
> > > > it
> > > > > will handle CubeQl and bypassed HQL seamlessly. I was wondering if
> > > there
> > > > is
> > > > > any better place to do this, i.e. any hooks available.
> > > > >
> > > > > Regards
> > > > > Vikas Singh
> > > > >
> > > >
> > >
> >
>
Re: Authorization Filter in lens
Posted by "Vikas Singh (Tech - VS)" <vi...@flipkart.com>.
Hi all
Created Jira Link
https://issues.apache.org/jira/browse/LENS-561
Regards
Vikas Singh
On Mon, May 18, 2015 at 12:44 PM, Vikas Singh (Tech - VS) <
vikas.singh@flipkart.com> wrote:
> Hi Amareshwari
>
> Thanks for the reply, I will create the Jira, Also I was wondering when
> the above changes will be merged/rebased to the main codebase.
>
> Regards
> Vikas Singh
>
> On Fri, May 15, 2015 at 9:18 AM, amareshwarisr . <am...@gmail.com>
> wrote:
>
>> Adding rewriter in HiveDriver should not be an issue. Will be different
>> jira than LENS-24.
>>
>> Thanks
>> Amareshwari
>>
>> On Wed, May 13, 2015 at 5:43 PM, Vikas Singh (Tech - VS) <
>> vikas.singh@flipkart.com> wrote:
>>
>> > Hi Amareshwari
>> >
>> > Thanks for the reply.
>> > Just to set the context, According to the above patch the rewrite flow
>> will
>> > be-
>> >
>> > query --> rewriteToCubeQL --> rewriteToHQL
>> > Here rewriteToHQL does the cube rewrites which are similar to
>> previous
>> > functionality. So the hook is added before any Cube rewrite is happening
>> > (before any actual fact table, storage table, candidate table resolution
>> > happens). In our use case we need rewrite to happen after all these
>> > resolutions happen. So having a rewriter at HQL Driver level, analogous
>> to
>> > JDBC Driver will give flexibility to do rewrite at the CubeLevel as
>> well as
>> > at the driver level (based on actual storage table, candidate tables).
>> >
>> > So I propose adding a rewriter to HiveDriver as well. Please share
>> your
>> > thoughts.
>> >
>> > Regards
>> > Vikas Singh
>> >
>> > On Thu, Apr 30, 2015 at 11:30 AM, amareshwarisr . <
>> amareshwari@gmail.com>
>> > wrote:
>> >
>> > > Hello Vikas,
>> > >
>> > > Sorry missed replying this mail.
>> > >
>> > > From the requirement put up, i see query needs to be rewritten with
>> where
>> > > clause appended. For plugging in any new rewriter, there is open issue
>> > > LENS-24 (https://issues.apache.org/jira/browse/LENS-24) and the
>> changes
>> > > put
>> > > up at https://reviews.apache.org/r/29110/. The changes need a rebase
>> > with
>> > > master. One the pluggable rewriter is available, plugging in
>> AuthRewriter
>> > > should be simple.
>> > >
>> > > Thanks
>> > > Amareshwari
>> > >
>> > >
>> > > On Thu, Apr 23, 2015 at 11:16 AM, Vikas Singh (Tech - VS) <
>> > > vikas.singh@flipkart.com> wrote:
>> > >
>> > > > Problem Statement: Implementing Authorisation filter in lens.
>> > > >
>> > > > The way we do Row level authorisation is -
>> > > > For row level security, we have auth_db which stores map of
>> significant
>> > > > columns(In which we are interested to do row level security) and
>> > allowed
>> > > > values per user. During run time we get all the Facts associated in
>> > > query,
>> > > > get their column set and do a intersection with set returned from
>> > > auth_db.
>> > > > Resulting set is appended as "IN Clause" or column!=value to the
>> query
>> > > > accordingly.
>> > > >
>> > > > My guess is the best place to do it will be driver ReWriter
>> level
>> > as
>> > > it
>> > > > will handle CubeQl and bypassed HQL seamlessly. I was wondering if
>> > there
>> > > is
>> > > > any better place to do this, i.e. any hooks available.
>> > > >
>> > > > Regards
>> > > > Vikas Singh
>> > > >
>> > >
>> >
>>
>
>
Re: Authorization Filter in lens
Posted by "Vikas Singh (Tech - VS)" <vi...@flipkart.com>.
Hi Amareshwari
Thanks for the reply, I will create the Jira, Also I was wondering when the
above changes will be merged/rebased to the main codebase.
Regards
Vikas Singh
On Fri, May 15, 2015 at 9:18 AM, amareshwarisr . <am...@gmail.com>
wrote:
> Adding rewriter in HiveDriver should not be an issue. Will be different
> jira than LENS-24.
>
> Thanks
> Amareshwari
>
> On Wed, May 13, 2015 at 5:43 PM, Vikas Singh (Tech - VS) <
> vikas.singh@flipkart.com> wrote:
>
> > Hi Amareshwari
> >
> > Thanks for the reply.
> > Just to set the context, According to the above patch the rewrite flow
> will
> > be-
> >
> > query --> rewriteToCubeQL --> rewriteToHQL
> > Here rewriteToHQL does the cube rewrites which are similar to
> previous
> > functionality. So the hook is added before any Cube rewrite is happening
> > (before any actual fact table, storage table, candidate table resolution
> > happens). In our use case we need rewrite to happen after all these
> > resolutions happen. So having a rewriter at HQL Driver level, analogous
> to
> > JDBC Driver will give flexibility to do rewrite at the CubeLevel as well
> as
> > at the driver level (based on actual storage table, candidate tables).
> >
> > So I propose adding a rewriter to HiveDriver as well. Please share
> your
> > thoughts.
> >
> > Regards
> > Vikas Singh
> >
> > On Thu, Apr 30, 2015 at 11:30 AM, amareshwarisr . <amareshwari@gmail.com
> >
> > wrote:
> >
> > > Hello Vikas,
> > >
> > > Sorry missed replying this mail.
> > >
> > > From the requirement put up, i see query needs to be rewritten with
> where
> > > clause appended. For plugging in any new rewriter, there is open issue
> > > LENS-24 (https://issues.apache.org/jira/browse/LENS-24) and the
> changes
> > > put
> > > up at https://reviews.apache.org/r/29110/. The changes need a rebase
> > with
> > > master. One the pluggable rewriter is available, plugging in
> AuthRewriter
> > > should be simple.
> > >
> > > Thanks
> > > Amareshwari
> > >
> > >
> > > On Thu, Apr 23, 2015 at 11:16 AM, Vikas Singh (Tech - VS) <
> > > vikas.singh@flipkart.com> wrote:
> > >
> > > > Problem Statement: Implementing Authorisation filter in lens.
> > > >
> > > > The way we do Row level authorisation is -
> > > > For row level security, we have auth_db which stores map of
> significant
> > > > columns(In which we are interested to do row level security) and
> > allowed
> > > > values per user. During run time we get all the Facts associated in
> > > query,
> > > > get their column set and do a intersection with set returned from
> > > auth_db.
> > > > Resulting set is appended as "IN Clause" or column!=value to the
> query
> > > > accordingly.
> > > >
> > > > My guess is the best place to do it will be driver ReWriter level
> > as
> > > it
> > > > will handle CubeQl and bypassed HQL seamlessly. I was wondering if
> > there
> > > is
> > > > any better place to do this, i.e. any hooks available.
> > > >
> > > > Regards
> > > > Vikas Singh
> > > >
> > >
> >
>
Re: Authorization Filter in lens
Posted by "amareshwarisr ." <am...@gmail.com>.
Adding rewriter in HiveDriver should not be an issue. Will be different
jira than LENS-24.
Thanks
Amareshwari
On Wed, May 13, 2015 at 5:43 PM, Vikas Singh (Tech - VS) <
vikas.singh@flipkart.com> wrote:
> Hi Amareshwari
>
> Thanks for the reply.
> Just to set the context, According to the above patch the rewrite flow will
> be-
>
> query --> rewriteToCubeQL --> rewriteToHQL
> Here rewriteToHQL does the cube rewrites which are similar to previous
> functionality. So the hook is added before any Cube rewrite is happening
> (before any actual fact table, storage table, candidate table resolution
> happens). In our use case we need rewrite to happen after all these
> resolutions happen. So having a rewriter at HQL Driver level, analogous to
> JDBC Driver will give flexibility to do rewrite at the CubeLevel as well as
> at the driver level (based on actual storage table, candidate tables).
>
> So I propose adding a rewriter to HiveDriver as well. Please share your
> thoughts.
>
> Regards
> Vikas Singh
>
> On Thu, Apr 30, 2015 at 11:30 AM, amareshwarisr . <am...@gmail.com>
> wrote:
>
> > Hello Vikas,
> >
> > Sorry missed replying this mail.
> >
> > From the requirement put up, i see query needs to be rewritten with where
> > clause appended. For plugging in any new rewriter, there is open issue
> > LENS-24 (https://issues.apache.org/jira/browse/LENS-24) and the changes
> > put
> > up at https://reviews.apache.org/r/29110/. The changes need a rebase
> with
> > master. One the pluggable rewriter is available, plugging in AuthRewriter
> > should be simple.
> >
> > Thanks
> > Amareshwari
> >
> >
> > On Thu, Apr 23, 2015 at 11:16 AM, Vikas Singh (Tech - VS) <
> > vikas.singh@flipkart.com> wrote:
> >
> > > Problem Statement: Implementing Authorisation filter in lens.
> > >
> > > The way we do Row level authorisation is -
> > > For row level security, we have auth_db which stores map of significant
> > > columns(In which we are interested to do row level security) and
> allowed
> > > values per user. During run time we get all the Facts associated in
> > query,
> > > get their column set and do a intersection with set returned from
> > auth_db.
> > > Resulting set is appended as "IN Clause" or column!=value to the query
> > > accordingly.
> > >
> > > My guess is the best place to do it will be driver ReWriter level
> as
> > it
> > > will handle CubeQl and bypassed HQL seamlessly. I was wondering if
> there
> > is
> > > any better place to do this, i.e. any hooks available.
> > >
> > > Regards
> > > Vikas Singh
> > >
> >
>