You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Shrey Mehrotra <sh...@impetus.co.in> on 2015/01/23 15:28:17 UTC
Multiple repository creation for single cluster.
Hi All,
We are creating repositories in Ranger and found that multiple HDFS or Hive repositories could be created for a single cluster (pointing to single namenode IP or metastore URI), but in agents we provide only one repository that would be used by the agent interceptors. Is this the expected behavior or could we restrict creation of 1 repository for one cluster (say hadoopdev for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining multiple repositories for a single cluster as only one is going to be used and would not be changed after Ranger deployment?
Any inputs would be helpful.
Thanks & Regards,
Shrey Mehrotra
Senior Software Engineer, iLabs
Impetus Infotech Pvt. Ltd.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Multiple repository creation for single cluster.
Posted by Selvamohan Neethiraj <sn...@apache.org>.
Hi:
Sure, Please do create a JIRA and add your proposal to solve on the same
JIRA.
+1 on the proposed solution from me.
Thanks,
Selva-
From: Shrey Mehrotra <sh...@impetus.co.in>
Reply-To: "user@ranger.incubator.apache.org"
<us...@ranger.incubator.apache.org>
Date: Friday, January 23, 2015 at 12:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>,
"dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
Subject: RE: Multiple repository creation for single cluster.
Hi Selva,
The feature of defining multiple repository from a single RANGER-ADMIN UI is
a good offering. We could restrict the creation of duplicate repositories
for same cluster by checking against the unique identifier for the
repository in the ranger DB at the time of creating a new one. It could be
namenode IP for HDFS and hive metastore uri for Hive and similarly for
others. Could I create a minor jira for the same and we could then discuss
and implement the solution for it.
Let me know your thoughts.
Thanks & Regards,
Shrey Mehrotra
Senior Software Engineer, iLabs
Impetus Infotech Pvt. Ltd.
From: Selvamohan Neethiraj [mailto:sneethir@apache.org]
Sent: Friday, January 23, 2015 8:53 PM
To: user@ranger.incubator.apache.org; dev@ranger.incubator.apache.org
Subject: Re: Multiple repository creation for single cluster.
Hi Shrey:
You raise a valid point here.
The indent for creation of multiple repositories was to support multiple
cluster from a single RANGER-ADMIN UI.
For example, the RANGER-ADMIN UI should be able to store policies for
hadoopdev as well as hadooptest clusters.
Instead of hosting a separate RANGER-ADMIN UI, you can use a single
RANGER-ADMIN UI to support multiple hadoop cluster instances.
Currently, the ranger ui does not have a way to STOP the admin from creating
two separate hadoop repositories (hadoop1, hadoop2) pointing to same hadoop
cluster instances.
However, hdfs plugin can be configured to pull policies from only one of the
hadoop repository.
Is there a good way to preventing the admin from creating two different
repositories for the same cluster?
Thanks,
Selva-
>
> On Jan 23, 2015, at 9:28 AM, Shrey Mehrotra <sh...@impetus.co.in>
> wrote:
>
>
> Hi All,
>
>
>
> We are creating repositories in Ranger and found that multiple HDFS or Hive
> repositories could be created for a single cluster (pointing to single
> namenode IP or metastore URI), but in agents we provide only one repository
> that would be used by the agent interceptors. Is this the expected behavior
> or could we restrict creation of 1 repository for one cluster (say hadoopdev
> for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining
> multiple repositories for a single cluster as only one is going to be used and
> would not be changed after Ranger deployment?
>
>
>
> Any inputs would be helpful.
>
>
>
> Thanks & Regards,
> Shrey Mehrotra
> Senior Software Engineer, iLabs
> Impetus Infotech Pvt. Ltd.
>
>
>
>
>
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary,
> privileged or otherwise protected by law. The message is intended solely for
> the named addressee. If received in error, please destroy and notify the
> sender. Any use of this email is prohibited when received in error. Impetus
> does not represent, warrant and/or guarantee, that the integrity of this
> communication has been maintained nor that the communication is free of
> errors, virus, interception or interference.
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
No virus found in this message.
Checked by AVG - www.avg.com <http://www.avg.com>
Version: 2014.0.4800 / Virus Database: 4257/8983 - Release Date: 01/23/15
NOTE: This message may contain information that is confidential,
proprietary, privileged or otherwise protected by law. The message is
intended solely for the named addressee. If received in error, please
destroy and notify the sender. Any use of this email is prohibited when
received in error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained nor that the
communication is free of errors, virus, interception or interference.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.
Re: Multiple repository creation for single cluster.
Posted by Selvamohan Neethiraj <sn...@apache.org>.
Hi:
Sure, Please do create a JIRA and add your proposal to solve on the same
JIRA.
+1 on the proposed solution from me.
Thanks,
Selva-
From: Shrey Mehrotra <sh...@impetus.co.in>
Reply-To: "user@ranger.incubator.apache.org"
<us...@ranger.incubator.apache.org>
Date: Friday, January 23, 2015 at 12:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>,
"dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
Subject: RE: Multiple repository creation for single cluster.
Hi Selva,
The feature of defining multiple repository from a single RANGER-ADMIN UI is
a good offering. We could restrict the creation of duplicate repositories
for same cluster by checking against the unique identifier for the
repository in the ranger DB at the time of creating a new one. It could be
namenode IP for HDFS and hive metastore uri for Hive and similarly for
others. Could I create a minor jira for the same and we could then discuss
and implement the solution for it.
Let me know your thoughts.
Thanks & Regards,
Shrey Mehrotra
Senior Software Engineer, iLabs
Impetus Infotech Pvt. Ltd.
From: Selvamohan Neethiraj [mailto:sneethir@apache.org]
Sent: Friday, January 23, 2015 8:53 PM
To: user@ranger.incubator.apache.org; dev@ranger.incubator.apache.org
Subject: Re: Multiple repository creation for single cluster.
Hi Shrey:
You raise a valid point here.
The indent for creation of multiple repositories was to support multiple
cluster from a single RANGER-ADMIN UI.
For example, the RANGER-ADMIN UI should be able to store policies for
hadoopdev as well as hadooptest clusters.
Instead of hosting a separate RANGER-ADMIN UI, you can use a single
RANGER-ADMIN UI to support multiple hadoop cluster instances.
Currently, the ranger ui does not have a way to STOP the admin from creating
two separate hadoop repositories (hadoop1, hadoop2) pointing to same hadoop
cluster instances.
However, hdfs plugin can be configured to pull policies from only one of the
hadoop repository.
Is there a good way to preventing the admin from creating two different
repositories for the same cluster?
Thanks,
Selva-
>
> On Jan 23, 2015, at 9:28 AM, Shrey Mehrotra <sh...@impetus.co.in>
> wrote:
>
>
> Hi All,
>
>
>
> We are creating repositories in Ranger and found that multiple HDFS or Hive
> repositories could be created for a single cluster (pointing to single
> namenode IP or metastore URI), but in agents we provide only one repository
> that would be used by the agent interceptors. Is this the expected behavior
> or could we restrict creation of 1 repository for one cluster (say hadoopdev
> for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining
> multiple repositories for a single cluster as only one is going to be used and
> would not be changed after Ranger deployment?
>
>
>
> Any inputs would be helpful.
>
>
>
> Thanks & Regards,
> Shrey Mehrotra
> Senior Software Engineer, iLabs
> Impetus Infotech Pvt. Ltd.
>
>
>
>
>
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary,
> privileged or otherwise protected by law. The message is intended solely for
> the named addressee. If received in error, please destroy and notify the
> sender. Any use of this email is prohibited when received in error. Impetus
> does not represent, warrant and/or guarantee, that the integrity of this
> communication has been maintained nor that the communication is free of
> errors, virus, interception or interference.
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
No virus found in this message.
Checked by AVG - www.avg.com <http://www.avg.com>
Version: 2014.0.4800 / Virus Database: 4257/8983 - Release Date: 01/23/15
NOTE: This message may contain information that is confidential,
proprietary, privileged or otherwise protected by law. The message is
intended solely for the named addressee. If received in error, please
destroy and notify the sender. Any use of this email is prohibited when
received in error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained nor that the
communication is free of errors, virus, interception or interference.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.
RE: Multiple repository creation for single cluster.
Posted by Shrey Mehrotra <sh...@impetus.co.in>.
Hi Selva,
The feature of defining multiple repository from a single RANGER-ADMIN UI is a good offering. We could restrict the creation of duplicate repositories for same cluster by checking against the unique identifier for the repository in the ranger DB at the time of creating a new one. It could be namenode IP for HDFS and hive metastore uri for Hive and similarly for others. Could I create a minor jira for the same and we could then discuss and implement the solution for it.
Let me know your thoughts.
Thanks & Regards,
Shrey Mehrotra
Senior Software Engineer, iLabs
Impetus Infotech Pvt. Ltd.
From: Selvamohan Neethiraj [mailto:sneethir@apache.org]
Sent: Friday, January 23, 2015 8:53 PM
To: user@ranger.incubator.apache.org; dev@ranger.incubator.apache.org
Subject: Re: Multiple repository creation for single cluster.
Hi Shrey:
You raise a valid point here.
The indent for creation of multiple repositories was to support multiple cluster from a single RANGER-ADMIN UI.
For example, the RANGER-ADMIN UI should be able to store policies for hadoopdev as well as hadooptest clusters.
Instead of hosting a separate RANGER-ADMIN UI, you can use a single RANGER-ADMIN UI to support multiple hadoop cluster instances.
Currently, the ranger ui does not have a way to STOP the admin from creating two separate hadoop repositories (hadoop1, hadoop2) pointing to same hadoop cluster instances.
However, hdfs plugin can be configured to pull policies from only one of the hadoop repository.
Is there a good way to preventing the admin from creating two different repositories for the same cluster?
Thanks,
Selva-
On Jan 23, 2015, at 9:28 AM, Shrey Mehrotra <sh...@impetus.co.in>> wrote:
Hi All,
We are creating repositories in Ranger and found that multiple HDFS or Hive repositories could be created for a single cluster (pointing to single namenode IP or metastore URI), but in agents we provide only one repository that would be used by the agent interceptors. Is this the expected behavior or could we restrict creation of 1 repository for one cluster (say hadoopdev for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining multiple repositories for a single cluster as only one is going to be used and would not be changed after Ranger deployment?
Any inputs would be helpful.
Thanks & Regards,
Shrey Mehrotra
Senior Software Engineer, iLabs
Impetus Infotech Pvt. Ltd.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 2014.0.4800 / Virus Database: 4257/8983 - Release Date: 01/23/15
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Multiple repository creation for single cluster.
Posted by Selvamohan Neethiraj <sn...@apache.org>.
Hi Shrey:
You raise a valid point here.
The indent for creation of multiple repositories was to support multiple cluster from a single RANGER-ADMIN UI.
For example, the RANGER-ADMIN UI should be able to store policies for hadoopdev as well as hadooptest clusters.
Instead of hosting a separate RANGER-ADMIN UI, you can use a single RANGER-ADMIN UI to support multiple hadoop cluster instances.
Currently, the ranger ui does not have a way to STOP the admin from creating two separate hadoop repositories (hadoop1, hadoop2) pointing to same hadoop cluster instances.
However, hdfs plugin can be configured to pull policies from only one of the hadoop repository.
Is there a good way to preventing the admin from creating two different repositories for the same cluster?
Thanks,
Selva-
> On Jan 23, 2015, at 9:28 AM, Shrey Mehrotra <sh...@impetus.co.in> wrote:
>
> Hi All,
>
> We are creating repositories in Ranger and found that multiple HDFS or Hive repositories could be created for a single cluster (pointing to single namenode IP or metastore URI), but in agents we provide only one repository that would be used by the agent interceptors. Is this the expected behavior or could we restrict creation of 1 repository for one cluster (say hadoopdev for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining multiple repositories for a single cluster as only one is going to be used and would not be changed after Ranger deployment?
>
> Any inputs would be helpful.
>
> Thanks & Regards,
> Shrey Mehrotra
> Senior Software Engineer, iLabs
> Impetus Infotech Pvt. Ltd.
>
>
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Multiple repository creation for single cluster.
Posted by Selvamohan Neethiraj <sn...@apache.org>.
Hi Shrey:
You raise a valid point here.
The indent for creation of multiple repositories was to support multiple cluster from a single RANGER-ADMIN UI.
For example, the RANGER-ADMIN UI should be able to store policies for hadoopdev as well as hadooptest clusters.
Instead of hosting a separate RANGER-ADMIN UI, you can use a single RANGER-ADMIN UI to support multiple hadoop cluster instances.
Currently, the ranger ui does not have a way to STOP the admin from creating two separate hadoop repositories (hadoop1, hadoop2) pointing to same hadoop cluster instances.
However, hdfs plugin can be configured to pull policies from only one of the hadoop repository.
Is there a good way to preventing the admin from creating two different repositories for the same cluster?
Thanks,
Selva-
> On Jan 23, 2015, at 9:28 AM, Shrey Mehrotra <sh...@impetus.co.in> wrote:
>
> Hi All,
>
> We are creating repositories in Ranger and found that multiple HDFS or Hive repositories could be created for a single cluster (pointing to single namenode IP or metastore URI), but in agents we provide only one repository that would be used by the agent interceptors. Is this the expected behavior or could we restrict creation of 1 repository for one cluster (say hadoopdev for namenode ip 192.168.25.47:8020) ?. What would be the benefit of defining multiple repositories for a single cluster as only one is going to be used and would not be changed after Ranger deployment?
>
> Any inputs would be helpful.
>
> Thanks & Regards,
> Shrey Mehrotra
> Senior Software Engineer, iLabs
> Impetus Infotech Pvt. Ltd.
>
>
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.