You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Mugdha Varadkar <mu...@freestoneinfotech.com> on 2017/06/14 13:37:07 UTC

Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/
-----------------------------------------------------------

Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.


Bugs: AMBARI-21190
    https://issues.apache.org/jira/browse/AMBARI-21190


Repository: ambari


Description
-------

populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
2) it will be good to add this properly in advanced-ranger-hive-security


Diffs
-----

  ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
  ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
  ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 


Diff: https://reviews.apache.org/r/60078/diff/1/


Testing
-------

Tested Ranger Install + Enabled Hive Plugin.


Thanks,

Mugdha Varadkar


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178310
-----------------------------------------------------------


Ship it!




Ship It!

- Alejandro Fernandez


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.

> On June 16, 2017, 11:20 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> > Lines 61 (patched)
> > <https://reviews.apache.org/r/60078/diff/1/?file=1750495#file1750495line61>
> >
> >     If an existing cluster with Hive and Ranger is Kerberized, this logic may also need to exist in Stack Advisor so that config changes cause it to be saved.

Hi Alejandro,

This property is used for Hive when Hive is configured to use different file-system schemes supported by HDFS, it is also independent of whether the cluster is kerberized or not. It is a new property introduced recently.
For existing cluster, stack advisor logic will not be needed to recommend. If user want this property, they can add it via custom-config or it will be added during minor upgrade of 2.6.x to 2.6.y


- Mugdha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178136
-----------------------------------------------------------


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178136
-----------------------------------------------------------




ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
Lines 61 (patched)
<https://reviews.apache.org/r/60078/#comment251972>

    If an existing cluster with Hive and Ranger is Kerberized, this logic may also need to exist in Stack Advisor so that config changes cause it to be saved.


- Alejandro Fernandez


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.

> On June 14, 2017, 3:12 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml
> > Lines 1-33 (patched)
> > <https://reviews.apache.org/r/60078/diff/1/?file=1750494#file1750494line1>
> >
> >     This file and the new property already exist at the common-services level; why add it here too? It should just inherit from common-services.

Hi Jonathan,

Hive service for stack 3.0 is extending new common-service version 2.1.0.3.0. All configs from HDP 2.0.6 – 2.6 are flattend out in common-services version 2.1.0.3.0 for stack 3.0. So adding the same property in common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml.

Adding this property here for stack 2.6


- Mugdha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review177899
-----------------------------------------------------------


On June 14, 2017, 1:37 p.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 1:37 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review177899
-----------------------------------------------------------


Fix it, then Ship it!





ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml
Lines 1-33 (patched)
<https://reviews.apache.org/r/60078/#comment251599>

    This file and the new property already exist at the common-services level; why add it here too? It should just inherit from common-services.


- Jonathan Hurley


On June 14, 2017, 9:37 a.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 9:37 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>


Re: Review Request 60078: AMBARI-21190 Populate urlauth filesystem schemes in Ranger hive plugin

Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60078/#review178106
-----------------------------------------------------------


Ship it!




Ship It!

- Jonathan Hurley


On June 14, 2017, 9:37 a.m., Mugdha Varadkar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60078/
> -----------------------------------------------------------
> 
> (Updated June 14, 2017, 9:37 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21190
>     https://issues.apache.org/jira/browse/AMBARI-21190
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> populate ranger.plugin.hive.urlauth.filesystem.schemes default value to hdfs:,file:,wasb:,adl:
> 1) it will be better if we keep default value of ranger.plugin.hive.urlauth.filesystem.schemes=hdfs:,file:,wasb:,adl:, if it does not harm to have all of the fs values in this properly it will be good to add them.
> 2) it will be good to add this properly in advanced-ranger-hive-security
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/configuration/ranger-hive-security.xml d98bc21 
>   ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-ugsync-site.xml 439c495 
>   ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml 2c62851 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/ranger-hive-security.xml PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 1610bb5 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml 1cdd184 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml 3e7e3d7 
> 
> 
> Diff: https://reviews.apache.org/r/60078/diff/1/
> 
> 
> Testing
> -------
> 
> Tested Ranger Install + Enabled Hive Plugin.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>