You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by Kevin Wilfong <ke...@fb.com> on 2011/10/13 00:48:52 UTC

Review Request: Add whitelist for hosts used in table/partition locations

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2371/
-----------------------------------------------------------

Review request for hive and Paul Yang.


Summary
-------

I added a whitelist configuration variable, whose value should be a comma separated list of hosts.  A set of strings is created based on this list and the host name in a location given to a create/alter table/partition command to the HiveMetastore is compared to the values in that set.  If the list is nonempty, the table being altered/created or containing the partition is not external/non-native, and the location has an explicit host, the command is failed if the host is not in that set, otherwise the command continues as normal.


This addresses bug HIVE-2502.
    https://issues.apache.org/jira/browse/HIVE-2502


Diffs
-----

  trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1182505 
  trunk/conf/hive-default.xml 1182505 
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1182505 
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1182505 
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1182505 

Diff: https://reviews.apache.org/r/2371/diff


Testing
-------

I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:

Issuing the command with a location with a host that is not in the whitelist fails.
Issuing the command with a location with a host in the whitelist does not fail.


Thanks,

Kevin

Re: Review Request: Add whitelist for hosts used in table/partition locations

Posted by Kevin Wilfong <ke...@fb.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2371/#review2881
-----------------------------------------------------------



trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
<https://reviews.apache.org/r/2371/#comment6452>

    This is the only change I made in r4, it now only checks the location if the table is not a view.


- Kevin


On 2011-10-27 17:38:58, Kevin Wilfong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2371/
> -----------------------------------------------------------
> 
> (Updated 2011-10-27 17:38:58)
> 
> 
> Review request for hive and Paul Yang.
> 
> 
> Summary
> -------
> 
> I added a whitelist configuration variable, whose value should be a comma separated list of hosts.  A set of strings is created based on this list and the host name in a location given to a create/alter table/partition command to the HiveMetastore is compared to the values in that set.  If the list is nonempty, the table being altered/created or containing the partition is not external/non-native, and the location has an explicit host, the command is failed if the host is not in that set, otherwise the command continues as normal.
> 
> 
> This addresses bug HIVE-2502.
>     https://issues.apache.org/jira/browse/HIVE-2502
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1189406 
>   trunk/conf/hive-default.xml 1189406 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1189406 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1189406 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1189406 
> 
> Diff: https://reviews.apache.org/r/2371/diff
> 
> 
> Testing
> -------
> 
> I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:
> 
> Issuing the command with a location with a host that is not in the whitelist fails.
> Issuing the command with a location with a host in the whitelist does not fail.
> 
> 
> Thanks,
> 
> Kevin
> 
>

Re: Review Request: Add whitelist for hosts used in table/partition locations

Posted by Kevin Wilfong <ke...@fb.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2371/
-----------------------------------------------------------

(Updated 2011-10-27 17:38:58.120670)

Review request for hive and Paul Yang.

Changes
-------

I fixed an issue that was causing altering views to fail.

I ran all the test queries and they succeeded.

Summary
-------

I added a whitelist configuration variable, whose value should be a comma separated list of hosts. A set of strings is created based on this list and the host name in a location given to a create/alter table/partition command to the HiveMetastore is compared to the values in that set. If the list is nonempty, the table being altered/created or containing the partition is not external/non-native, and the location has an explicit host, the command is failed if the host is not in that set, otherwise the command continues as normal.

This addresses bug HIVE-2502.
https://issues.apache.org/jira/browse/HIVE-2502

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1189406
trunk/conf/hive-default.xml 1189406
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1189406
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1189406
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1189406

Diff: https://reviews.apache.org/r/2371/diff

Testing
-------

I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:

Issuing the command with a location with a host that is not in the whitelist fails.
Issuing the command with a location with a host in the whitelist does not fail.

Thanks,

Kevin

Re: Review Request: Add whitelist for hosts used in table/partition locations

Posted by Kevin Wilfong <ke...@fb.com>.

(Updated 2011-10-14 16:56:01.946053)

Review request for hive and Paul Yang.

Changes
-------

All lines are within the 100 char limit.

Summary
-------

This addresses bug HIVE-2502.
https://issues.apache.org/jira/browse/HIVE-2502

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1182505
trunk/conf/hive-default.xml 1182505
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1182505
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1182505
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1182505

Diff: https://reviews.apache.org/r/2371/diff

Testing
-------

I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:

Issuing the command with a location with a host that is not in the whitelist fails.
Issuing the command with a location with a host in the whitelist does not fail.

Thanks,

Kevin

Re: Review Request: Add whitelist for hosts used in table/partition locations

Posted by Kevin Wilfong <ke...@fb.com>.

(Updated 2011-10-14 01:27:18.448099)

Review request for hive and Paul Yang.

Changes
-------

Thanks for the feedback Paul. I changed it to use host:port pairs instead of just hosts.

Summary
-------

This addresses bug HIVE-2502.
https://issues.apache.org/jira/browse/HIVE-2502

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/2371/diff

Testing
-------

I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:

Issuing the command with a location with a host that is not in the whitelist fails.
Issuing the command with a location with a host in the whitelist does not fail.

Thanks,

Kevin

Re: Review Request: Add whitelist for hosts used in table/partition locations

Posted by Paul Yang <py...@fb.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2371/#review2575
-----------------------------------------------------------


Looks good, but can we do host:port combinations instead of just the host?

- Paul


On 2011-10-12 22:48:52, Kevin Wilfong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2371/
> -----------------------------------------------------------
> 
> (Updated 2011-10-12 22:48:52)
> 
> 
> Review request for hive and Paul Yang.
> 
> 
> Summary
> -------
> 
> I added a whitelist configuration variable, whose value should be a comma separated list of hosts.  A set of strings is created based on this list and the host name in a location given to a create/alter table/partition command to the HiveMetastore is compared to the values in that set.  If the list is nonempty, the table being altered/created or containing the partition is not external/non-native, and the location has an explicit host, the command is failed if the host is not in that set, otherwise the command continues as normal.
> 
> 
> This addresses bug HIVE-2502.
>     https://issues.apache.org/jira/browse/HIVE-2502
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1182505 
>   trunk/conf/hive-default.xml 1182505 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1182505 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1182505 
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1182505 
> 
> Diff: https://reviews.apache.org/r/2371/diff
> 
> 
> Testing
> -------
> 
> I used Thrift calls to test the following for each of the 4 commands create/alter table/partition:
> 
> Issuing the command with a location with a host that is not in the whitelist fails.
> Issuing the command with a location with a host in the whitelist does not fail.
> 
> 
> Thanks,
> 
> Kevin
> 
>