You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Na Li via Review Board <no...@reviews.apache.org> on 2018/12/18 17:33:20 UTC
Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side
in addition to client-side
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/
-----------------------------------------------------------
Review request for hive, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
Bugs: HIVE-20776
https://issues.apache.org/jira/browse/HIVE-20776
Repository: hive-git
Description
-------
add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
Diffs
-----
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe
standalone-metastore/metastore-server/pom.xml 895abfc
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96d
Diff: https://reviews.apache.org/r/69585/diff/1/
Testing
-------
Existing unit tests passed.
Will add new unit tests for filtering at HMS server in next update of this jira
Will add code to enabled/disable filtering at HMS client based on configuration in next update of this jira
Thanks,
Na Li
Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on
server-side in addition to client-side
Posted by Na Li via Review Board <no...@reviews.apache.org>.
> On Dec. 19, 2018, 4:43 p.m., Karthik Manamcheri wrote:
> > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 2956 (patched)
> > <https://reviews.apache.org/r/69585/diff/2/?file=2114853#file2114853line2956>
> >
> > All of this logic of endFunction seems to be inside getTableInternal. Why duplicate it here?
I wanted to show the filtering overhead. That is why. Now I move the filter code in getTableInternal(), and remove the added code in get_table()
- Na
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review211436
-----------------------------------------------------------
On Dec. 19, 2018, 5:07 p.m., Na Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2018, 5:07 p.m.)
>
>
> Review request for hive, Adam Holley, Morio Ramdenbourg, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
>
>
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
>
>
> Diffs
> -----
>
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9eb1193a27120b5167f92daf67bf6a1c4e1d9927
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd
> standalone-metastore/metastore-server/pom.xml 895abfc423f00b121ee63e40904f5b3e57aea8ed
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96dcf62d3536cab2ce074d27a6225b2d3443
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/69585/diff/2/
>
>
> Testing
> -------
>
> Existing unit tests passed.
> add new unit tests for filtering at HMS server and HMS client
> add code to enabled/disable filtering at HMS client based on configuration
>
>
> Thanks,
>
> Na Li
>
>
Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on
server-side in addition to client-side
Posted by Na Li via Review Board <no...@reviews.apache.org>.
> On Dec. 19, 2018, 4:43 p.m., Karthik Manamcheri wrote:
> > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 5235 (patched)
> > <https://reviews.apache.org/r/69585/diff/2/?file=2114853#file2114853line5235>
> >
> > Can we make a helper fuction called "filterTablesIfEnabled"?
Done.
- Na
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review211436
-----------------------------------------------------------
On Dec. 19, 2018, 5:07 p.m., Na Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2018, 5:07 p.m.)
>
>
> Review request for hive, Adam Holley, Morio Ramdenbourg, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
>
>
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
>
>
> Diffs
> -----
>
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9eb1193a27120b5167f92daf67bf6a1c4e1d9927
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd
> standalone-metastore/metastore-server/pom.xml 895abfc423f00b121ee63e40904f5b3e57aea8ed
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96dcf62d3536cab2ce074d27a6225b2d3443
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/69585/diff/2/
>
>
> Testing
> -------
>
> Existing unit tests passed.
> add new unit tests for filtering at HMS server and HMS client
> add code to enabled/disable filtering at HMS client based on configuration
>
>
> Thanks,
>
> Na Li
>
>
Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on
server-side in addition to client-side
Posted by Karthik Manamcheri via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review211436
-----------------------------------------------------------
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
Lines 662 (patched)
<https://reviews.apache.org/r/69585/#comment296545>
We should rename this to "metastore.client.filter.enabled" and "metastore.server.filter.enabled"
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 2948 (patched)
<https://reviews.apache.org/r/69585/#comment296546>
getTableInternal also fires the PreReadTableEvent. I don't think we will need to fire this twice.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 2956 (patched)
<https://reviews.apache.org/r/69585/#comment296547>
All of this logic of endFunction seems to be inside getTableInternal. Why duplicate it here?
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 5235 (patched)
<https://reviews.apache.org/r/69585/#comment296548>
Can we make a helper fuction called "filterTablesIfEnabled"?
- Karthik Manamcheri
On Dec. 19, 2018, 6:57 a.m., Na Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2018, 6:57 a.m.)
>
>
> Review request for hive, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
>
>
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
>
>
> Diffs
> -----
>
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9eb1193a27120b5167f92daf67bf6a1c4e1d9927
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd
> standalone-metastore/metastore-server/pom.xml 895abfc423f00b121ee63e40904f5b3e57aea8ed
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96dcf62d3536cab2ce074d27a6225b2d3443
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/69585/diff/2/
>
>
> Testing
> -------
>
> Existing unit tests passed.
> add new unit tests for filtering at HMS server and HMS client
> add code to enabled/disable filtering at HMS client based on configuration
>
>
> Thanks,
>
> Na Li
>
>
Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on
server-side in addition to client-side
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/
-----------------------------------------------------------
(Updated Dec. 19, 2018, 6:57 a.m.)
Review request for hive, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
Bugs: HIVE-20776
https://issues.apache.org/jira/browse/HIVE-20776
Repository: hive-git
Description
-------
add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
Diffs
-----
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9eb1193a27120b5167f92daf67bf6a1c4e1d9927
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd
standalone-metastore/metastore-server/pom.xml 895abfc423f00b121ee63e40904f5b3e57aea8ed
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96dcf62d3536cab2ce074d27a6225b2d3443
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java PRE-CREATION
Diff: https://reviews.apache.org/r/69585/diff/2/
Testing (updated)
-------
Existing unit tests passed.
add new unit tests for filtering at HMS server and HMS client
add code to enabled/disable filtering at HMS client based on configuration
Thanks,
Na Li
Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on
server-side in addition to client-side
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/
-----------------------------------------------------------
(Updated Dec. 19, 2018, 6:56 a.m.)
Review request for hive, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
Bugs: HIVE-20776
https://issues.apache.org/jira/browse/HIVE-20776
Repository: hive-git
Description
-------
add filtering to read result at HMS server, so user cannot see metadata he/she has no privileges. Filtering is enabled/disabled based on configuration.
Diffs (updated)
-----
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9eb1193a27120b5167f92daf67bf6a1c4e1d9927
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd
standalone-metastore/metastore-server/pom.xml 895abfc423f00b121ee63e40904f5b3e57aea8ed
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0a1b96dcf62d3536cab2ce074d27a6225b2d3443
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java PRE-CREATION
Diff: https://reviews.apache.org/r/69585/diff/2/
Changes: https://reviews.apache.org/r/69585/diff/1-2/
Testing
-------
Existing unit tests passed.
Will add new unit tests for filtering at HMS server in next update of this jira
Will add code to enabled/disable filtering at HMS client based on configuration in next update of this jira
Thanks,
Na Li