You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Rajesh Balamohan <ra...@gmail.com> on 2016/06/02 00:05:30 UTC
Review Request 48159: HIVE-13901: Hivemetastore add partitions can be
slow depending on filesystems
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs
-----
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review136027
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2306)
<https://reviews.apache.org/r/48159/#comment201030>
In case the cluster is configured with StorageBasedAuth*, based on the listener implementations, it can have similar issues where it ends up scanning all dirs for permissions etc. Similar approach can be followed there as well, but out of scope of this JIRA. Will create a separate jira for it.
- Rajesh Balamohan
On June 3, 2016, 3:03 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 3, 2016, 3:03 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
> On June 14, 2016, 6:31 p.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 365
> > <https://reviews.apache.org/r/48159/diff/4-5/?file=1405612#file1405612line365>
> >
> > two nested synchornized-s on the same thing.
> > Also nit: is it possible to have an object to sync on, rather than syncing on something with global visiblity
removed a nesting. Retaining sync on HMSHandler.class, as threadPool itself is static.
> On June 14, 2016, 6:31 p.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2369
> > <https://reviews.apache.org/r/48159/diff/4-5/?file=1405612#file1405612line2369>
> >
> > hmm...
ThreadPool changes in add_partitions_pspec also got missed out in earlier patch. This was wrong patch which got uploaded. Uploading the correct one now.
- Rajesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review137561
-----------------------------------------------------------
On June 17, 2016, 5:56 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 17, 2016, 5:56 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cc95008
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c0827ea
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On June 14, 2016, 6:31 p.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 365
> > <https://reviews.apache.org/r/48159/diff/4-5/?file=1405612#file1405612line365>
> >
> > two nested synchornized-s on the same thing.
> > Also nit: is it possible to have an object to sync on, rather than syncing on something with global visiblity
>
> Rajesh Balamohan wrote:
> removed a nesting. Retaining sync on HMSHandler.class, as threadPool itself is static.
I was more concerned with visibility, rather than static. I.e. when SomeClass does synchronized (this) it can backfire if someone decides to save on Object-s and instead does SomeClass foo; synchronized (foo). I guess with .class it's much less likely, so it should be ok
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review137561
-----------------------------------------------------------
On June 17, 2016, 5:56 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 17, 2016, 5:56 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cc95008
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c0827ea
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review137561
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 365)
<https://reviews.apache.org/r/48159/#comment202705>
two nested synchornized-s on the same thing.
Also nit: is it possible to have an object to sync on, rather than syncing on something with global visiblity
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2366)
<https://reviews.apache.org/r/48159/#comment202706>
hmm...
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/48159/#comment202708>
looks like TP was removed from here. Intended?
- Sergey Shelukhin
On June 14, 2016, 5:06 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 14, 2016, 5:06 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 285caa3
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c0827ea
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated July 8, 2016, 3:52 a.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
The earlier patch caused test failures in TestHiveMetastore where it was trying to have duplicate partitions. In earlier patch, any entry in addedPartitions was getting nuked in the finally block, so even valid partitions were getting deleted.
Changes in latest patch:
1. Making use of Map<PartValEqWrapperLite, Boolean> for addedPartitions (in synchronized hashmap) to take care of the condition mentioned earlier.
2. Still adding entry to addPartitions isn't threadsafe, when something goes wrong between time of creating the directory and adding it to addedPartitions (e.g killing server). But that is something which was already existing and patch isn't trying to change the behaviour.
Ran the following for verification with latest patch.
mvn clean install -DskipTests && cd itests && mvn clean install -DskipTests && cd hive-unit && mvn test -Dtest=TestEmbeddedHiveMetaStore,TestRemoteHiveMetaStore,TestSetUGIOnBothClientServer,TestSetUGIOnOnlyClient,TestSetUGIOnOnlyServer
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b13fc65
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f45b90d
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated June 17, 2016, 5:56 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cc95008
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c0827ea
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated June 14, 2016, 5:06 a.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
1. HCatalog tests were failing as they ended up creating client/HMS too many times and threadPool was getting created every time. This caused rejection exceptions. FS threadpool created here can be static and be shared across. Made it static.
2. Renamed variable to "hive.metastore.fshandler.threads" as Ashutosh suggested.
3. Ran HCatalog tests locally to ensure it passes.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 285caa3
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c0827ea
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated June 3, 2016, 3:03 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated June 3, 2016, 1:40 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2357
> > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2357>
> >
> > cleanup code needs to be updated below. As far as I can tell, when this throws there can be completed operations yet to be added to the set, unless they are decisively terminated or allowed to finish
>
> Rajesh Balamohan wrote:
> Same approach is followed as in earlier case (where it was all or none gets updated). Plz let me know if i am missing anything here.
addedPartitions set is used to determine which directories to delete on clean up.
In the patch, the access to addedPartitions itself doesn't appear to be thread safe. If it was, there's a question of tasks that are somewhere between creating the directory and adding to the map.
Actually, I don't think addedPartitions.put inside the callable is thread safe.
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review136018
-----------------------------------------------------------
On June 3, 2016, 1:40 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 3, 2016, 1:40 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2350
> > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2350>
> >
> > is there the expectation that results are in the same order as request? should be relatively easy to achieve.
It is not expected that it has to be same order. But fixed it to be in the order in latest patch.
> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2356
> > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2356>
> >
> > terminate other tasks if possible?
Cancelling tasks on exception.
> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2357
> > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2357>
> >
> > cleanup code needs to be updated below. As far as I can tell, when this throws there can be completed operations yet to be added to the set, unless they are decisively terminated or allowed to finish
Same approach is followed as in earlier case (where it was all or none gets updated). Plz let me know if i am missing anything here.
- Rajesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review136018
-----------------------------------------------------------
On June 3, 2016, 1:40 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 3, 2016, 1:40 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 2357
> > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2357>
> >
> > cleanup code needs to be updated below. As far as I can tell, when this throws there can be completed operations yet to be added to the set, unless they are decisively terminated or allowed to finish
>
> Rajesh Balamohan wrote:
> Same approach is followed as in earlier case (where it was all or none gets updated). Plz let me know if i am missing anything here.
>
> Sergey Shelukhin wrote:
> addedPartitions set is used to determine which directories to delete on clean up.
> In the patch, the access to addedPartitions itself doesn't appear to be thread safe. If it was, there's a question of tasks that are somewhere between creating the directory and adding to the map.
> Actually, I don't think addedPartitions.put inside the callable is thread safe.
Agreed. Fixed this in latest patch. In corner case, it would end up invoking deleteDir which isn't created yet, which is not harmful.
- Rajesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review136018
-----------------------------------------------------------
On June 3, 2016, 3:03 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 3, 2016, 3:03 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/#review136018
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 269)
<https://reviews.apache.org/r/48159/#comment201017>
nit: private
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2347)
<https://reviews.apache.org/r/48159/#comment201018>
is there the expectation that results are in the same order as request? should be relatively easy to achieve.
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2353)
<https://reviews.apache.org/r/48159/#comment201019>
terminate other tasks if possible?
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2354)
<https://reviews.apache.org/r/48159/#comment201020>
cleanup code needs to be updated below. As far as I can tell, when this throws there can be completed operations yet to be added to the set, unless they are decisively terminated or allowed to finish
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2518)
<https://reviews.apache.org/r/48159/#comment201021>
same concerns as above
- Sergey Shelukhin
On June 3, 2016, 12:33 a.m., Rajesh Balamohan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48159/
> -----------------------------------------------------------
>
> (Updated June 3, 2016, 12:33 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-13901
> https://issues.apache.org/jira/browse/HIVE-13901
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ff5c858
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
>
> Diff: https://reviews.apache.org/r/48159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajesh Balamohan
>
>
Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions
can be slow depending on filesystems
Posted by Rajesh Balamohan <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48159/
-----------------------------------------------------------
(Updated June 3, 2016, 12:33 a.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
Missed out changes to HiveConf.
Bugs: HIVE-13901
https://issues.apache.org/jira/browse/HIVE-13901
Repository: hive-git
Description
-------
Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions).
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ff5c858
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e
Diff: https://reviews.apache.org/r/48159/diff/
Testing
-------
Thanks,
Rajesh Balamohan