You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Adesh Rao <ad...@gmail.com> on 2020/05/03 05:26:39 UTC
Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
Review request for hive and Sankar Hariappan.
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs
-----
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 7c4e129738
Diff: https://reviews.apache.org/r/72462/diff/1/
Testing
-------
Thanks,
Adesh Rao
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
> On May 14, 2020, 8:12 p.m., Sankar Hariappan wrote:
> > ql/src/test/queries/clientpositive/msck_repair_4.q
> > Lines 8 (patched)
> > <https://reviews.apache.org/r/72462/diff/1/?file=2229577#file2229577line8>
> >
> > Add a testcase with table path repairtable_n4 having upper case. You can achieve it by setting location in create table.
Done.
> On May 14, 2020, 8:12 p.m., Sankar Hariappan wrote:
> > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java
> > Lines 116 (patched)
> > <https://reviews.apache.org/r/72462/diff/1/?file=2229579#file2229579line116>
> >
> > Why do we need pathSet? Can we check if path == null instead?
Removed.
> On May 14, 2020, 8:12 p.m., Sankar Hariappan wrote:
> > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
> > Line 1441 (original), 1441 (patched)
> > <https://reviews.apache.org/r/72462/diff/1/?file=2229582#file2229582line1441>
> >
> > Even table name in directory can be any case. Should we use equalsIgnoresCase here?
Msck list all the directories under the actual table path, and then checks if the new child directories are valid partitions. Since all the partition directories are child of the actual table path, it won't matter if we are using equals/equalsIgnoreCase.
The second question which comes is, should we check for partitions under all paths that matches the actual table path ignoring the case. I don't think we should do that, because, then msck will have to check in too many combinations of base directory in hdfs (which will be ~2^length of table name).
> On May 14, 2020, 8:12 p.m., Sankar Hariappan wrote:
> > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
> > Line 1454 (original), 1456 (patched)
> > <https://reviews.apache.org/r/72462/diff/1/?file=2229582#file2229582line1456>
> >
> > Need to check how Hive treats ptn='A' and ptn='a' as ptn keys are lowercase but values can be any case.
Added a test case. These two values should be treated as different as they both are different strings.
- Adesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/#review220771
-----------------------------------------------------------
On May 15, 2020, 5:42 a.m., Adesh Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72462/
> -----------------------------------------------------------
>
> (Updated May 15, 2020, 5:42 a.m.)
>
>
> Review request for hive and Sankar Hariappan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
>
>
> Diffs
> -----
>
> ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
>
>
> Diff: https://reviews.apache.org/r/72462/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adesh Rao
>
>
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Sankar Hariappan <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/#review220771
-----------------------------------------------------------
ql/src/test/queries/clientpositive/msck_repair_4.q
Lines 8 (patched)
<https://reviews.apache.org/r/72462/#comment309467>
Add a testcase with table path repairtable_n4 having upper case. You can achieve it by setting location in create table.
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java
Lines 116 (patched)
<https://reviews.apache.org/r/72462/#comment309464>
Why do we need pathSet? Can we check if path == null instead?
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
Line 1441 (original), 1441 (patched)
<https://reviews.apache.org/r/72462/#comment309465>
Even table name in directory can be any case. Should we use equalsIgnoresCase here?
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
Line 1454 (original), 1456 (patched)
<https://reviews.apache.org/r/72462/#comment309466>
Need to check how Hive treats ptn='A' and ptn='a' as ptn keys are lowercase but values can be any case.
- Sankar Hariappan
On May 3, 2020, 5:26 a.m., Adesh Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72462/
> -----------------------------------------------------------
>
> (Updated May 3, 2020, 5:26 a.m.)
>
>
> Review request for hive and Sankar Hariappan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
>
>
> Diffs
> -----
>
> ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 7c4e129738
>
>
> Diff: https://reviews.apache.org/r/72462/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adesh Rao
>
>
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/#review220868
-----------------------------------------------------------
ql/src/test/results/clientnegative/msck_repair_6.q.out
Lines 30 (patched)
<https://reviews.apache.org/r/72462/#comment309587>
Most of the other ddl commands throw the same error code when finished.
Do we need to create a new MetastoreException subclass, and then use it to catch and throw new error code?
- Adesh Rao
On May 18, 2020, 10:43 a.m., Adesh Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72462/
> -----------------------------------------------------------
>
> (Updated May 18, 2020, 10:43 a.m.)
>
>
> Review request for hive and Sankar Hariappan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
>
>
> Diffs
> -----
>
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
> ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
>
>
> Diff: https://reviews.apache.org/r/72462/diff/5/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adesh Rao
>
>
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Sankar Hariappan <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/#review220823
-----------------------------------------------------------
ql/src/test/results/clientnegative/msck_repair_6.q.out
Lines 30 (patched)
<https://reviews.apache.org/r/72462/#comment309534>
Can we return more specific error code instead of 1?
ql/src/test/results/clientpositive/msck_repair_4.q.out
Lines 34 (patched)
<https://reviews.apache.org/r/72462/#comment309533>
Why the show partitions output is listed in the output file? We need to have it to validate this scenario.
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java
Lines 184 (patched)
<https://reviews.apache.org/r/72462/#comment309584>
Do we need this override as it just calls super method?
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java
Lines 415 (patched)
<https://reviews.apache.org/r/72462/#comment309585>
Shall log the table name as well.
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java
Line 405 (original), 419 (patched)
<https://reviews.apache.org/r/72462/#comment309586>
Should we also validate the case where old path(lower case) of partition is removed but added same path with different case? In this case, partition exist in both partitionsNotOnFs and partitionsNotInMs lists. Not sure, if it can cause any failures.
- Sankar Hariappan
On May 18, 2020, 10:43 a.m., Adesh Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72462/
> -----------------------------------------------------------
>
> (Updated May 18, 2020, 10:43 a.m.)
>
>
> Review request for hive and Sankar Hariappan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
>
>
> Diffs
> -----
>
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
> ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
>
>
> Diff: https://reviews.apache.org/r/72462/diff/5/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adesh Rao
>
>
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Sankar Hariappan <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/#review220910
-----------------------------------------------------------
Ship it!
Ship It!
- Sankar Hariappan
On May 27, 2020, 5:02 a.m., Adesh Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72462/
> -----------------------------------------------------------
>
> (Updated May 27, 2020, 5:02 a.m.)
>
>
> Review request for hive and Sankar Hariappan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
>
>
> Diffs
> -----
>
> itests/src/test/resources/testconfiguration.properties 92ae8c28e8
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
> ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
> ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
> ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
>
>
> Diff: https://reviews.apache.org/r/72462/diff/6/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adesh Rao
>
>
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
(Updated May 27, 2020, 5:02 a.m.)
Review request for hive and Sankar Hariappan.
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties 92ae8c28e8
ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
Diff: https://reviews.apache.org/r/72462/diff/6/
Changes: https://reviews.apache.org/r/72462/diff/5-6/
Testing
-------
Thanks,
Adesh Rao
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
(Updated May 18, 2020, 10:43 a.m.)
Review request for hive and Sankar Hariappan.
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs (updated)
-----
ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
Diff: https://reviews.apache.org/r/72462/diff/5/
Changes: https://reviews.apache.org/r/72462/diff/4-5/
Testing
-------
Thanks,
Adesh Rao
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
(Updated May 18, 2020, 8:40 a.m.)
Review request for hive and Sankar Hariappan.
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs (updated)
-----
ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java 7821f40a82
ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
Diff: https://reviews.apache.org/r/72462/diff/4/
Changes: https://reviews.apache.org/r/72462/diff/3-4/
Testing
-------
Thanks,
Adesh Rao
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
(Updated May 15, 2020, 12:28 p.m.)
Review request for hive and Sankar Hariappan.
Changes
-------
Validating that each partition should correspond to single path in fs
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs (updated)
-----
ql/src/test/queries/clientnegative/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientnegative/msck_repair_6.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientnegative/msck_repair_6.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
Diff: https://reviews.apache.org/r/72462/diff/3/
Changes: https://reviews.apache.org/r/72462/diff/2-3/
Testing
-------
Thanks,
Adesh Rao
Re: Review Request 72462: MSCK REPAIR cannot discover partitions with
upper case directory names
Posted by Adesh Rao <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72462/
-----------------------------------------------------------
(Updated May 15, 2020, 5:42 a.m.)
Review request for hive and Sankar Hariappan.
Repository: hive-git
Description
-------
The fix converts partition keys to lowercase present in hdfs directory, but store the hdfs directory as is for partition path.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/msck_repair_4.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_5.q PRE-CREATION
ql/src/test/queries/clientpositive/msck_repair_6.q PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_4.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_5.q.out PRE-CREATION
ql/src/test/results/clientpositive/msck_repair_6.q.out PRE-CREATION
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/CheckResult.java 5287f47e21
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java 6f4400a8ef
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/Msck.java f4e109d1b0
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java 92d10cd0e1
Diff: https://reviews.apache.org/r/72462/diff/2/
Changes: https://reviews.apache.org/r/72462/diff/1-2/
Testing
-------
Thanks,
Adesh Rao