You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "dengzhhu653 (via GitHub)" <gi...@apache.org> on 2025/06/30 02:07:23 UTC
[PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
dengzhhu653 opened a new pull request, #5910:
URL: https://github.com/apache/hive/pull/5910
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176155484
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4836,74 +4827,6 @@ private void initializePartitionParameters(final Table tbl,
}
}
- private Partition add_partition_core(final RawStore ms,
Review Comment:
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174479410
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
don't we need https://github.com/apache/hive/pull/5910/files#diff-90c669c961ab250c01087a409b31126cce7ac0672c40f465e7ee262ed7bcdaddR4598 here as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174464799
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
should we deprecate an older API if not already? why don't we pass catName?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3021206520
Performance:
java -jar hmsbench.jar --savedata /tmp/benchdata --sanitize -N 1000 -N 10000 -o 0302bench_results_http_modified-1.csv -C -d testbench_http --params=100 -M 'addPartitions.*' -T 5 --runMode NONACID
After:
Operation Mean Med Min Max Err%
addPartitions.1000 1144 1130 936.0 1772 14.32
addPartitions.10000 9577 9363 9094 1.262e+04 6.880
Before:
Operation Mean Med Min Max Err%
addPartitions.1000 1558 1536 1362 2037 8.306
addPartitions.10000 1.402e+04 1.394e+04 1.257e+04 1.988e+04 7.596
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3021361693
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174510541
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
Review Comment:
Should it return anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174520359
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
Review Comment:
after the code move, you can simply pass `partitionsToAdd`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174532551
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4836,74 +4827,6 @@ private void initializePartitionParameters(final Table tbl,
}
}
- private Partition add_partition_core(final RawStore ms,
Review Comment:
was this a dead code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176158078
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
+ List<String> existedPartNames = new ArrayList<>();
+ for (Partition part : existedParts) {
+ String partName = Warehouse.makePartName(partitionKeys, part.getValues());
+ existedPartNames.add(partName);
+ existingParts.add(nameToPart.remove(partName));
+ }
+ if (!request.isIfNotExists() && !existedPartNames.isEmpty()) {
+ throw new AlreadyExistsException("Partition(s) already exist, partition name(s): " + existedPartNames);
}
+ List<Partition> partitionsToAdd = new ArrayList<>(nameToPart.values());
+ if (partitionsToAdd.isEmpty()) {
Review Comment:
Now the `nameToPart` filters all existing partitions, it contains the partitions to be added.
We should always have the list of partition names when calling `getPartitionsByNames`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176158742
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174503696
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4264,13 +4264,15 @@ public boolean equals(Object obj) {
}
}
- private List<Partition> add_partitions_core(final RawStore ms, String catName,
- String dbName, String tblName, List<Partition> parts, final boolean ifNotExists,
- boolean isSkipColSchemaForPartition, EnvironmentContext envContext) throws TException {
+ private List<Partition> add_partitions_core(final RawStore ms,
+ AddPartitionsRequest request) throws TException {
+ String catName = request.isSetCatName() ? request.getCatName() : getDefaultCatalog(conf);
Review Comment:
Should we move https://github.com/apache/hive/pull/5910/files#diff-90c669c961ab250c01087a409b31126cce7ac0672c40f465e7ee262ed7bcdaddR4598-R4599 here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3026026939
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176576498
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
Review Comment:
oh, i missed that we do the drop from the same list
````
existingParts.add(nameToPart.remove(partName));
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3017626691
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174464799
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
should we deprecate an older API if not already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176161301
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
Review Comment:
the `partitionsToAdd` only contains the partition that not exist in the HMS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174495278
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4292,17 +4294,16 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
tbl = ms.getTable(catName, dbName, tblName, null);
if (tbl == null) {
throw new InvalidObjectException("Unable to add partitions because "
- + TableName.getQualified(catName, dbName, tblName) +
- " does not exist");
+ + TableName.getQualified(catName, dbName, tblName) + " does not exist");
}
MTable mTable = getMS().ensureGetMTable(catName, dbName, tblName);
db = ms.getDatabase(catName, dbName);
- Set<PartValEqWrapperLite> partsToAdd = new HashSet<>(parts.size());
- List<Partition> partitionsToAdd = new ArrayList<>(parts.size());
+ Set<PartValEqWrapperLite> partsToAdd = new HashSet<>();
List<FieldSchema> partitionKeys = tbl.getPartitionKeys();
- for (final Partition part : parts) {
- if(isSkipColSchemaForPartition) {
+ Map<String, Partition> nameToPart = new LinkedHashMap<>();
+ for (final Partition part : request.getParts()) {
+ if(request.isSkipColumnSchemaForPartition()) {
Review Comment:
nit: space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3023155357
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 merged PR #5910:
URL: https://github.com/apache/hive/pull/5910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5910:
URL: https://github.com/apache/hive/pull/5910#issuecomment-3018319147
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5910&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5910&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5910&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5910)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174464799
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
should we deprecate an older API if not already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174516062
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
+ List<String> existedPartNames = new ArrayList<>();
+ for (Partition part : existedParts) {
+ String partName = Warehouse.makePartName(partitionKeys, part.getValues());
+ existedPartNames.add(partName);
+ existingParts.add(nameToPart.remove(partName));
+ }
+ if (!request.isIfNotExists() && !existedPartNames.isEmpty()) {
+ throw new AlreadyExistsException("Partition(s) already exist, partition name(s): " + existedPartNames);
}
+ List<Partition> partitionsToAdd = new ArrayList<>(nameToPart.values());
+ if (partitionsToAdd.isEmpty()) {
Review Comment:
why do we need to call `getPartitionsByNames` when `nameToPart` is empty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174510541
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
Review Comment:
Should it return anything, if not, maybe change the return type to void?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174479410
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
don't we need https://github.com/apache/hive/pull/5910/files#diff-90c669c961ab250c01087a409b31126cce7ac0672c40f465e7ee262ed7bcdaddR4598 here as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174516062
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
+ List<String> existedPartNames = new ArrayList<>();
+ for (Partition part : existedParts) {
+ String partName = Warehouse.makePartName(partitionKeys, part.getValues());
+ existedPartNames.add(partName);
+ existingParts.add(nameToPart.remove(partName));
+ }
+ if (!request.isIfNotExists() && !existedPartNames.isEmpty()) {
+ throw new AlreadyExistsException("Partition(s) already exist, partition name(s): " + existedPartNames);
}
+ List<Partition> partitionsToAdd = new ArrayList<>(nameToPart.values());
+ if (partitionsToAdd.isEmpty()) {
Review Comment:
why do we need to call `getPartitionsByNames` when `nameToPart` is empty? Maybe move this check higher
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176161301
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
Review Comment:
the `partitionsToAdd` only contains the partition that not exists in the HMS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2174464799
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4693,8 +4696,9 @@ public int add_partitions_pspec(final List<PartitionSpec> partSpecs)
partitionsToAdd.add(part);
partitionIterator.next();
}
- ret = add_partitions_core(getMS(), catName, dbName, tableName, partitionsToAdd,
- false, false, null).size();
+ AddPartitionsRequest request = new AddPartitionsRequest(dbName, tableName, partitionsToAdd, false);
Review Comment:
should we deprecate an older API if not already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29050: Optimize the partition validation in add_partitions [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5910:
URL: https://github.com/apache/hive/pull/5910#discussion_r2176576498
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -4319,19 +4320,30 @@ private List<Partition> add_partitions_core(final RawStore ms, String catName,
// incorrect, an exception will be thrown before the threads which create the partition
// folders are submitted. This way we can be sure that no partition and no partition
// folder will be created if the list contains an invalid partition.
- if (validatePartition(part, catName, tblName, dbName, partsToAdd, ms, ifNotExists,
- partitionKeys)) {
- partitionsToAdd.add(part);
- } else {
- existingParts.add(part);
- }
+ validatePartition(part, catName, tblName, dbName, partsToAdd);
+ nameToPart.put(Warehouse.makePartName(partitionKeys, part.getValues()), part);
}
- // Only authorize on newly created partitions
- if (!partitionsToAdd.isEmpty()) {
- firePreEvent(new PreAddPartitionEvent(tbl, partitionsToAdd, this));
+ List<Partition> existedParts =
+ ms.getPartitionsByNames(catName, dbName, tblName,
+ new GetPartitionsArgs.GetPartitionsArgsBuilder().partNames(new ArrayList<>(nameToPart.keySet())).build());
Review Comment:
oh, i missed that we alter the Map
````
existingParts.add(nameToPart.remove(partName));
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org