You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "Indhumathi27 (via GitHub)" <gi...@apache.org> on 2025/07/04 20:38:31 UTC
[PR] WIP: HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Indhumathi27 opened a new pull request, #5932:
URL: https://github.com/apache/hive/pull/5932
### What changes were proposed in this pull request?
The code was modified to check if a partition value is numeric using NumberUtils.isCreatable(partVal) before parsing it, instead of only filtering out the configured default partition name. This ensures that only valid numeric partition values are processed, and any non-numeric values are skipped.
### Why are the changes needed?
If the value of hive.exec.default.partition.name is set to one value during data insertion (for example, 'somevalue') and then changed to another value (for example, 'othervalue') during querying, Hive can encounter partition directories named with non-numeric strings. The old logic would try to parse these unexpected strings as numbers, causing a NumberFormatException. The new logic prevents this error by ignoring any partition value that isn't numeric, making the code more robust to configuration changes and avoiding runtime failures.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
New test case added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] WIP: HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5932:
URL: https://github.com/apache/hive/pull/5932#discussion_r2186141582
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -647,7 +647,7 @@ private static Range getRangePartitionColumn(PartitionIterable partitions, Strin
double max = Double.MIN_VALUE;
for (Partition partition : partitions) {
partVal = partition.getSpec().get(partColName);
- if (!partVal.equals(defaultPartName)) {
+ if (NumberUtils.isCreatable(partVal)) {
Review Comment:
can we merge `DECIMAL_TYPE` with `DOUBLE_TYPE` handling ? The use of BigDecimal followed by a cast to double seems unnecessary unless preserving precision is required
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] WIP: HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5932:
URL: https://github.com/apache/hive/pull/5932#issuecomment-3037332231
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5932) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5932&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5932&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5932&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5932&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5932&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5932)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ merged PR #5932:
URL: https://github.com/apache/hive/pull/5932
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5932:
URL: https://github.com/apache/hive/pull/5932#discussion_r2198902795
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -647,7 +647,7 @@ private static Range getRangePartitionColumn(PartitionIterable partitions, Strin
double max = Double.MIN_VALUE;
for (Partition partition : partitions) {
partVal = partition.getSpec().get(partColName);
- if (!partVal.equals(defaultPartName)) {
+ if (NumberUtils.isCreatable(partVal)) {
Review Comment:
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5932:
URL: https://github.com/apache/hive/pull/5932#issuecomment-3059996441
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5932) **Quality Gate passed**
Issues
 [22 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5932&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5932&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5932&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5932&metric=new_coverage&view=list)
 [1.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5932&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5932)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] WIP: HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "Indhumathi27 (via GitHub)" <gi...@apache.org>.
Indhumathi27 commented on code in PR #5932:
URL: https://github.com/apache/hive/pull/5932#discussion_r2198448853
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -647,7 +647,7 @@ private static Range getRangePartitionColumn(PartitionIterable partitions, Strin
double max = Double.MIN_VALUE;
for (Partition partition : partitions) {
partVal = partition.getSpec().get(partColName);
- if (!partVal.equals(defaultPartName)) {
+ if (NumberUtils.isCreatable(partVal)) {
Review Comment:
In this case, Directly converting to double should work. made changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "zhangbutao (via GitHub)" <gi...@apache.org>.
zhangbutao commented on PR #5932:
URL: https://github.com/apache/hive/pull/5932#issuecomment-3060851608
@deniskuzZ CI is green. Can we merge this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] WIP: HIVE-29069: Fix Query Failure while computing Range Partition Column stats on Numeric partition column types [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5932:
URL: https://github.com/apache/hive/pull/5932#discussion_r2186141582
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -647,7 +647,7 @@ private static Range getRangePartitionColumn(PartitionIterable partitions, Strin
double max = Double.MIN_VALUE;
for (Partition partition : partitions) {
partVal = partition.getSpec().get(partColName);
- if (!partVal.equals(defaultPartName)) {
+ if (NumberUtils.isCreatable(partVal)) {
Review Comment:
can we merge `DECIMAL_TYPE` with `DOUBLE_TYPE` handling ? The use of BigDecimal followed by a cast to double seems unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org