You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "deniskuzZ (via GitHub)" <gi...@apache.org> on 2026/10/15 19:03:24 UTC
[PR] HIVE-30183: LLAP: a reduce-side merge join skips the minimum reducers per executor [hive]
deniskuzZ opened a new pull request, #6883:
URL: https://github.com/apache/hive/pull/6883
### What changes were proposed in this pull request?
`LlapDecider.adjustAutoParallelism` applies the minimum reducers per executor to the reduce works of a reduce-side merge join (`MergeJoinWork`): its main work and each merged work.
### Why are the changes needed?
The floor (`hive.tez.llap.min.reducer.per.executor` x executors) keeps auto-reduce from shrinking a reducer to a few tasks on a large cluster, but it was applied only to a `ReduceWork`, so a merge join vertex was skipped. TPC-DS q51 at 10 TB on 24 LLAP daemons (288 executors): the FULL OUTER merge join ran 13 tasks while the reducers around it ran 253. The merge join's vertex takes its parallelism from the edges of its reduce works, so each of them gets the floor; their edge properties are the ones `GenTezWork` reconnects to the merge join.
### Does this PR introduce _any_ user-facing change?
Reduce-side merge joins under LLAP get at least `hive.tez.llap.min.reducer.per.executor` x executors tasks, as other auto-reduced reducers do.
### How was this patch tested?
`TestLlapDecider`: on a 288-executor cluster, the main and merged reduce works of a merge join and their edges get the floor of 96, as a lone reduce work does; it fails without the change (13).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30183: LLAP: a reduce-side merge join skips the minimum reducers per executor [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6883:
URL: https://github.com/apache/hive/pull/6883#issuecomment-6233279403
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883) **Quality Gate passed**
Issues
 [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30183: LLAP: a reduce-side merge join skips the minimum reducers per executor [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6883:
URL: https://github.com/apache/hive/pull/6883#issuecomment-6202171605
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30183: LLAP: a reduce-side merge join skips the minimum reducers per executor [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6883:
URL: https://github.com/apache/hive/pull/6883#issuecomment-6229088590
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883) **Quality Gate passed**
Issues
 [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-30183: LLAP: a reduce-side merge join skips the minimum reducers per executor [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #6883:
URL: https://github.com/apache/hive/pull/6883#issuecomment-6225950918
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883) **Quality Gate passed**
Issues
 [40 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6883&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=6883&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=6883&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=6883)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org