[jira] [Work logged] (HIVE-25801) Custom queue settings is not honoured by Query based compaction StatsUpdater

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25801?focusedWorklogId=696591&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-696591 ]

ASF GitHub Bot logged work on HIVE-25801:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 15/Dec/21 12:43
            Start Date: 15/Dec/21 12:43
    Worklog Time Spent: 10m 
      Work Description: veghlaci05 opened a new pull request #2879:
URL: https://github.com/apache/hive/pull/2879


   ### What changes were proposed in this pull request?
   
   This PR fixes the issue that the statistics gathering for Query based compactions are not honouring the custom queue settings. It also deprecates the hive.mr.compactor.gather.stats setting which is honoured only by the MR based compactor, and introduces the hive.compactor.gather.stats setting, which is honoured by both MR and Query based compactors. The values of the two settings are combined with logical OR in case of MR based compaction. The stats gathering also changed since there is no need for column based statistics gathering in case of compaction, the basic statistics should be enough.
   
   
   ### Why are the changes needed?
   
   hive.compactor.job.queue config limits resources available for compaction, so users can limit the effects of compaction on the cluster. However this setting does not affect stats collection which uses Driver.
   HIVE-25595 is addressing the above issue for MR-based compaction. We need to incorporate the same thing for the Query-based compaction.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   It was tested using the already existing automated tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 696591)
    Remaining Estimate: 0h
            Time Spent: 10m

> Custom queue settings is not honoured by Query based compaction StatsUpdater
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-25801
>                 URL: https://issues.apache.org/jira/browse/HIVE-25801
>             Project: Hive
>          Issue Type: Bug
>            Reporter: László Végh
>            Assignee: László Végh
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{hive.compactor.job.queue}} config limits resources available for compaction, so users can limit the effects of compaction on the cluster. However this settings does not affect stats collection which uses Driver.
> HIVE-25595 is addressing the above issue for MR-based compaction. We need to incorporate the same thing for the Query-based compaction.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)