[jira] [Work logged] (HIVE-25970) Missing messages in HS2 operation logs

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25970?focusedWorklogId=731630&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731630 ]

ASF GitHub Bot logged work on HIVE-25970:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 23/Feb/22 15:13
            Start Date: 23/Feb/22 15:13
    Worklog Time Spent: 10m 
      Work Description: zabetak opened a new pull request #3048:
URL: https://github.com/apache/hive/pull/3048


   ### What changes were proposed in this pull request?
   
   Revert HIVE-22753 (commit 6a5c0cd04a2e88a545a96d10a942c86b2be18daa).
   
   ### Why are the changes needed?
   Preventing the creation of the appender (by returning null) leads to the message triggering the creation to be lost forever. Given that the memory leak that was observed in HIVE-22753 is no longer feasible with the fix for HIVE-24590 the changes in HIVE-22753 are redundant and problematic.
   
   ### Does this PR introduce _any_ user-facing change?
   Fixes a bug.
   
   ### How was this patch tested?
   `mvn test -Dtest=TestOperationLoggingAPIWithTez#testFetchResultsOfLogWithExecutionMode`
   
   I changed the property (https://github.com/apache/hive/commit/bc6ac75266aa360ce32aa975cf4f7b655cbfd22e) controlling how often appenders are closed to increase the likelihood of hitting the problem and run `TestOperationLoggingAPIWithTez` multiple times.
   
   Without the fix the test fails 80% of the time on my machine.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 731630)
    Remaining Estimate: 0h
            Time Spent: 10m

> Missing messages in HS2 operation logs
> --------------------------------------
>
>                 Key: HIVE-25970
>                 URL: https://issues.apache.org/jira/browse/HIVE-25970
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>            Reporter: Stamatis Zampetakis
>            Assignee: Stamatis Zampetakis
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> After HIVE-22753 & HIVE-24590, with some unlucky timing of events, operation log messages can get lost and never appear in the appropriate files.
> The changes in HIVE-22753 will prevent a {{HushableRandomAccessFileAppender}} from being created if the latter refers to a file that has been closed in the last second. Preventing the creation of the appender also means that the message which triggered the creation will be lost forever. In fact any message (for the same query) that comes in the interval of 1 second will be lost forever.
> Before HIVE-24590 the appender/file was closed only once (explicitly by HS2) and thus the problem may be very hard to notice in practice. However, with the arrival of HIVE-24590 appenders may close much more frequently (and not via HS2) making the issue reproducible rather easily. It suffices to set _hive.server2.operation.log.purgePolicy.timeToLive_ property very low and check the operation logs.
> The problem was discovered by investigating some intermittent failures in operation logging tests (e.g.,  TestOperationLoggingAPIWithTez).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)