You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "Flyangz (via GitHub)" <gi...@apache.org> on 2023/12/14 03:40:57 UTC

[PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Flyangz opened a new pull request, #5854:
URL: https://github.com/apache/kyuubi/pull/5854

   # :mag: Description
   ## Issue References ๐Ÿ”—
   <!-- Append the issue number after #. If there is no issue for you to link create one or -->
   <!-- If there are no issues to link, please provide details here. -->
   
   This pull request fixes #5832
   
   ## Describe Your Solution ๐Ÿ”ง
   
   We should always close writer regardless of whether OperationLog is initialized or not, to prevent other threads from continuing to write.
   
   
   ## Types of changes :bookmark:
   <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
   - [x] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing functionality to change)
   
   ## Test Plan ๐Ÿงช
   
   #### Behavior Without This Pull Request :coffin:
   
   
   #### Behavior With This Pull Request :tada:
   
   
   #### Related Unit Tests
   
   
   ---
   
   # Checklists
   ## ๐Ÿ“ Author Self Checklist
   <!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
   <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
   - [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
   - [x] I have performed a self-review
   - [ ] I have commented my code, particularly in hard-to-understand areas
   - [ ] I have made corresponding changes to the documentation
   - [x] My changes generate no new warnings
   - [ ] I have added tests that prove my fix is effective or that my feature works
   - [ ] New and existing unit tests pass locally with my changes
   - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
   
   ## ๐Ÿ“ Committer Pre-Merge Checklist
   
   - [ ] Pull request title is okay.
   - [ ] No license issues.
   - [ ] Milestone correctly set?
   - [ ] Test coverage is ok
   - [ ] Assignees are selected.
   - [ ] Minimum number of approvals
   - [ ] No changes are requested
   
   
   **Be nice. Be informative.**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Posted by "Flyangz (via GitHub)" <gi...@apache.org>.
Flyangz commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855232903

   @pan3793 Would you mind reviewing this pr or do you know who can help?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855360993

   ## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/5854?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Comparison is base [(`4c029f9`)](https://app.codecov.io/gh/apache/kyuubi/commit/4c029f991f3eb189912efe634da9472401da7ff0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 61.23% compared to head [(`2549928`)](https://app.codecov.io/gh/apache/kyuubi/pull/5854?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 61.21%.
   
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #5854      +/-   ##
   ============================================
   - Coverage     61.23%   61.21%   -0.03%     
     Complexity       23       23              
   ============================================
     Files           609      609              
     Lines         36281    36280       -1     
     Branches       4976     4976              
   ============================================
   - Hits          22217    22207      -10     
   - Misses        11669    11673       +4     
   - Partials       2395     2400       +5     
   ```
   
   
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/5854?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak [kyuubi]

Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855765747

   Looking forward to your feedback.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855261543

   LGTM overall.
   Could you attach some screenshots as evidence showing the fd leaks properly fixed by this PR? Eg. comparing opened fd numbers, or the server logs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1857367895

   Anyway, this fix looks good, let me merge it first, looking forward to your testing result once you have time @Flyangz 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #5854: [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
URL: https://github.com/apache/kyuubi/pull/5854


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Posted by "Flyangz (via GitHub)" <gi...@apache.org>.
Flyangz commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855293845

   > would you like to try 1.7.3 or 1.8.0? this issue looks like was solved by #5211
   
   @pan3793 My online test in issue #5832 had already included this #5211 patch, but the operation logs still leaked.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1857548999

   Merged to master/1.8/1.7


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [BUG] fix fd leak when closing KyuubiSession but OperationLog not initialized [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855263671

   would you like to try 1.7.3 or 1.8.0? this issue looks like was solved by https://github.com/apache/kyuubi/pull/5211


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak [kyuubi]

Posted by "Flyangz (via GitHub)" <gi...@apache.org>.
Flyangz commented on PR #5854:
URL: https://github.com/apache/kyuubi/pull/5854#issuecomment-1855520641

   Ok, I'll provide it as soon as possible. But I am busy with another more important project recently, and the upgrade of Kyuubi has been stoped. It will probably take a week or two before I can get back to it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org