You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Gunther Hagleitner (JIRA)" <ji...@apache.org> on 2012/11/20 23:54:58 UTC

[jira] [Created] (HIVE-3726) History file closed in finalize method

Gunther Hagleitner created HIVE-3726:
----------------------------------------

             Summary: History file closed in finalize method
                 Key: HIVE-3726
                 URL: https://issues.apache.org/jira/browse/HIVE-3726
             Project: Hive
          Issue Type: Bug
            Reporter: Gunther Hagleitner
            Assignee: Gunther Hagleitner


TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3726) History file closed in finalize method

Posted by "Gunther Hagleitner (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gunther Hagleitner updated HIVE-3726:
-------------------------------------

    Status: Patch Available  (was: Open)
    
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3726) History file closed in finalize method

Posted by "Gunther Hagleitner (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gunther Hagleitner updated HIVE-3726:
-------------------------------------

    Attachment: HIVE-3726.2-r1411423.patch
    
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3726) History file closed in finalize method

Posted by "Gunther Hagleitner (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505047#comment-13505047 ] 

Gunther Hagleitner commented on HIVE-3726:
------------------------------------------

Good points. I'll look into it.
                
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.9.0, 0.10.0
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3726) History file closed in finalize method

Posted by "Gunther Hagleitner (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gunther Hagleitner updated HIVE-3726:
-------------------------------------

    Attachment: HIVE-3736.1-r1411423.patch

creates explicit close method for history and calls it from sessionstate
                
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3726) History file closed in finalize method

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Chauhan updated HIVE-3726:
-----------------------------------

    Affects Version/s: 0.10.0
                       0.9.0
               Status: Open  (was: Patch Available)
    
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.9.0, 0.10.0
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3726) History file closed in finalize method

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504178#comment-13504178 ] 

Ashutosh Chauhan commented on HIVE-3726:
----------------------------------------

Couple of comments:

* HiveServer uses TmpOutputFile, which it deletes (now via session.close()), but never closes the input stream which it opens on that file in readResults() and setupSessionIO() methods. This will result in leaking of file-descriptors. HiveServer2 has copy-pasted this code, so this leak occurs there too.

* I am not sure about changes in SessionState per task. With your patch, each task will close the {{SessionState}} when it finishes. A query may have multiple tasks, so this implies every task will create SessionState when it begin executing, which seems counter-intuitive, since there should be one sessionstate object across all tasks of the query which was the case earlier too, isnt it?
                
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3726) History file closed in finalize method

Posted by "Gunther Hagleitner (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502415#comment-13502415 ] 

Gunther Hagleitner commented on HIVE-3726:
------------------------------------------

Updated patch uses a new SessionState per task (parallel execution), so they don't step on one another.

                
> History file closed in finalize method
> --------------------------------------
>
>                 Key: HIVE-3726
>                 URL: https://issues.apache.org/jira/browse/HIVE-3726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch
>
>
> TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira