You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Adam Kramer (Created) (JIRA)" <ji...@apache.org> on 2011/12/24 07:36:30 UTC

[jira] [Created] (HIVE-2678) Programmatically limit CLI status updates

Programmatically limit CLI status updates
-----------------------------------------

                 Key: HIVE-2678
                 URL: https://issues.apache.org/jira/browse/HIVE-2678
             Project: Hive
          Issue Type: Improvement
          Components: CLI, Logging
            Reporter: Adam Kramer


Provide a way to configure the frequency of Hive logging output, i.e., these:

2011-12-23 22:31:20,979 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 567.27 sec

Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).

Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-2678) Programmatically limit CLI status updates

Posted by "Adam Kramer (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259820#comment-13259820 ] 

Adam Kramer commented on HIVE-2678:
-----------------------------------

No, that is not enough. For some queries ms makes sense, but for others % done makes sense. If a query increases in more than 1% done in the interval, the lack of output is maddening.
                
> Programmatically limit CLI status updates
> -----------------------------------------
>
>                 Key: HIVE-2678
>                 URL: https://issues.apache.org/jira/browse/HIVE-2678
>             Project: Hive
>          Issue Type: Improvement
>          Components: CLI, Logging
>            Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-2678) Programmatically limit CLI status updates

Posted by "tamtam180 (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259808#comment-13259808 ] 

tamtam180 commented on HIVE-2678:
---------------------------------

This issue seem to had resolved by HIVE-1950.
                
> Programmatically limit CLI status updates
> -----------------------------------------
>
>                 Key: HIVE-2678
>                 URL: https://issues.apache.org/jira/browse/HIVE-2678
>             Project: Hive
>          Issue Type: Improvement
>          Components: CLI, Logging
>            Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-2678) Programmatically limit CLI status updates

Posted by "tamtam180 (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259556#comment-13259556 ] 

tamtam180 commented on HIVE-2678:
---------------------------------

Hi,
You can use parameter "hive.exec.counters.pull.interval" after version-0.8.
Default value is 1000[ms].

                
> Programmatically limit CLI status updates
> -----------------------------------------
>
>                 Key: HIVE-2678
>                 URL: https://issues.apache.org/jira/browse/HIVE-2678
>             Project: Hive
>          Issue Type: Improvement
>          Components: CLI, Logging
>            Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira