You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Adam Kramer (Created) (JIRA)" <ji...@apache.org> on 2011/12/24 07:36:30 UTC
[jira] [Created] (HIVE-2678) Programmatically limit CLI status
updates
Programmatically limit CLI status updates
-----------------------------------------
Key: HIVE-2678
URL: https://issues.apache.org/jira/browse/HIVE-2678
Project: Hive
Issue Type: Improvement
Components: CLI, Logging
Reporter: Adam Kramer
Provide a way to configure the frequency of Hive logging output, i.e., these:
2011-12-23 22:31:20,979 Stage-1 map = 16%, reduce = 0%, Cumulative CPU 567.27 sec
Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2678) Programmatically limit CLI status
updates
Posted by "Adam Kramer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259820#comment-13259820 ]
Adam Kramer commented on HIVE-2678:
-----------------------------------
No, that is not enough. For some queries ms makes sense, but for others % done makes sense. If a query increases in more than 1% done in the interval, the lack of output is maddening.
> Programmatically limit CLI status updates
> -----------------------------------------
>
> Key: HIVE-2678
> URL: https://issues.apache.org/jira/browse/HIVE-2678
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Logging
> Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%, reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2678) Programmatically limit CLI status
updates
Posted by "tamtam180 (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259808#comment-13259808 ]
tamtam180 commented on HIVE-2678:
---------------------------------
This issue seem to had resolved by HIVE-1950.
> Programmatically limit CLI status updates
> -----------------------------------------
>
> Key: HIVE-2678
> URL: https://issues.apache.org/jira/browse/HIVE-2678
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Logging
> Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%, reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2678) Programmatically limit CLI status
updates
Posted by "tamtam180 (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259556#comment-13259556 ]
tamtam180 commented on HIVE-2678:
---------------------------------
Hi,
You can use parameter "hive.exec.counters.pull.interval" after version-0.8.
Default value is 1000[ms].
> Programmatically limit CLI status updates
> -----------------------------------------
>
> Key: HIVE-2678
> URL: https://issues.apache.org/jira/browse/HIVE-2678
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Logging
> Reporter: Adam Kramer
>
> Provide a way to configure the frequency of Hive logging output, i.e., these:
> 2011-12-23 22:31:20,979 Stage-1 map = 16%, reduce = 0%, Cumulative CPU 567.27 sec
> Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).
> Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira