You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (JIRA)" <ji...@apache.org> on 2012/07/05 19:34:35 UTC
[jira] [Created] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Kevin Wilfong created HIVE-3230:
-----------------------------------
Summary: Make logging of plan progress in HadoopJobExecHelper configurable
Key: HIVE-3230
URL: https://issues.apache.org/jira/browse/HIVE-3230
Project: Hive
Issue Type: Bug
Components: Logging
Affects Versions: 0.10.0
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
Priority: Minor
Fix For: 0.10.0
Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3230:
--------------------------------
Attachment: HIVE-3230.3.patch.txt
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417021#comment-13417021 ]
Hudson commented on HIVE-3230:
------------------------------
Integrated in Hive-trunk-h0.21 #1550 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1550/])
HIVE-3230 Make logging of plan progress in HadoopJobExecHelper configurable
(Kevin Wilfong via namit) (Revision 1362782)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362782
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml.template
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt, HIVE-3230.4.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411467#comment-13411467 ]
Edward Capriolo commented on HIVE-3230:
---------------------------------------
Will commit today if namit does not get to it.
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3230:
--------------------------------
Status: Patch Available (was: Open)
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3230:
--------------------------------
Attachment: HIVE-3230.4.patch.txt
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt, HIVE-3230.4.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3230:
--------------------------------
Attachment: HIVE-3230.2.patch.txt
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3230:
--------------------------------
Attachment: HIVE-3230.1.patch.txt
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412523#comment-13412523 ]
Namit Jain commented on HIVE-3230:
----------------------------------
running tests
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt, HIVE-3230.4.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411283#comment-13411283 ]
Namit Jain commented on HIVE-3230:
----------------------------------
+1
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3230:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Kevin
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3230.1.patch.txt, HIVE-3230.2.patch.txt, HIVE-3230.3.patch.txt, HIVE-3230.4.patch.txt
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3230) Make logging of plan progress in
HadoopJobExecHelper configurable
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407346#comment-13407346 ]
Kevin Wilfong commented on HIVE-3230:
-------------------------------------
diff here https://reviews.facebook.net/D4005
> Make logging of plan progress in HadoopJobExecHelper configurable
> -----------------------------------------------------------------
>
> Key: HIVE-3230
> URL: https://issues.apache.org/jira/browse/HIVE-3230
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 0.10.0
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Priority: Minor
> Fix For: 0.10.0
>
>
> Currently, by default, every second a job is run a massive JSON string containing the query plan, the tasks, and some counters is logged to the hive_job_log. For large, long running jobs that can easily reach gigabytes of data. This logging should be configurable as average user doesn't need this logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira