You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Paul Yang (JIRA)" <ji...@apache.org> on 2010/08/21 01:48:16 UTC
[jira] Created: (HIVE-1578) Add conf. variable for displaying link
to the task with most failures
Add conf. variable for displaying link to the task with most failures
---------------------------------------------------------------------
Key: HIVE-1578
URL: https://issues.apache.org/jira/browse/HIVE-1578
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Paul Yang
Assignee: Paul Yang
Fix For: 0.7.0
If a job fails, Hive currently displays a link to the task with the most number of failures, for easy access to the error logs. However, generating the link may require many RPC calls to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901207#action_12901207 ]
Paul Yang commented on HIVE-1578:
---------------------------------
@Carl
The message to the user about the conf var is a good idea. I can put info level logging statements, but I don't think it's possible to know the number of task completion events before retrieving them, so there won't be a % complete message.
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1578:
-----------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed. Thanks Paul.
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900948#action_12900948 ]
Carl Steinbach commented on HIVE-1578:
--------------------------------------
It would probably also be a good idea to add some INFO level logging statements to the loop in showJobDebugFailInfo() so that the user can roughy gauge the rate of progress.
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900946#action_12900946 ]
Carl Steinbach commented on HIVE-1578:
--------------------------------------
Hi Paul, before calling showJobDebugFailInfo() can you please print out a message telling the user that you're going to do this, that it may take a long time to complete, and that this feature can be disabled by setting the conf property {{hive.exec.show.job.failure.debug.info}}?
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link
to the task with most failures
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1578:
----------------------------
Attachment: HIVE-1578.1.patch
> Add conf. variable for displaying link to the task with most failures
> ---------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures, for easy access to the error logs. However, generating the link may require many RPC calls to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link
to the task with most failures
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1578:
----------------------------
Status: Patch Available (was: Open)
> Add conf. variable for displaying link to the task with most failures
> ---------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902774#action_12902774 ]
Carl Steinbach commented on HIVE-1578:
--------------------------------------
@Paul: I understand that showing % complete is not possible, but I do think it would help to log a message as each task completion event is retrieved so that the user sees progress is being made. The most frustrating thing about the current behavior is that the CLI appears to just freeze leaving the user with no idea what's going on.
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1578:
---------------------------------
Summary: Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures (was: Add conf. variable for displaying link to the task with most failures)
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. variable for displaying
link to the task with most failures
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900927#action_12900927 ]
Namit Jain commented on HIVE-1578:
----------------------------------
+1
> Add conf. variable for displaying link to the task with most failures
> ---------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1578) Add conf. property
hive.exec.show.job.failure.debug.info to enable/disable displaying link to
the task with most failures
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900991#action_12900991 ]
Joydeep Sen Sarma commented on HIVE-1578:
-----------------------------------------
looks like the CHANGES.txt message of this commit and the merge commit got mixed up
> Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1578) Add conf. variable for displaying link
to the task with most failures
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1578:
----------------------------
Description: If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs. (was: If a job fails, Hive currently displays a link to the task with the most number of failures, for easy access to the error logs. However, generating the link may require many RPC calls to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.)
> Add conf. variable for displaying link to the task with most failures
> ---------------------------------------------------------------------
>
> Key: HIVE-1578
> URL: https://issues.apache.org/jira/browse/HIVE-1578
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1578.1.patch
>
>
> If a job fails, Hive currently displays a link to the task with the most number of failures for easy access to the error logs. However, generating the link may require many RPC's to get all the task completion events, adding a delay of up to 30 minutes. This patch adds a configuration variable to control whether the link is generated. Turning off this feature would also disable automatic debugging tips generated by heuristics reading from the error logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.