You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/09/08 00:16:33 UTC
[jira] Created: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
ScriptOperator's AutoProgressor can lead to an infinite loop
------------------------------------------------------------
Key: HIVE-1617
URL: https://issues.apache.org/jira/browse/HIVE-1617
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Namit Jain
Fix For: 0.7.0
In the default settings, the auto progressor can result in a infinite loop.
There should be another configurable parameter which stops the auto progress if the script has not made an progress.
The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910386#action_12910386 ]
Namit Jain commented on HIVE-1617:
----------------------------------
Can you add a negative testcase - which times out
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910419#action_12910419 ]
Namit Jain commented on HIVE-1617:
----------------------------------
Otherwise it looks good
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang reassigned HIVE-1617:
-------------------------------
Assignee: Paul Yang
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910903#action_12910903 ]
Namit Jain commented on HIVE-1617:
----------------------------------
Committed. Thanks Paul
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1617:
-----------------------------
Status: Open (was: Patch Available)
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang resolved HIVE-1617.
-----------------------------
Resolution: Fixed
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1617:
----------------------------
Status: Patch Available (was: Open)
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910839#action_12910839 ]
Paul Yang commented on HIVE-1617:
---------------------------------
I've tried creating a test for this but it seems like for local mode, the task timeouts don't work. Unless we change how hadoop's local mode works, we might have to go without a test case.
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1617) ScriptOperator's AutoProgressor can
lead to an infinite loop
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1617:
----------------------------
Attachment: HIVE-1617.1.patch
This patch can be tested with the following commands:
set hive.auto.progress.timeout=1200;
set hive.exec.script.allow.partial.consumption=true;
set hive.exec.mode.local.auto=false;
set hive.script.auto.progress=true;
select transform(*) using 'sleep 3600' from src;
The logs should show reporter messages at regular intervals, up to the 20 minute mark. Then the task should time out.
> ScriptOperator's AutoProgressor can lead to an infinite loop
> ------------------------------------------------------------
>
> Key: HIVE-1617
> URL: https://issues.apache.org/jira/browse/HIVE-1617
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.7.0
>
> Attachments: HIVE-1617.1.patch
>
>
> In the default settings, the auto progressor can result in a infinite loop.
> There should be another configurable parameter which stops the auto progress if the script has not made an progress.
> The default can be an hour or so - this way we will not get indefinitely stuck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.