You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/04/13 23:31:50 UTC
[jira] Created: (HIVE-1305) add progress in join and groupby
add progress in join and groupby
--------------------------------
Key: HIVE-1305
URL: https://issues.apache.org/jira/browse/HIVE-1305
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Paul Yang
Fix For: 0.6.0
The operators join and groupby can consume a lot of rows before producing any output.
All operators which do not have a output for every input should report progress periodically.
Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1305) add progress in join and groupby
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886608#action_12886608 ]
He Yongqiang commented on HIVE-1305:
------------------------------------
Overall looks good to me.
minor comments:
1. in GroupByOp's flush,
countAfterReport = 0; should put in the beginning of the function?
2. in AbstractMapjoin
heartbeatInterval = HiveConf.getIntVar(hconf,
HiveConf.ConfVars.HIVESENDHEARTBEAT);
is not needed? because the parent common join op already has that.
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Attachments: hive.1305.1.patch, hive.1305.2.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1305) add progress in join and groupby
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain reassigned HIVE-1305:
--------------------------------
Assignee: Siying Dong (was: Paul Yang)
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1305) add progress in join and groupby
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-1305:
------------------------------
Attachment: hive.1305.3.patch
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Attachments: hive.1305.1.patch, hive.1305.2.patch, hive.1305.3.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1305) add progress in join and groupby
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang resolved HIVE-1305.
--------------------------------
Fix Version/s: 0.7.0
Resolution: Fixed
I just committed. Thanks Siying!
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Fix For: 0.7.0
>
> Attachments: hive.1305.1.patch, hive.1305.2.patch, hive.1305.3.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1305) add progress in join and groupby
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-1305:
------------------------------
Attachment: hive.1305.2.patch
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Attachments: hive.1305.1.patch, hive.1305.2.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1305) add progress in join and groupby
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886593#action_12886593 ]
He Yongqiang commented on HIVE-1305:
------------------------------------
will take a look.
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Attachments: hive.1305.1.patch, hive.1305.2.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1305) add progress in join and groupby
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-1305:
------------------------------
Attachment: hive.1305.1.patch
> add progress in join and groupby
> --------------------------------
>
> Key: HIVE-1305
> URL: https://issues.apache.org/jira/browse/HIVE-1305
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Siying Dong
> Attachments: hive.1305.1.patch
>
>
> The operators join and groupby can consume a lot of rows before producing any output.
> All operators which do not have a output for every input should report progress periodically.
> Currently, it is only being done for ScriptOperator and FilterOperator.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.