You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2010/12/09 23:51:03 UTC
[jira] Created: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
change hive assumption that local mode mappers/reducers always run in same jvm
-------------------------------------------------------------------------------
Key: HIVE-1846
URL: https://issues.apache.org/jira/browse/HIVE-1846
Project: Hive
Issue Type: Bug
Reporter: Joydeep Sen Sarma
Assignee: Joydeep Sen Sarma
we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-1846.
----------------------------------
Resolution: Fixed
Assignee: Ramkumar Vadali (was: Joydeep Sen Sarma)
Opening a new ticket for the test failure.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Ramkumar Vadali
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.2.patch, HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma updated HIVE-1846:
------------------------------------
Assignee: Ramkumar Vadali (was: Joydeep Sen Sarma)
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Ramkumar Vadali
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma resolved HIVE-1846.
-------------------------------------
Resolution: Fixed
Fix Version/s: 0.7.0
committed - thanks Ram.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1846) change hive assumption that local
mode mappers/reducers always run in same jvm
Posted by "Bennie Schut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970118#action_12970118 ]
Bennie Schut commented on HIVE-1846:
------------------------------------
One of the problems I ran into on local mode was a missing plan. If this is solved perhaps it also solves part of HIVE-1675.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Ramkumar Vadali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramkumar Vadali updated HIVE-1846:
----------------------------------
Attachment: HIVE-1846.patch
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Ramkumar Vadali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramkumar Vadali updated HIVE-1846:
----------------------------------
Attachment: HIVE-1846.2.patch
Tested with a modified hadoop that runs child mappers in a separate process.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Ramkumar Vadali
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.2.patch, HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma resolved HIVE-1846.
-------------------------------------
Resolution: Fixed
committed the new patch. thanks Ram.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Ramkumar Vadali
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.2.patch, HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Ramkumar Vadali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramkumar Vadali reopened HIVE-1846:
-----------------------------------
There is a string comparison bug, will attach patch.
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Ramkumar Vadali
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HIVE-1846) change hive assumption that local mode
mappers/reducers always run in same jvm
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach reopened HIVE-1846:
----------------------------------
Assignee: Joydeep Sen Sarma (was: Ramkumar Vadali)
This patch is a causing a failure in TestNegativeCliDriver/dyn_part_empty.q
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/449/
@Joy: Can you please take a look?
> change hive assumption that local mode mappers/reducers always run in same jvm
> -------------------------------------------------------------------------------
>
> Key: HIVE-1846
> URL: https://issues.apache.org/jira/browse/HIVE-1846
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Fix For: 0.7.0
>
> Attachments: HIVE-1846.2.patch, HIVE-1846.patch
>
>
> we are trying out a version of hadoop local mode that runs multiple mappers/reducers by spawning jvm's for them. In this mode hive mappers fail in reading the plan file. it seems that we assume (in the setMapredWork call) that local mode mappers/reducers will run in the same jvm (we can cache the current plan in a global var and don't serialize to a path). this needs to get fixed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.