You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2009/06/17 16:24:08 UTC
[jira] Created: (HIVE-566) support for submitting jobs to Elastic
mapreduce cluster
support for submitting jobs to Elastic mapreduce cluster
--------------------------------------------------------
Key: HIVE-566
URL: https://issues.apache.org/jira/browse/HIVE-566
Project: Hadoop Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Joydeep Sen Sarma
EMR allows job submission to a cluster spawned by EMR via a webservices api or a ruby client (currently). See http://docs.amazonwebservices.com/ElasticMapReduce/2009-03-31/DeveloperGuide/.
Hive cli (or services) can submit jobs to EMR by using a command line just like MapRedTask currently does for local mode execution. It should be possible to configure hive to use this mode for job submission as well as configure any additional required EMR parameters.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-566) support for submitting jobs to Elastic
mapreduce cluster
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720690#action_12720690 ]
Joydeep Sen Sarma commented on HIVE-566:
----------------------------------------
as a side-note - we will not be able to create tables over hdfs (at this moment) when using EMR (at least going by the public documentation). all the tables (whether tmp or not) and target directories will have to be in s3. however hive will be able to use hdfs to store intermediate data when chaining map-reduce jobs.
> support for submitting jobs to Elastic mapreduce cluster
> --------------------------------------------------------
>
> Key: HIVE-566
> URL: https://issues.apache.org/jira/browse/HIVE-566
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
>
> EMR allows job submission to a cluster spawned by EMR via a webservices api or a ruby client (currently). See http://docs.amazonwebservices.com/ElasticMapReduce/2009-03-31/DeveloperGuide/.
> Hive cli (or services) can submit jobs to EMR by using a command line just like MapRedTask currently does for local mode execution. It should be possible to configure hive to use this mode for job submission as well as configure any additional required EMR parameters.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.