[jira] [Created] (SPARK-5086) Specifying the master instance type

["inkyu lee (JIRA)" <ji...@apache.org>]

inkyu lee created SPARK-5086:
--------------------------------

             Summary: Specifying the master instance type
                 Key: SPARK-5086
                 URL: https://issues.apache.org/jira/browse/SPARK-5086
             Project: Spark
          Issue Type: New Feature
          Components: EC2
    Affects Versions: 1.3.0
         Environment: aws ec2
            Reporter: inkyu lee
             Fix For: 1.3.0


I am using the m2.xlarge as master node, and r3.xlarge as slave nodes since they provide large memories and I do not need quad-core CPU in the master node.

But current ec2 script does not make this setting possible, since this script shares the same AMI. So, I got this error:

ERROR:boto:400 Bad Request
ERROR:boto:<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidParameterCombination</Code><Message>Virtualization type 'hvm' is required for instances of type 'r3.xlarge'.</Message></Error></Errors><RequestID>b02b5ce9-116a-40bc-b994-68d106195577</RequestID></Response>
Traceback (most recent call last):
  File "./spark_ec2.py", line 1176, in <module>
    main()
  File "./spark_ec2.py", line 1168, in main
    real_main()
  File "./spark_ec2.py", line 1017, in real_main
    (master_nodes, slave_nodes) = launch_cluster(conn, opts, cluster_name)
  File "./spark_ec2.py", line 554, in launch_cluster
    user_data=user_data_content)
  File "/Users/q/Downloads/spaark/spark-1.2.0-bin-hadoop2.4/ec2/lib/boto-2.34.0/boto/ec2/image.py", line 329, in run
    tenancy=tenancy, dry_run=dry_run)
  File "/Users/q/Downloads/spaark/spark-1.2.0-bin-hadoop2.4/ec2/lib/boto-2.34.0/boto/ec2/connection.py", line 974, in run_instances
    verb='POST')
  File "/Users/q/Downloads/spaark/spark-1.2.0-bin-hadoop2.4/ec2/lib/boto-2.34.0/boto/connection.py", line 1204, in get_object
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidParameterCombination</Code><Message>Virtualization type 'hvm' is required for instances of type 'r3.xlarge'.</Message></Error></Errors><RequestID>b02b5ce9-116a-40bc-b994-68d106195577</RequestID></Response>


So, If you agree with this, I will make a patch to make this possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org