You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by sa...@eucalyptus.com on 2011/02/24 07:56:10 UTC
initial eucalyptus support
Hello,
This is the initial patch to support Eucalyptus as one of Deltacloud Clouds. The eucalyptus_driver.rb was forked from ec2_driver.rb.
The driver now supports all functions supported by EC2 driver, except for load balancer. To be more specific, all "core" deltacloud APIs
(http://incubator.apache.org/deltacloud/api.html) are supported, and additional APIs supported include Buckets (blobs), Keys, StorageVolume,
and StorageSnapshot. We found there was a bug with detroy_storage_snapshot in server.rb and fixed it as well. The driver was tested with
Eucalyptus 2.0.3.
There are two issues with current AWS gem, and we pulled the request to fix them:
* In AWS gem, a bucket is always named in virtual hosted-style request (e.g., http://yourbucket.s3.amazonaws.com/yourobject),
while many Eucalyptus installations doesn't use that style. Because naming a bucket with URL path (e.g., http://yourbucket.s3.amazonaws.com/yourobject)
universally work, we patched AWS to use URL path.
* ec2.create_volume had an issue with Eucalyptus backend and patched to resolve it.
Thanks and let me know if the patch has any issues. Our next milestone is to write cucumber tests.
Sang-min
RE: [PATCH] initial eucalyptus support
Posted by Sang-Min Park <sa...@eucalyptus.com>.
The pull request to aws gem was accepted and released as aws-2.4.3.
So the functions listed in the previous email will all work fine.
Sang-min
-----Original Message-----
From: sang-min.park@eucalyptus.com [mailto:sang-min.park@eucalyptus.com]
Sent: Wednesday, February 23, 2011 10:56 PM
To: deltacloud-dev@incubator.apache.org
Cc: sang-min.park@eucalyptus.com; Sang-Min Park
Subject: [PATCH] initial eucalyptus support
From: Sang-Min Park <sp...@eucalyptus.com>
---
server/config/drivers.yaml | 4 +
.../drivers/eucalyptus/eucalyptus_driver.rb | 640
++++++++++++++++++++
server/server.rb | 4 +-
3 files changed, 646 insertions(+), 2 deletions(-) create mode 100644
server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
diff --git a/server/config/drivers.yaml b/server/config/drivers.yaml index
f28234c..e21ef47 100644
--- a/server/config/drivers.yaml
+++ b/server/config/drivers.yaml
@@ -28,6 +28,10 @@
:name: Rackspace
:azure:
:name: Azure
+:eucalyptus:
+ :name: Eucalyptus
+ :username: Access Key ID
+ :password: Secret Access Key
:ec2:
:entrypoints:
s3:
diff --git a/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
b/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
new file mode 100644
index 0000000..1b88bca
--- /dev/null
+++ b/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
@@ -0,0 +1,640 @@
+# Copyright (C) 2009-2011 Eucalyptus Systems, Inc.
+#
+# Copyright (C) 2009, 2010 Red Hat, Inc.
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more #
+contributor license agreements. See the NOTICE file distributed with #
+this work for additional information regarding copyright ownership.
+The # ASF licenses this file to you under the Apache License, Version
+2.0 (the # "License"); you may not use this file except in compliance
+with the # License. You may obtain a copy of the License at #
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software #
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+implied. See the # License for the specific language governing
+permissions and limitations # under the License.
+#
+
+require 'deltacloud/base_driver'
+require 'aws'
+
+class Instance
+ attr_accessor :keyname
+ attr_accessor :authn_error
+
+ def authn_feature_failed?
+ return true unless authn_error.nil?
+ end
+
+end
+
+module Deltacloud
+ module Drivers
+ module Eucalyptus
+ class EucalyptusDriver < Deltacloud::BaseDriver
+
+ def supported_collections
+ DEFAULT_COLLECTIONS + [ :keys, :buckets ]
+ end
+
+ feature :instances, :user_data
+ feature :instances, :authentication_key
+ feature :instances, :security_group
+ feature :instances, :instance_count
+ feature :images, :owner_id
+ feature :buckets, :bucket_location
+ #feature :instances, :register_to_load_balancer # not
+ supported by Eucalyptus 2.0.3
+
+ DEFAULT_REGION = 'us-east-1'
+
+ define_hardware_profile('m1.small') do
+ cpu 1
+ memory 128
+ storage 2
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('c1.medium') do
+ cpu 1
+ memory 256
+ storage 5
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('m1.large') do
+ cpu 2
+ memory 512
+ storage 10
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('m1.xlarge') do
+ cpu 2
+ memory 1024
+ storage 20
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('c1.xlarge') do
+ cpu 4
+ memory 2048
+ storage 20
+ architecture 'x86_64'
+ end
+
+ define_instance_states do
+ start.to( :pending ) .automatically
+ pending.to( :running ) .automatically
+ pending.to( :stopping ) .on( :stop )
+ pending.to( :stopped ) .automatically
+ stopped.to( :running ) .on( :start )
+ running.to( :running ) .on( :reboot )
+ running.to( :stopping ) .on( :stop )
+ shutting_down.to( :stopped ) .automatically
+ stopped.to( :finish ) .automatically
+ end
+
+ def images(credentials, opts={})
+ ec2 = new_client(credentials)
+ img_arr = []
+ opts ||= {}
+ if opts[:id]
+ safely do
+ img_arr = ec2.describe_images(opts[:id]).collect do |image|
+ convert_image(image)
+ end
+ end
+ return img_arr
+ end
+ owner_id = opts[:owner_id] || "self"
+ safely do
+ img_arr = ec2.describe_images_by_owner(owner_id).collect do
|image|
+ convert_image(image)
+ end
+ end
+ #img_arr = filter_on( img_arr, :architecture, opts )
+ img_arr.sort_by { |e| [e.owner_id, e.name] }
+ end
+
+ def realms(credentials, opts={})
+ ec2 = new_client(credentials)
+ zone_id = opts ? opts[:id] : nil
+ safely do
+ return ec2.describe_availability_zones(zone_id).collect do
|realm|
+ convert_realm(realm)
+ end
+ end
+ end
+
+ def instances(credentials, opts={})
+ ec2 = new_client(credentials)
+ inst_arr = []
+ safely do
+ inst_arr = ec2.describe_instances.collect do |instance|
+ convert_instance(instance) if instance
+ end.flatten
+ end
+ filter_on( inst_arr, :state, opts )
+ end
+
+ def create_instance(credentials, image_id, opts={})
+ ec2 = new_client(credentials)
+ instance_options = {}
+ instance_options.merge!(:user_data => opts[:user_data]) if
opts[:user_data]
+ instance_options.merge!(:key_name => opts[:keyname]) if
opts[:keyname]
+ instance_options.merge!(:availability_zone => opts[:realm_id])
if opts[:realm_id]
+ instance_options.merge!(:instance_type => opts[:hwp_id]) if
opts[:hwp_id] && opts[:hwp_id].length > 0
+ instance_options.merge!(:group_ids => opts[:security_group]) if
opts[:security_group]
+ instance_options.merge!(
+ :min_count => opts[:instance_count],
+ :max_count => opts[:instance_count]
+ ) if opts[:instance_count] and opts[:instance_count].length!=0
+ safely do
+ new_instance =
convert_instance(ec2.launch_instances(image_id, instance_options).first)
+ # TODO: Rework this to use client_id for name instead of tag
+ # Tags should be keept as an optional feature
+ #if opts[:name]
+ # tag_instance(credentials, new_instance, opts[:name])
+ #end
+ # Register Instance to Load Balancer if load_balancer_id is
set.
+ # This parameter is a feature parameter
+ #if opts[:load_balancer_id] and
opts[:load_balancer_id].length>0
+ # elb = new_client(credentials, :elb)
+ #
elb.register_instances_with_load_balancer(opts[:load_balancer_id],
[new_instance.id])
+ #end
+ new_instance
+ end
+ end
+
+ def run_on_instance(credentials, opts={})
+ target = instance(credentials, :id => opts[:id])
+ param = {}
+ param[:credentials] = {
+ :username => 'root', # Default for EC2 Linux instances
+ }
+ param[:port] = opts[:port] || '22'
+ param[:ip] = target.public_addresses
+ param[:private_key] = (opts[:private_key].length > 1) ?
opts[:private_key] : nil
+ safely do
+ Deltacloud::Runner.execute(opts[:cmd], param)
+ end
+ end
+
+ def reboot_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ if ec2.reboot_instances([instance_id])
+ instance(credentials, instance_id)
+ else
+ raise Deltacloud::BackendError.new(500, "Instance", "Instance
reboot failed", "")
+ end
+ end
+
+ def destroy_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ instance_id = instance_id
+ if ec2.terminate_instances([instance_id])
+ #untag_instance(credentials, instance_id)
+ instance(credentials, instance_id)
+ else
+ raise Deltacloud::BackendError.new(500, "Instance", "Instance
cannot be terminated", "")
+ end
+ end
+
+ alias :stop_instance :destroy_instance
+
+ def keys(credentials, opts={})
+ ec2 = new_client(credentials)
+ opts ||= {}
+ safely do
+ ec2.describe_key_pairs(opts[:id] || nil).collect do |key|
+ convert_key(key)
+ end
+ end
+ end
+
+ def create_key(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_key(ec2.create_key_pair(opts[:key_name]))
+ end
+ end
+
+ def destroy_key(credentials, opts={})
+ ec2 = new_client(credentials)
+ original_key = key(credentials, opts)
+ safely do
+ ec2.delete_key_pair(original_key.id)
+ original_key= original_key.state = "DELETED"
+ end
+ original_key
+ end
+
+ def load_balancer(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+ def load_balancers(credentials, opts=nil)
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+ def create_load_balancer(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+ def destroy_load_balancer(credentials, id)
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+ def lb_register_instance(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+ def lb_unregister_instance(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer",
"Loadbalancer not supported yet", "")
+ end
+
+
+ def buckets(credentials, opts={})
+ buckets = []
+ safely do
+ s3_client = new_client(credentials, :s3)
+ bucket_list = s3_client.buckets
+
+ bucket_list.each do |current|
+ buckets << convert_bucket(current)
+ end
+ end
+ buckets
+ end
+
+ def create_bucket(credentials, name, opts={})
+ bucket = nil
+ safely do
+ s3_client = new_client(credentials, :s3)
+ bucket_location = opts['location']
+ if bucket_location
+ bucket = Aws::S3::Bucket.create(s3_client, name, true, nil,
:location => bucket_location)
+ else
+ bucket = Aws::S3::Bucket.create(s3_client, name, true)
+ end
+ end
+ convert_bucket(bucket)
+ end
+
+ def delete_bucket(credentials, name, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.delete_bucket(name)
+ end
+ end
+
+ def blobs(credentials, opts = {})
+ s3_client = new_client(credentials, :s3)
+ blobs = []
+ safely do
+ s3_bucket = s3_client.bucket(opts['bucket'])
+ s3_bucket.keys({}, true).each do |s3_object|
+ blobs << convert_object(s3_object)
+ end
+ end
+ blobs = filter_on(blobs, :id, opts)
+ blobs
+ end
+
+ #--
+ # Create Blob
+ #--
+ def create_blob(credentials, bucket_id, blob_id, data = nil, opts
= {})
+ s3_client = new_client(credentials, :s3)
+ #data is a construct with the temporary file created by server
@.tempfile
+ #also file[:type] will give us the content-type
+ res = nil
+ # File stream needs to be reopened in binary mode for whatever
reason
+ file = File::open(data[:tempfile].path, 'rb')
+ #insert ec2-specific header for user metadata ...
x-amz-meta-KEY = VALUE
+ opts.gsub_keys('HTTP_X_Deltacloud_Blobmeta_', 'x-amz-meta-')
+ opts["Content-Type"] = data[:type]
+ safely do
+ res = s3_client.interface.put(bucket_id,
+ blob_id,
+ file,
+ opts)
+ end
+ #create a new Blob object and return that
+ Blob.new( { :id => blob_id,
+ :bucket => bucket_id,
+ :content_length => data[:tempfile].length,
+ :content_type => data[:type],
+ :last_modified => '',
+ :user_metadata => opts.select{|k,v|
k.match(/^x-amz-meta-/i)}
+ }
+ )
+ end
+
+ #--
+ # Delete Blob
+ #--
+ def delete_blob(credentials, bucket_id, blob_id, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.delete(bucket_id, blob_id)
+ end
+ end
+
+
+ def blob_data(credentials, bucket_id, blob_id, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.get(bucket_id, blob_id) do |chunk|
+ yield chunk
+ end
+ end
+ end
+
+ def storage_volumes(credentials, opts={})
+ ec2 = new_client( credentials )
+ volume_list = (opts and opts[:id]) ? opts[:id] : nil
+ safely do
+ ec2.describe_volumes(volume_list).collect do |volume|
+ convert_volume(volume)
+ end
+ end
+ end
+
+ def create_storage_volume(credentials, opts=nil)
+ ec2 = new_client(credentials)
+ opts ||= {}
+ opts[:snapshot_id] ||= ""# -> this causes signature validation
error with Euca backend
+ opts[:capacity] ||= "1"
+ opts[:realm_id] ||= realms(credentials).first.id
+ safely do
+ convert_volume(ec2.create_volume(opts[:snapshot_id],
+ opts[:capacity], opts[:realm_id]))
+
+ end
+ end
+
+ def destroy_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ unless ec2.delete_volume(opts[:id])
+ raise Deltacloud::BackendError.new(500, "StorageVolume",
"Cannot delete storage volume")
+ end
+ storage_volume(credentials, opts[:id])
+ end
+ end
+
+ def attach_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_volume(ec2.attach_volume(opts[:id],
opts[:instance_id], opts[:device]))
+ end
+ end
+
+ def detach_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_volume(ec2.detach_volume(opts[:id],
opts[:instance_id], opts[:device], true))
+ end
+ end
+
+ def storage_snapshots(credentials, opts={})
+ ec2 = new_client(credentials)
+ snapshot_list = (opts and opts[:id]) ? opts[:id] : []
+ safely do
+ ec2.describe_snapshots(snapshot_list).collect do |snapshot|
+ convert_snapshot(snapshot)
+ end
+ end
+ end
+
+ def create_storage_snapshot(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_snapshot(ec2.try_create_snapshot(opts[:volume_id]))
+ end
+ end
+
+ def destroy_storage_snapshot(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ unless ec2.delete_snapshot(opts[:id])
+ raise Deltacloud::BackendError.new(500, "StorageSnapshot",
"Cannot destroy this snapshot")
+ end
+ end
+ end
+
+ def valid_credentials?(credentials)
+ retval = true
+ begin
+ realms(credentials)
+ rescue Deltacloud::BackendError
+ retval = false
+ end
+ retval
+ end
+
+ private
+
+ def new_client(credentials, type = :ec2)
+ klass = case type
+ when :elb then Aws::Elb
+ when :ec2 then Aws::Ec2
+ when :s3 then Aws::S3
+ end
+ klass.new(credentials.user, credentials.password,
+eucalyptus_endpoint) #:server => endpoint_for_service(type))
+ end
+
+ def eucalyptus_endpoint
+ endpoint = (Thread.current[:provider] || ENV['API_PROVIDER'])
+ #parse endpoint string into server, port, service, and protocol
+ if endpoint
+ {:server => URI.parse(endpoint).host, :port =>
URI.parse(endpoint).port, :service => URI.parse(endpoint).path, :protocol
=> URI.parse(endpoint).scheme}
+ else
+ { } #EC2_URL env variable will be used by AWS
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def tag_instance(credentials, instance, name)
+ ec2 = new_client(credentials)
+ safely do
+ ec2.create_tag(instance.id, 'name', name)
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def untag_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ safely do
+ ec2.delete_tag(instance_id, 'name')
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def delete_unused_tags(credentials, inst_ids)
+ ec2 = new_client(credentials)
+ tags = []
+ safely do
+ tags = ec2.describe_tags('Filter.1.Name' => 'resource-type',
'Filter.1.Value' => 'instance')
+ tags.collect! { |t| t[:aws_resource_id] }
+ inst_ids.each do |inst_id|
+ unless tags.include?(inst_id)
+ ec2.delete_tag(inst_id, 'name')
+ end
+ end
+ end
+ end
+
+ def convert_bucket(s3_bucket)
+ #get blob list:
+ blob_list = []
+ s3_bucket.keys.each do |s3_object|
+ blob_list << s3_object.name
+ end
+
+ #can use AWS::S3::Owner.current.display_name or current.id
+ Bucket.new(
+ :id => s3_bucket.name,
+ :name => s3_bucket.name,
+ :size => s3_bucket.keys.length,
+ :blob_list => blob_list
+ )
+ end
+
+ def convert_object(s3_object)
+ Blob.new(
+ :id => s3_object.name,
+ :bucket => s3_object.bucket.name.to_s,
+ :content_length => s3_object.headers['content-length'],
+ :content_type => s3_object.headers['content-type'],
+ :last_modified => s3_object.last_modified,
+ :user_metadata => s3_object.meta_headers
+ )
+ end
+
+ def convert_realm(realm)
+ Realm.new(
+ :id => realm[:zone_name],
+ :name => realm[:zone_name],
+ :state => realm[:zone_state],
+ :limit => realm[:zone_state].eql?('available') ? :unlimited :
0
+ )
+ end
+
+ def convert_image(image)
+ # There is not support for 'name' for now
+ Image.new(
+ :id => image[:aws_id],
+ :name => image[:aws_name] || image[:aws_id],
+ :description => image[:aws_description] ||
image[:aws_location],
+ :owner_id => image[:aws_owner],
+ :architecture => image[:aws_architecture],
+ :state => image[:state]
+ )
+ end
+
+ def convert_instance(instance)
+ Instance.new(
+ :id => instance[:aws_instance_id],
+ :name => instance[:aws_image_id],
+ :state => convert_state(instance[:aws_state]),
+ :image_id => instance[:aws_image_id],
+ :owner_id => instance[:aws_owner],
+ :actions =>
instance_actions_for(convert_state(instance[:aws_state])),
+ :key_name => instance[:ssh_key_name],
+ :launch_time => instance[:aws_launch_time],
+ :instance_profile =>
InstanceProfile.new(instance[:aws_instance_type]),
+ :realm_id => instance[:aws_availability_zone],
+ :private_addresses => instance[:private_dns_name],
+ :public_addresses => instance[:dns_name]
+ )
+ end
+
+ def convert_key(key)
+ Key.new(
+ :id => key[:aws_key_name],
+ :fingerprint => key[:aws_fingerprint],
+ :credential_type => :key,
+ :pem_rsa_key => key[:aws_material],
+ :state => "AVAILABLE"
+ )
+ end
+
+ def convert_volume(volume)
+ StorageVolume.new(
+ :id => volume[:aws_id],
+ :created => volume[:aws_created_at],
+ :state => volume[:aws_status] ? volume[:aws_status].upcase :
'unknown',
+ :capacity => volume[:aws_size],
+ :instance_id => volume[:aws_instance_id],
+ :realm_id => volume[:zone],
+ :device => volume[:aws_device],
+ # TODO: the available actions should be tied to the current
+ # volume state
+ :actions => [:attach, :detach, :destroy]
+ )
+ end
+
+ def convert_snapshot(snapshot)
+ StorageSnapshot.new(
+ :id => snapshot[:aws_id],
+ :state => snapshot[:aws_status],
+ :storage_volume_id => snapshot[:aws_volume_id],
+ :created => snapshot[:aws_started_at]
+ )
+ end
+
+ def convert_load_balancer(credentials, loadbalancer)
+ puts loadbalancer.inspect
+ realms = []
+ balancer_realms = loadbalancer[:availability_zones].each do
|zone|
+ realms << realm(credentials, zone)
+ end
+ balancer = LoadBalancer.new({
+ :id => loadbalancer[:name],
+ :created_at => loadbalancer[:created_time],
+ :public_addresses => [loadbalancer[:dns_name]],
+ :realms => realms
+ })
+ balancer.listeners = []
+ balancer.instances = []
+ loadbalancer[:listeners].each do |listener|
+ balancer.add_listener(listener)
+ end
+ loadbalancer[:instances].each do |instance|
+ balancer.instances << instance(credentials, :id =>
instance[:id])
+ end
+ balancer
+ end
+
+ def convert_state(ec2_state)
+ case ec2_state
+ when "terminated"
+ "STOPPED"
+ when "stopped"
+ "STOPPED"
+ when "running"
+ "RUNNING"
+ when "pending"
+ "PENDING"
+ when "shutting-down"
+ "STOPPED"
+ end
+ end
+
+ def catched_exceptions_list
+ {
+ :auth => [], # [ ::Aws::AuthFailure ],
+ :error => [ ::Aws::AwsError ],
+ :glob => [ /AWS::(\w+)/, /Deltacloud::Runner::(\w+)/ ]
+ }
+ end
+
+ end
+ end
+ end
+end
diff --git a/server/server.rb b/server/server.rb index fe723c5..ff7c988
100644
--- a/server/server.rb
+++ b/server/server.rb
@@ -478,8 +478,8 @@ collection :storage_snapshots do
with_capability :destroy_storage_snapshot
param :id, :string, :required
control do
- driver.create_storage_snapshot(credentials, params)
- redirect(storage_snapshot_url(params[:id]))
+ driver.destroy_storage_snapshot(credentials, params)
+ redirect(storage_snapshots_url)
end
end
--
1.7.1
Re: [PATCH] initial eucalyptus support
Posted by David Lutterkort <lu...@redhat.com>.
On Wed, 2011-03-02 at 08:30 -0500, Adrian Cole wrote:
> FYI, in jclouds, the way we manage this is have a base ec2 driver,
> which euca and aws-ec2 extend.
Yeah, that's really the right way to do it. I was hoping by using some
of Ruby's metaprogramming we could get away with a base ec2 driver, and
a euca subclass. But the three-class approach is definitely cleaner.
David
Re: [PATCH] initial eucalyptus support
Posted by Adrian Cole <fe...@gmail.com>.
FYI, in jclouds, the way we manage this is have a base ec2 driver,
which euca and aws-ec2 extend. The reason for this is that there are
many features that aws have that are offering specific (ex. billing
codes, spot market, cluster nodes etc), or are way ahead of the pack.
In other words, we took a LCD approach with the EC2 api and have the
aws-only features or offerings in the aws-ec2 extension.
-A
On Wed, Mar 2, 2011 at 3:07 AM, Sang-Min Park
<sa...@eucalyptus.com> wrote:
> Thanks David for your careful review.
> Yes, I agree that subclassing Euca driver from EC2 is less costly way to
> maintain both drivers. I'll work on it and resubmit the patch soon.
>
> I suspect EC2 driver has been modified since I forked it, because some
> changes in the diff isn't what I intended for.
> Below, I tried to answer some of your questions/comments in line.
>
> Sang-min
>
> -----Original Message-----
> From: David Lutterkort [mailto:lutter@redhat.com]
> Sent: Tuesday, March 01, 2011 5:20 PM
> To: deltacloud-dev@incubator.apache.org
> Cc: sang-min.park@eucalyptus.com; Sang-Min Park
> Subject: Re: [PATCH] initial eucalyptus support
>
> On Wed, 2011-02-23 at 22:56 -0800, sang-min.park@eucalyptus.com wrote:
>> From: Sang-Min Park <sp...@eucalyptus.com>
>>
>> ---
>> server/config/drivers.yaml | 4 +
>> .../drivers/eucalyptus/eucalyptus_driver.rb | 640
> ++++++++++++++++++++
>> server/server.rb | 4 +-
>> 3 files changed, 646 insertions(+), 2 deletions(-) create mode
>> 100644 server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
>
> This needs a lot more work - the driver is an almost verbatim copy of the
> EC2 driver with changes in very specific places. With this approach of
> cloning and then maintaining the code separately, I fear we'll have a much
> higher maintenance burden going forward, especially as we add features to
> the API.
>
> ---> Yes, I wholeheartedly agree with you.
>
> The patch though very nicely outlines the changes that are needed between
> the EC2 and Euca drivers.
>
> Rather than review the submitted patch, here is a diff between
> server/lib/deltacloud/drivers/ec2/ec2_driver.rb and
> server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb:
>
>> eucalyptus_driver.rb | 198
>> +++++++++++++++++----------------------------------
>> 1 file changed, 69 insertions(+), 129 deletions(-)
>> --- ec2/ec2_driver.rb 2011-02-28 16:46:51.000000000 -0800
>> +++ eucalyptus/eucalyptus_driver.rb 2011-02-28 16:47:11.000000000
> -0800
>> @@ -1,3 +1,5 @@
>> +# Copyright (C) 2009-2011 Eucalyptus Systems, Inc.
>> +#
>> # Copyright (C) 2009, 2010 Red Hat, Inc.
>
> Only one of us can claim copyright; given the size of the change, I'd
> still think it's (C) Red Hat. But I'd be just as happy dropping the
> copyright notice altogether.
>
>
> --> Yes, I'd agree that we leave only Red Hat copyright.
>
>
>> # Licensed to the Apache Software Foundation (ASF) under one or more
>> @@ -31,11 +33,11 @@
>>
>> module Deltacloud
>> module Drivers
>> - module EC2
>> - class EC2Driver < Deltacloud::BaseDriver
>> + module Eucalyptus
>> + class EucalyptusDriver < Deltacloud::BaseDriver
>>
>> def supported_collections
>> - DEFAULT_COLLECTIONS + [ :keys, :buckets, :load_balancers ]
>> + DEFAULT_COLLECTIONS + [ :keys, :buckets ]
>
> This will be a recurring theme: we need to find a way to have the Euca
> driver reuse the EC2 driver with cutting and pasting code. There's a
> number of ways to do that in Ruby, most notably subclassing and mixing in
> modules.
>
> I'd suggest we start with making EucalyptusDriver a subclass of EC2Driver.
> With that, supported_collections for Euca can just override the
> corresponding method for EC2.
>
>> @@ -43,71 +45,43 @@
>> feature :instances, :security_group
>> feature :instances, :instance_count
>> feature :images, :owner_id
>> - feature :buckets, :bucket_location
>> - feature :instances, :register_to_load_balancer
>> + feature :buckets, :bucket_location
>> + #feature :instances, :register_to_load_balancer # not
>> + supported by Eucalyptus 2.0.3
>
> We'll need a 'remove_feature' helper in the base driver to do the above.
>
>> DEFAULT_REGION = 'us-east-1'
>>
>> - define_hardware_profile('t1.micro') do
>> - cpu 1
>> - memory 0.63 * 1024
>> - storage 160
>> - architecture 'i386'
>> - end
>
> For hardware profiles, I'd actually prefer if we listed them out
> separately for EC2 and Euca. To make that work, we'd need a
> 'clear_hardware_profiles' helper that can be called before the first
> 'define_hardware_profile' in the Euca driver.
>
> We could of course also read them in from YAML files, with a clever naming
> convention to make sure each driver reads the right one. Seems too
> complicated to me though.
>
> ---> YAML way of doing this might be necessary for Euca, because we allow
> administrators change the instance types (hardware profiles). The hardware
> profile in this patch matches the default instance types after fresh
> installation.
>
>> @@ -135,13 +109,13 @@
>> end
>> return img_arr
>> end
>> - owner_id = opts[:owner_id] || "amazon"
>> + owner_id = opts[:owner_id] || "self"
>
> Factor into a 'default_owner' method.
>
>> safely do
>> - img_arr = ec2.describe_images_by_owner(owner_id,
> "machine").collect do |image|
>> + img_arr = ec2.describe_images_by_owner(owner_id).collect
>> + do |image|
>
> Why doesn't AWS catch this difference (i.e. making the 2nd arg a dummy for
> Euca) ?
>
> --> the second arg is image_type, which if presented to AWS, triggers
> tagging by image type (and Euca doesn't support tagging yet).
>
>> convert_image(image)
>> end
>> end
>> - img_arr = filter_on( img_arr, :architecture, opts )
>> + #img_arr = filter_on( img_arr, :architecture, opts )
>
> Why ?
>
> --> I think it's a bug which I'll fix.
>
>> @@ -162,18 +136,7 @@
>> inst_arr = ec2.describe_instances.collect do |instance|
>> convert_instance(instance) if instance
>> end.flatten
>> - tags = ec2.describe_tags(
>> - 'Filter.1.Name' => 'resource-type', 'Filter.1.Value' =>
> 'instance'
>> - )
>> - inst_arr.each do |inst|
>> - name_tag = tags.select { |t| (t[:aws_resource_id] ==
> inst.id) and t[:aws_key] == 'name' }
>> - unless name_tag.empty?
>> - inst.name = name_tag.first[:aws_value]
>> - end
>> - end
>> - delete_unused_tags(credentials, inst_arr.collect {|inst|
> inst.id})
>> end
>> - inst_arr = filter_on( inst_arr, :id, opts )
>
> Should be factored into a method in EC2Driver; Euca driver should override
> that and return []
>
>> @@ -183,7 +146,7 @@
>> instance_options.merge!(:user_data => opts[:user_data]) if
> opts[:user_data]
>> instance_options.merge!(:key_name => opts[:keyname]) if
> opts[:keyname]
>> instance_options.merge!(:availability_zone =>
> opts[:realm_id]) if opts[:realm_id]
>> - instance_options.merge!(:instance_type => opts[:hwp_id]) if
> opts[:hwp_id]
>> + instance_options.merge!(:instance_type => opts[:hwp_id]) if
>> + opts[:hwp_id] && opts[:hwp_id].length > 0
>
> That seems a safe thing to do for both drivers - under what circumstances
> is opts[:hwp_id] the empty string ?
>
> --> the issue is that the line "(:instance_type => opts[:hwp_id]) if
> opts[:hwp_id]" results in 'InstanceType=''' appended into EC2's
> RunInstance request.
> It works for EC2, but causes message signature validation error with Euca.
> I think the patch is safely applied to EC2 as well.
>
>> @@ -193,15 +156,15 @@
>> new_instance =
> convert_instance(ec2.launch_instances(image_id, instance_options).first)
>> # TODO: Rework this to use client_id for name instead of
> tag
>> # Tags should be keept as an optional feature
>> - if opts[:name]
>> - tag_instance(credentials, new_instance, opts[:name])
>> - end
>> + #if opts[:name]
>> + # tag_instance(credentials, new_instance, opts[:name])
>> + #end
>
> We should pull the 'if opts[:name]' check into tag_instance and have the
> Euca driver replace it with a dummy method.
>
>> # Register Instance to Load Balancer if load_balancer_id is
> set.
>> # This parameter is a feature parameter
>> - if opts[:load_balancer_id] and
> opts[:load_balancer_id].length>0
>> - elb = new_client(credentials, :elb)
>> -
> elb.register_instances_with_load_balancer(opts[:load_balancer_id],
> [new_instance.id])
>> - end
>> + #if opts[:load_balancer_id] and
> opts[:load_balancer_id].length>0
>> + # elb = new_client(credentials, :elb)
>> + #
> elb.register_instances_with_load_balancer(opts[:load_balancer_id],
> [new_instance.id])
>> + #end
>
> Should go into a method similar to tag_instance.
>
>> @@ -233,7 +196,7 @@
>> ec2 = new_client(credentials)
>> instance_id = instance_id
>> if ec2.terminate_instances([instance_id])
>> - untag_instance(credentials, instance_id)
>> + #untag_instance(credentials, instance_id)
>
> Again, override untag_instance with a dummy.
>
>> @@ -270,73 +233,41 @@
>> end
>>
>> def load_balancer(credentials, opts={})
>> - load_balancers(credentials, {
>> - :names => [opts[:id]]
>> - }).first
>> + raise Deltacloud::BackendError.new(500, "Loadbalancer",
>> +"Loadbalancer not supported yet", "")
>> end
>
> This will work nicely in the subclass - similar for the other changes in
> this hunk.
>
>> +
>> def buckets(credentials, opts={})
>> buckets = []
>> safely do
>> s3_client = new_client(credentials, :s3)
>> - unless (opts[:id].nil?)
>> - bucket = s3_client.bucket(opts[:id])
>> - buckets << convert_bucket(bucket)
>> - else
>> - bucket_list = s3_client.buckets
>> - bucket_list.each do |current|
>> - buckets << Bucket.new({:name => current.name, :id =>
> current.name})
>> - end #bucket_list.each
>> - end #if
>> - end #safely
>> - filter_on(buckets, :id, opts)
>> + bucket_list = s3_client.buckets
>> +
>> + bucket_list.each do |current|
>> + buckets << convert_bucket(current)
>> + end
>> + end
>> + buckets
>
> Doesn't this mean that if I ask for a specific bucket, I'd still get all
> of them back ? Is this needed because Euca doesn't support retrieving an
> individual bucket ?
>
> Either way, the filter_on needs to stay.
>
> --> I suspect EC2 driver has been modified since I forked it. I'll leave
> filter_on in the next patch.
>
>> @@ -436,11 +367,12 @@
>> def create_storage_volume(credentials, opts=nil)
>> ec2 = new_client(credentials)
>> opts ||= {}
>> - opts[:snapshot_id] ||= ""
>> + opts[:snapshot_id] ||= ""# -> this causes signature
>> + validation error with Euca backend
>
> Any idea why ?
>
> --> So this is fixed by patching AWS. Similarly to InstanceType in
> RunInstance request, it leaves 'SnapshotId='' ' in the EC2 CreateVolume
> request, which generates signature error with Euca.
>
>> opts[:capacity] ||= "1"
>> opts[:realm_id] ||= realms(credentials).first.id
>> safely do
>> - convert_volume(ec2.create_volume(opts[:snapshot_id],
> opts[:capacity], opts[:realm_id]))
>> + convert_volume(ec2.create_volume(opts[:snapshot_id],
>> + opts[:capacity], opts[:realm_id]))
>> +
>
> Only whitespace change AFAICT
>
>> @@ -488,7 +420,7 @@
>> def destroy_storage_snapshot(credentials, opts={})
>> ec2 = new_client(credentials)
>> safely do
>> - unless convert_snapshot(opts[:id])
>> + unless ec2.delete_snapshot(opts[:id])
>
> This seems to be a bug in the EC2 driver, and should be broken out into a
> separate patch.
>
>> @@ -512,16 +444,21 @@
>> when :ec2 then Aws::Ec2
>> when :s3 then Aws::S3
>> end
>> - klass.new(credentials.user, credentials.password, {:server =>
> endpoint_for_service(type), :connection_mode => :per_thread})
>> + klass.new(credentials.user, credentials.password,
>> +eucalyptus_endpoint) #:server => endpoint_for_service(type))
>> end
>
> Just override new_client wholesale.
>
>> + # shouldn't be called now; eucalyptus doens't support tagging yet
>> def tag_instance(credentials, instance, name)
>> ec2 = new_client(credentials)
>> safely do
>
> As mentioned above, should become a dummy method.
>
>> @@ -529,6 +466,7 @@
>> end
>> end
>>
>> + # shouldn't be called now; eucalyptus doens't support tagging
>> + yet
>> def untag_instance(credentials, instance_id)
>
> As mentioned above, should become a dummy method.
>
>> @@ -536,6 +474,7 @@
>> end
>> end
>>
>> + # shouldn't be called now; eucalyptus doens't support tagging
>> + yet
>> def delete_unused_tags(credentials, inst_ids)
>
> As mentioned above, should become a dummy method.
>
>> @@ -556,11 +495,12 @@
>>
>> #can use AWS::S3::Owner.current.display_name or current.id
>> Bucket.new(
>> :id => s3_bucket.name,
>> :name => s3_bucket.name,
>> - :size => blob_list.length,
>> + :size => s3_bucket.keys.length,
>> :blob_list => blob_list
>
> Why ?
>
> --> again, I guess EC2 driver has been modified since my fork of it.
> There's no need to do this.
>
> David
>
RE: [PATCH] initial eucalyptus support
Posted by Sang-Min Park <sa...@eucalyptus.com>.
Thanks David for your careful review.
Yes, I agree that subclassing Euca driver from EC2 is less costly way to
maintain both drivers. I'll work on it and resubmit the patch soon.
I suspect EC2 driver has been modified since I forked it, because some
changes in the diff isn't what I intended for.
Below, I tried to answer some of your questions/comments in line.
Sang-min
-----Original Message-----
From: David Lutterkort [mailto:lutter@redhat.com]
Sent: Tuesday, March 01, 2011 5:20 PM
To: deltacloud-dev@incubator.apache.org
Cc: sang-min.park@eucalyptus.com; Sang-Min Park
Subject: Re: [PATCH] initial eucalyptus support
On Wed, 2011-02-23 at 22:56 -0800, sang-min.park@eucalyptus.com wrote:
> From: Sang-Min Park <sp...@eucalyptus.com>
>
> ---
> server/config/drivers.yaml | 4 +
> .../drivers/eucalyptus/eucalyptus_driver.rb | 640
++++++++++++++++++++
> server/server.rb | 4 +-
> 3 files changed, 646 insertions(+), 2 deletions(-) create mode
> 100644 server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
This needs a lot more work - the driver is an almost verbatim copy of the
EC2 driver with changes in very specific places. With this approach of
cloning and then maintaining the code separately, I fear we'll have a much
higher maintenance burden going forward, especially as we add features to
the API.
---> Yes, I wholeheartedly agree with you.
The patch though very nicely outlines the changes that are needed between
the EC2 and Euca drivers.
Rather than review the submitted patch, here is a diff between
server/lib/deltacloud/drivers/ec2/ec2_driver.rb and
server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb:
> eucalyptus_driver.rb | 198
> +++++++++++++++++----------------------------------
> 1 file changed, 69 insertions(+), 129 deletions(-)
> --- ec2/ec2_driver.rb 2011-02-28 16:46:51.000000000 -0800
> +++ eucalyptus/eucalyptus_driver.rb 2011-02-28 16:47:11.000000000
-0800
> @@ -1,3 +1,5 @@
> +# Copyright (C) 2009-2011 Eucalyptus Systems, Inc.
> +#
> # Copyright (C) 2009, 2010 Red Hat, Inc.
Only one of us can claim copyright; given the size of the change, I'd
still think it's (C) Red Hat. But I'd be just as happy dropping the
copyright notice altogether.
--> Yes, I'd agree that we leave only Red Hat copyright.
> # Licensed to the Apache Software Foundation (ASF) under one or more
> @@ -31,11 +33,11 @@
>
> module Deltacloud
> module Drivers
> - module EC2
> - class EC2Driver < Deltacloud::BaseDriver
> + module Eucalyptus
> + class EucalyptusDriver < Deltacloud::BaseDriver
>
> def supported_collections
> - DEFAULT_COLLECTIONS + [ :keys, :buckets, :load_balancers ]
> + DEFAULT_COLLECTIONS + [ :keys, :buckets ]
This will be a recurring theme: we need to find a way to have the Euca
driver reuse the EC2 driver with cutting and pasting code. There's a
number of ways to do that in Ruby, most notably subclassing and mixing in
modules.
I'd suggest we start with making EucalyptusDriver a subclass of EC2Driver.
With that, supported_collections for Euca can just override the
corresponding method for EC2.
> @@ -43,71 +45,43 @@
> feature :instances, :security_group
> feature :instances, :instance_count
> feature :images, :owner_id
> - feature :buckets, :bucket_location
> - feature :instances, :register_to_load_balancer
> + feature :buckets, :bucket_location
> + #feature :instances, :register_to_load_balancer # not
> + supported by Eucalyptus 2.0.3
We'll need a 'remove_feature' helper in the base driver to do the above.
> DEFAULT_REGION = 'us-east-1'
>
> - define_hardware_profile('t1.micro') do
> - cpu 1
> - memory 0.63 * 1024
> - storage 160
> - architecture 'i386'
> - end
For hardware profiles, I'd actually prefer if we listed them out
separately for EC2 and Euca. To make that work, we'd need a
'clear_hardware_profiles' helper that can be called before the first
'define_hardware_profile' in the Euca driver.
We could of course also read them in from YAML files, with a clever naming
convention to make sure each driver reads the right one. Seems too
complicated to me though.
---> YAML way of doing this might be necessary for Euca, because we allow
administrators change the instance types (hardware profiles). The hardware
profile in this patch matches the default instance types after fresh
installation.
> @@ -135,13 +109,13 @@
> end
> return img_arr
> end
> - owner_id = opts[:owner_id] || "amazon"
> + owner_id = opts[:owner_id] || "self"
Factor into a 'default_owner' method.
> safely do
> - img_arr = ec2.describe_images_by_owner(owner_id,
"machine").collect do |image|
> + img_arr = ec2.describe_images_by_owner(owner_id).collect
> + do |image|
Why doesn't AWS catch this difference (i.e. making the 2nd arg a dummy for
Euca) ?
--> the second arg is image_type, which if presented to AWS, triggers
tagging by image type (and Euca doesn't support tagging yet).
> convert_image(image)
> end
> end
> - img_arr = filter_on( img_arr, :architecture, opts )
> + #img_arr = filter_on( img_arr, :architecture, opts )
Why ?
--> I think it's a bug which I'll fix.
> @@ -162,18 +136,7 @@
> inst_arr = ec2.describe_instances.collect do |instance|
> convert_instance(instance) if instance
> end.flatten
> - tags = ec2.describe_tags(
> - 'Filter.1.Name' => 'resource-type', 'Filter.1.Value' =>
'instance'
> - )
> - inst_arr.each do |inst|
> - name_tag = tags.select { |t| (t[:aws_resource_id] ==
inst.id) and t[:aws_key] == 'name' }
> - unless name_tag.empty?
> - inst.name = name_tag.first[:aws_value]
> - end
> - end
> - delete_unused_tags(credentials, inst_arr.collect {|inst|
inst.id})
> end
> - inst_arr = filter_on( inst_arr, :id, opts )
Should be factored into a method in EC2Driver; Euca driver should override
that and return []
> @@ -183,7 +146,7 @@
> instance_options.merge!(:user_data => opts[:user_data]) if
opts[:user_data]
> instance_options.merge!(:key_name => opts[:keyname]) if
opts[:keyname]
> instance_options.merge!(:availability_zone =>
opts[:realm_id]) if opts[:realm_id]
> - instance_options.merge!(:instance_type => opts[:hwp_id]) if
opts[:hwp_id]
> + instance_options.merge!(:instance_type => opts[:hwp_id]) if
> + opts[:hwp_id] && opts[:hwp_id].length > 0
That seems a safe thing to do for both drivers - under what circumstances
is opts[:hwp_id] the empty string ?
--> the issue is that the line "(:instance_type => opts[:hwp_id]) if
opts[:hwp_id]" results in 'InstanceType=''' appended into EC2's
RunInstance request.
It works for EC2, but causes message signature validation error with Euca.
I think the patch is safely applied to EC2 as well.
> @@ -193,15 +156,15 @@
> new_instance =
convert_instance(ec2.launch_instances(image_id, instance_options).first)
> # TODO: Rework this to use client_id for name instead of
tag
> # Tags should be keept as an optional feature
> - if opts[:name]
> - tag_instance(credentials, new_instance, opts[:name])
> - end
> + #if opts[:name]
> + # tag_instance(credentials, new_instance, opts[:name])
> + #end
We should pull the 'if opts[:name]' check into tag_instance and have the
Euca driver replace it with a dummy method.
> # Register Instance to Load Balancer if load_balancer_id is
set.
> # This parameter is a feature parameter
> - if opts[:load_balancer_id] and
opts[:load_balancer_id].length>0
> - elb = new_client(credentials, :elb)
> -
elb.register_instances_with_load_balancer(opts[:load_balancer_id],
[new_instance.id])
> - end
> + #if opts[:load_balancer_id] and
opts[:load_balancer_id].length>0
> + # elb = new_client(credentials, :elb)
> + #
elb.register_instances_with_load_balancer(opts[:load_balancer_id],
[new_instance.id])
> + #end
Should go into a method similar to tag_instance.
> @@ -233,7 +196,7 @@
> ec2 = new_client(credentials)
> instance_id = instance_id
> if ec2.terminate_instances([instance_id])
> - untag_instance(credentials, instance_id)
> + #untag_instance(credentials, instance_id)
Again, override untag_instance with a dummy.
> @@ -270,73 +233,41 @@
> end
>
> def load_balancer(credentials, opts={})
> - load_balancers(credentials, {
> - :names => [opts[:id]]
> - }).first
> + raise Deltacloud::BackendError.new(500, "Loadbalancer",
> +"Loadbalancer not supported yet", "")
> end
This will work nicely in the subclass - similar for the other changes in
this hunk.
> +
> def buckets(credentials, opts={})
> buckets = []
> safely do
> s3_client = new_client(credentials, :s3)
> - unless (opts[:id].nil?)
> - bucket = s3_client.bucket(opts[:id])
> - buckets << convert_bucket(bucket)
> - else
> - bucket_list = s3_client.buckets
> - bucket_list.each do |current|
> - buckets << Bucket.new({:name => current.name, :id =>
current.name})
> - end #bucket_list.each
> - end #if
> - end #safely
> - filter_on(buckets, :id, opts)
> + bucket_list = s3_client.buckets
> +
> + bucket_list.each do |current|
> + buckets << convert_bucket(current)
> + end
> + end
> + buckets
Doesn't this mean that if I ask for a specific bucket, I'd still get all
of them back ? Is this needed because Euca doesn't support retrieving an
individual bucket ?
Either way, the filter_on needs to stay.
--> I suspect EC2 driver has been modified since I forked it. I'll leave
filter_on in the next patch.
> @@ -436,11 +367,12 @@
> def create_storage_volume(credentials, opts=nil)
> ec2 = new_client(credentials)
> opts ||= {}
> - opts[:snapshot_id] ||= ""
> + opts[:snapshot_id] ||= ""# -> this causes signature
> + validation error with Euca backend
Any idea why ?
--> So this is fixed by patching AWS. Similarly to InstanceType in
RunInstance request, it leaves 'SnapshotId='' ' in the EC2 CreateVolume
request, which generates signature error with Euca.
> opts[:capacity] ||= "1"
> opts[:realm_id] ||= realms(credentials).first.id
> safely do
> - convert_volume(ec2.create_volume(opts[:snapshot_id],
opts[:capacity], opts[:realm_id]))
> + convert_volume(ec2.create_volume(opts[:snapshot_id],
> + opts[:capacity], opts[:realm_id]))
> +
Only whitespace change AFAICT
> @@ -488,7 +420,7 @@
> def destroy_storage_snapshot(credentials, opts={})
> ec2 = new_client(credentials)
> safely do
> - unless convert_snapshot(opts[:id])
> + unless ec2.delete_snapshot(opts[:id])
This seems to be a bug in the EC2 driver, and should be broken out into a
separate patch.
> @@ -512,16 +444,21 @@
> when :ec2 then Aws::Ec2
> when :s3 then Aws::S3
> end
> - klass.new(credentials.user, credentials.password, {:server =>
endpoint_for_service(type), :connection_mode => :per_thread})
> + klass.new(credentials.user, credentials.password,
> +eucalyptus_endpoint) #:server => endpoint_for_service(type))
> end
Just override new_client wholesale.
> + # shouldn't be called now; eucalyptus doens't support tagging yet
> def tag_instance(credentials, instance, name)
> ec2 = new_client(credentials)
> safely do
As mentioned above, should become a dummy method.
> @@ -529,6 +466,7 @@
> end
> end
>
> + # shouldn't be called now; eucalyptus doens't support tagging
> + yet
> def untag_instance(credentials, instance_id)
As mentioned above, should become a dummy method.
> @@ -536,6 +474,7 @@
> end
> end
>
> + # shouldn't be called now; eucalyptus doens't support tagging
> + yet
> def delete_unused_tags(credentials, inst_ids)
As mentioned above, should become a dummy method.
> @@ -556,11 +495,12 @@
>
> #can use AWS::S3::Owner.current.display_name or current.id
> Bucket.new(
> :id => s3_bucket.name,
> :name => s3_bucket.name,
> - :size => blob_list.length,
> + :size => s3_bucket.keys.length,
> :blob_list => blob_list
Why ?
--> again, I guess EC2 driver has been modified since my fork of it.
There's no need to do this.
David
Re: [PATCH] initial eucalyptus support
Posted by David Lutterkort <lu...@redhat.com>.
On Wed, 2011-02-23 at 22:56 -0800, sang-min.park@eucalyptus.com wrote:
> From: Sang-Min Park <sp...@eucalyptus.com>
>
> ---
> server/config/drivers.yaml | 4 +
> .../drivers/eucalyptus/eucalyptus_driver.rb | 640 ++++++++++++++++++++
> server/server.rb | 4 +-
> 3 files changed, 646 insertions(+), 2 deletions(-)
> create mode 100644 server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
This needs a lot more work - the driver is an almost verbatim copy of
the EC2 driver with changes in very specific places. With this approach
of cloning and then maintaining the code separately, I fear we'll have a
much higher maintenance burden going forward, especially as we add
features to the API.
The patch though very nicely outlines the changes that are needed
between the EC2 and Euca drivers.
Rather than review the submitted patch, here is a diff between
server/lib/deltacloud/drivers/ec2/ec2_driver.rb and
server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb:
> eucalyptus_driver.rb | 198 +++++++++++++++++----------------------------------
> 1 file changed, 69 insertions(+), 129 deletions(-)
> --- ec2/ec2_driver.rb 2011-02-28 16:46:51.000000000 -0800
> +++ eucalyptus/eucalyptus_driver.rb 2011-02-28 16:47:11.000000000 -0800
> @@ -1,3 +1,5 @@
> +# Copyright (C) 2009-2011 Eucalyptus Systems, Inc.
> +#
> # Copyright (C) 2009, 2010 Red Hat, Inc.
Only one of us can claim copyright; given the size of the change, I'd
still think it's (C) Red Hat. But I'd be just as happy dropping the
copyright notice altogether.
> # Licensed to the Apache Software Foundation (ASF) under one or more
> @@ -31,11 +33,11 @@
>
> module Deltacloud
> module Drivers
> - module EC2
> - class EC2Driver < Deltacloud::BaseDriver
> + module Eucalyptus
> + class EucalyptusDriver < Deltacloud::BaseDriver
>
> def supported_collections
> - DEFAULT_COLLECTIONS + [ :keys, :buckets, :load_balancers ]
> + DEFAULT_COLLECTIONS + [ :keys, :buckets ]
This will be a recurring theme: we need to find a way to have the Euca
driver reuse the EC2 driver with cutting and pasting code. There's a
number of ways to do that in Ruby, most notably subclassing and mixing
in modules.
I'd suggest we start with making EucalyptusDriver a subclass of
EC2Driver. With that, supported_collections for Euca can just override
the corresponding method for EC2.
> @@ -43,71 +45,43 @@
> feature :instances, :security_group
> feature :instances, :instance_count
> feature :images, :owner_id
> - feature :buckets, :bucket_location
> - feature :instances, :register_to_load_balancer
> + feature :buckets, :bucket_location
> + #feature :instances, :register_to_load_balancer # not supported by Eucalyptus 2.0.3
We'll need a 'remove_feature' helper in the base driver to do the above.
> DEFAULT_REGION = 'us-east-1'
>
> - define_hardware_profile('t1.micro') do
> - cpu 1
> - memory 0.63 * 1024
> - storage 160
> - architecture 'i386'
> - end
For hardware profiles, I'd actually prefer if we listed them out
separately for EC2 and Euca. To make that work, we'd need a
'clear_hardware_profiles' helper that can be called before the first
'define_hardware_profile' in the Euca driver.
We could of course also read them in from YAML files, with a clever
naming convention to make sure each driver reads the right one. Seems
too complicated to me though.
> @@ -135,13 +109,13 @@
> end
> return img_arr
> end
> - owner_id = opts[:owner_id] || "amazon"
> + owner_id = opts[:owner_id] || "self"
Factor into a 'default_owner' method.
> safely do
> - img_arr = ec2.describe_images_by_owner(owner_id, "machine").collect do |image|
> + img_arr = ec2.describe_images_by_owner(owner_id).collect do |image|
Why doesn't AWS catch this difference (i.e. making the 2nd arg a dummy for Euca) ?
> convert_image(image)
> end
> end
> - img_arr = filter_on( img_arr, :architecture, opts )
> + #img_arr = filter_on( img_arr, :architecture, opts )
Why ?
> @@ -162,18 +136,7 @@
> inst_arr = ec2.describe_instances.collect do |instance|
> convert_instance(instance) if instance
> end.flatten
> - tags = ec2.describe_tags(
> - 'Filter.1.Name' => 'resource-type', 'Filter.1.Value' => 'instance'
> - )
> - inst_arr.each do |inst|
> - name_tag = tags.select { |t| (t[:aws_resource_id] == inst.id) and t[:aws_key] == 'name' }
> - unless name_tag.empty?
> - inst.name = name_tag.first[:aws_value]
> - end
> - end
> - delete_unused_tags(credentials, inst_arr.collect {|inst| inst.id})
> end
> - inst_arr = filter_on( inst_arr, :id, opts )
Should be factored into a method in EC2Driver; Euca driver should override that and return []
> @@ -183,7 +146,7 @@
> instance_options.merge!(:user_data => opts[:user_data]) if opts[:user_data]
> instance_options.merge!(:key_name => opts[:keyname]) if opts[:keyname]
> instance_options.merge!(:availability_zone => opts[:realm_id]) if opts[:realm_id]
> - instance_options.merge!(:instance_type => opts[:hwp_id]) if opts[:hwp_id]
> + instance_options.merge!(:instance_type => opts[:hwp_id]) if opts[:hwp_id] && opts[:hwp_id].length > 0
That seems a safe thing to do for both drivers - under what
circumstances is opts[:hwp_id] the empty string ?
> @@ -193,15 +156,15 @@
> new_instance = convert_instance(ec2.launch_instances(image_id, instance_options).first)
> # TODO: Rework this to use client_id for name instead of tag
> # Tags should be keept as an optional feature
> - if opts[:name]
> - tag_instance(credentials, new_instance, opts[:name])
> - end
> + #if opts[:name]
> + # tag_instance(credentials, new_instance, opts[:name])
> + #end
We should pull the 'if opts[:name]' check into tag_instance and have the
Euca driver replace it with a dummy method.
> # Register Instance to Load Balancer if load_balancer_id is set.
> # This parameter is a feature parameter
> - if opts[:load_balancer_id] and opts[:load_balancer_id].length>0
> - elb = new_client(credentials, :elb)
> - elb.register_instances_with_load_balancer(opts[:load_balancer_id], [new_instance.id])
> - end
> + #if opts[:load_balancer_id] and opts[:load_balancer_id].length>0
> + # elb = new_client(credentials, :elb)
> + # elb.register_instances_with_load_balancer(opts[:load_balancer_id], [new_instance.id])
> + #end
Should go into a method similar to tag_instance.
> @@ -233,7 +196,7 @@
> ec2 = new_client(credentials)
> instance_id = instance_id
> if ec2.terminate_instances([instance_id])
> - untag_instance(credentials, instance_id)
> + #untag_instance(credentials, instance_id)
Again, override untag_instance with a dummy.
> @@ -270,73 +233,41 @@
> end
>
> def load_balancer(credentials, opts={})
> - load_balancers(credentials, {
> - :names => [opts[:id]]
> - }).first
> + raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
> end
This will work nicely in the subclass - similar for the other changes in this hunk.
> +
> def buckets(credentials, opts={})
> buckets = []
> safely do
> s3_client = new_client(credentials, :s3)
> - unless (opts[:id].nil?)
> - bucket = s3_client.bucket(opts[:id])
> - buckets << convert_bucket(bucket)
> - else
> - bucket_list = s3_client.buckets
> - bucket_list.each do |current|
> - buckets << Bucket.new({:name => current.name, :id => current.name})
> - end #bucket_list.each
> - end #if
> - end #safely
> - filter_on(buckets, :id, opts)
> + bucket_list = s3_client.buckets
> +
> + bucket_list.each do |current|
> + buckets << convert_bucket(current)
> + end
> + end
> + buckets
Doesn't this mean that if I ask for a specific bucket, I'd still get all
of them back ? Is this needed because Euca doesn't support retrieving an
individual bucket ?
Either way, the filter_on needs to stay.
> @@ -436,11 +367,12 @@
> def create_storage_volume(credentials, opts=nil)
> ec2 = new_client(credentials)
> opts ||= {}
> - opts[:snapshot_id] ||= ""
> + opts[:snapshot_id] ||= ""# -> this causes signature validation error with Euca backend
Any idea why ?
> opts[:capacity] ||= "1"
> opts[:realm_id] ||= realms(credentials).first.id
> safely do
> - convert_volume(ec2.create_volume(opts[:snapshot_id], opts[:capacity], opts[:realm_id]))
> + convert_volume(ec2.create_volume(opts[:snapshot_id], opts[:capacity], opts[:realm_id]))
> +
Only whitespace change AFAICT
> @@ -488,7 +420,7 @@
> def destroy_storage_snapshot(credentials, opts={})
> ec2 = new_client(credentials)
> safely do
> - unless convert_snapshot(opts[:id])
> + unless ec2.delete_snapshot(opts[:id])
This seems to be a bug in the EC2 driver, and should be broken out into
a separate patch.
> @@ -512,16 +444,21 @@
> when :ec2 then Aws::Ec2
> when :s3 then Aws::S3
> end
> - klass.new(credentials.user, credentials.password, {:server => endpoint_for_service(type), :connection_mode => :per_thread})
> + klass.new(credentials.user, credentials.password, eucalyptus_endpoint)
> +#:server => endpoint_for_service(type))
> end
Just override new_client wholesale.
> + # shouldn't be called now; eucalyptus doens't support tagging yet
> def tag_instance(credentials, instance, name)
> ec2 = new_client(credentials)
> safely do
As mentioned above, should become a dummy method.
> @@ -529,6 +466,7 @@
> end
> end
>
> + # shouldn't be called now; eucalyptus doens't support tagging yet
> def untag_instance(credentials, instance_id)
As mentioned above, should become a dummy method.
> @@ -536,6 +474,7 @@
> end
> end
>
> + # shouldn't be called now; eucalyptus doens't support tagging yet
> def delete_unused_tags(credentials, inst_ids)
As mentioned above, should become a dummy method.
> @@ -556,11 +495,12 @@
>
> #can use AWS::S3::Owner.current.display_name or current.id
> Bucket.new(
> :id => s3_bucket.name,
> :name => s3_bucket.name,
> - :size => blob_list.length,
> + :size => s3_bucket.keys.length,
> :blob_list => blob_list
Why ?
David
[PATCH] initial eucalyptus support
Posted by sa...@eucalyptus.com.
From: Sang-Min Park <sp...@eucalyptus.com>
---
server/config/drivers.yaml | 4 +
.../drivers/eucalyptus/eucalyptus_driver.rb | 640 ++++++++++++++++++++
server/server.rb | 4 +-
3 files changed, 646 insertions(+), 2 deletions(-)
create mode 100644 server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
diff --git a/server/config/drivers.yaml b/server/config/drivers.yaml
index f28234c..e21ef47 100644
--- a/server/config/drivers.yaml
+++ b/server/config/drivers.yaml
@@ -28,6 +28,10 @@
:name: Rackspace
:azure:
:name: Azure
+:eucalyptus:
+ :name: Eucalyptus
+ :username: Access Key ID
+ :password: Secret Access Key
:ec2:
:entrypoints:
s3:
diff --git a/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb b/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
new file mode 100644
index 0000000..1b88bca
--- /dev/null
+++ b/server/lib/deltacloud/drivers/eucalyptus/eucalyptus_driver.rb
@@ -0,0 +1,640 @@
+# Copyright (C) 2009-2011 Eucalyptus Systems, Inc.
+#
+# Copyright (C) 2009, 2010 Red Hat, Inc.
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+require 'deltacloud/base_driver'
+require 'aws'
+
+class Instance
+ attr_accessor :keyname
+ attr_accessor :authn_error
+
+ def authn_feature_failed?
+ return true unless authn_error.nil?
+ end
+
+end
+
+module Deltacloud
+ module Drivers
+ module Eucalyptus
+ class EucalyptusDriver < Deltacloud::BaseDriver
+
+ def supported_collections
+ DEFAULT_COLLECTIONS + [ :keys, :buckets ]
+ end
+
+ feature :instances, :user_data
+ feature :instances, :authentication_key
+ feature :instances, :security_group
+ feature :instances, :instance_count
+ feature :images, :owner_id
+ feature :buckets, :bucket_location
+ #feature :instances, :register_to_load_balancer # not supported by Eucalyptus 2.0.3
+
+ DEFAULT_REGION = 'us-east-1'
+
+ define_hardware_profile('m1.small') do
+ cpu 1
+ memory 128
+ storage 2
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('c1.medium') do
+ cpu 1
+ memory 256
+ storage 5
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('m1.large') do
+ cpu 2
+ memory 512
+ storage 10
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('m1.xlarge') do
+ cpu 2
+ memory 1024
+ storage 20
+ architecture 'x86_64'
+ end
+
+ define_hardware_profile('c1.xlarge') do
+ cpu 4
+ memory 2048
+ storage 20
+ architecture 'x86_64'
+ end
+
+ define_instance_states do
+ start.to( :pending ) .automatically
+ pending.to( :running ) .automatically
+ pending.to( :stopping ) .on( :stop )
+ pending.to( :stopped ) .automatically
+ stopped.to( :running ) .on( :start )
+ running.to( :running ) .on( :reboot )
+ running.to( :stopping ) .on( :stop )
+ shutting_down.to( :stopped ) .automatically
+ stopped.to( :finish ) .automatically
+ end
+
+ def images(credentials, opts={})
+ ec2 = new_client(credentials)
+ img_arr = []
+ opts ||= {}
+ if opts[:id]
+ safely do
+ img_arr = ec2.describe_images(opts[:id]).collect do |image|
+ convert_image(image)
+ end
+ end
+ return img_arr
+ end
+ owner_id = opts[:owner_id] || "self"
+ safely do
+ img_arr = ec2.describe_images_by_owner(owner_id).collect do |image|
+ convert_image(image)
+ end
+ end
+ #img_arr = filter_on( img_arr, :architecture, opts )
+ img_arr.sort_by { |e| [e.owner_id, e.name] }
+ end
+
+ def realms(credentials, opts={})
+ ec2 = new_client(credentials)
+ zone_id = opts ? opts[:id] : nil
+ safely do
+ return ec2.describe_availability_zones(zone_id).collect do |realm|
+ convert_realm(realm)
+ end
+ end
+ end
+
+ def instances(credentials, opts={})
+ ec2 = new_client(credentials)
+ inst_arr = []
+ safely do
+ inst_arr = ec2.describe_instances.collect do |instance|
+ convert_instance(instance) if instance
+ end.flatten
+ end
+ filter_on( inst_arr, :state, opts )
+ end
+
+ def create_instance(credentials, image_id, opts={})
+ ec2 = new_client(credentials)
+ instance_options = {}
+ instance_options.merge!(:user_data => opts[:user_data]) if opts[:user_data]
+ instance_options.merge!(:key_name => opts[:keyname]) if opts[:keyname]
+ instance_options.merge!(:availability_zone => opts[:realm_id]) if opts[:realm_id]
+ instance_options.merge!(:instance_type => opts[:hwp_id]) if opts[:hwp_id] && opts[:hwp_id].length > 0
+ instance_options.merge!(:group_ids => opts[:security_group]) if opts[:security_group]
+ instance_options.merge!(
+ :min_count => opts[:instance_count],
+ :max_count => opts[:instance_count]
+ ) if opts[:instance_count] and opts[:instance_count].length!=0
+ safely do
+ new_instance = convert_instance(ec2.launch_instances(image_id, instance_options).first)
+ # TODO: Rework this to use client_id for name instead of tag
+ # Tags should be keept as an optional feature
+ #if opts[:name]
+ # tag_instance(credentials, new_instance, opts[:name])
+ #end
+ # Register Instance to Load Balancer if load_balancer_id is set.
+ # This parameter is a feature parameter
+ #if opts[:load_balancer_id] and opts[:load_balancer_id].length>0
+ # elb = new_client(credentials, :elb)
+ # elb.register_instances_with_load_balancer(opts[:load_balancer_id], [new_instance.id])
+ #end
+ new_instance
+ end
+ end
+
+ def run_on_instance(credentials, opts={})
+ target = instance(credentials, :id => opts[:id])
+ param = {}
+ param[:credentials] = {
+ :username => 'root', # Default for EC2 Linux instances
+ }
+ param[:port] = opts[:port] || '22'
+ param[:ip] = target.public_addresses
+ param[:private_key] = (opts[:private_key].length > 1) ? opts[:private_key] : nil
+ safely do
+ Deltacloud::Runner.execute(opts[:cmd], param)
+ end
+ end
+
+ def reboot_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ if ec2.reboot_instances([instance_id])
+ instance(credentials, instance_id)
+ else
+ raise Deltacloud::BackendError.new(500, "Instance", "Instance reboot failed", "")
+ end
+ end
+
+ def destroy_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ instance_id = instance_id
+ if ec2.terminate_instances([instance_id])
+ #untag_instance(credentials, instance_id)
+ instance(credentials, instance_id)
+ else
+ raise Deltacloud::BackendError.new(500, "Instance", "Instance cannot be terminated", "")
+ end
+ end
+
+ alias :stop_instance :destroy_instance
+
+ def keys(credentials, opts={})
+ ec2 = new_client(credentials)
+ opts ||= {}
+ safely do
+ ec2.describe_key_pairs(opts[:id] || nil).collect do |key|
+ convert_key(key)
+ end
+ end
+ end
+
+ def create_key(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_key(ec2.create_key_pair(opts[:key_name]))
+ end
+ end
+
+ def destroy_key(credentials, opts={})
+ ec2 = new_client(credentials)
+ original_key = key(credentials, opts)
+ safely do
+ ec2.delete_key_pair(original_key.id)
+ original_key= original_key.state = "DELETED"
+ end
+ original_key
+ end
+
+ def load_balancer(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+ def load_balancers(credentials, opts=nil)
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+ def create_load_balancer(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+ def destroy_load_balancer(credentials, id)
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+ def lb_register_instance(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+ def lb_unregister_instance(credentials, opts={})
+ raise Deltacloud::BackendError.new(500, "Loadbalancer", "Loadbalancer not supported yet", "")
+ end
+
+
+ def buckets(credentials, opts={})
+ buckets = []
+ safely do
+ s3_client = new_client(credentials, :s3)
+ bucket_list = s3_client.buckets
+
+ bucket_list.each do |current|
+ buckets << convert_bucket(current)
+ end
+ end
+ buckets
+ end
+
+ def create_bucket(credentials, name, opts={})
+ bucket = nil
+ safely do
+ s3_client = new_client(credentials, :s3)
+ bucket_location = opts['location']
+ if bucket_location
+ bucket = Aws::S3::Bucket.create(s3_client, name, true, nil, :location => bucket_location)
+ else
+ bucket = Aws::S3::Bucket.create(s3_client, name, true)
+ end
+ end
+ convert_bucket(bucket)
+ end
+
+ def delete_bucket(credentials, name, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.delete_bucket(name)
+ end
+ end
+
+ def blobs(credentials, opts = {})
+ s3_client = new_client(credentials, :s3)
+ blobs = []
+ safely do
+ s3_bucket = s3_client.bucket(opts['bucket'])
+ s3_bucket.keys({}, true).each do |s3_object|
+ blobs << convert_object(s3_object)
+ end
+ end
+ blobs = filter_on(blobs, :id, opts)
+ blobs
+ end
+
+ #--
+ # Create Blob
+ #--
+ def create_blob(credentials, bucket_id, blob_id, data = nil, opts = {})
+ s3_client = new_client(credentials, :s3)
+ #data is a construct with the temporary file created by server @.tempfile
+ #also file[:type] will give us the content-type
+ res = nil
+ # File stream needs to be reopened in binary mode for whatever reason
+ file = File::open(data[:tempfile].path, 'rb')
+ #insert ec2-specific header for user metadata ... x-amz-meta-KEY = VALUE
+ opts.gsub_keys('HTTP_X_Deltacloud_Blobmeta_', 'x-amz-meta-')
+ opts["Content-Type"] = data[:type]
+ safely do
+ res = s3_client.interface.put(bucket_id,
+ blob_id,
+ file,
+ opts)
+ end
+ #create a new Blob object and return that
+ Blob.new( { :id => blob_id,
+ :bucket => bucket_id,
+ :content_length => data[:tempfile].length,
+ :content_type => data[:type],
+ :last_modified => '',
+ :user_metadata => opts.select{|k,v| k.match(/^x-amz-meta-/i)}
+ }
+ )
+ end
+
+ #--
+ # Delete Blob
+ #--
+ def delete_blob(credentials, bucket_id, blob_id, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.delete(bucket_id, blob_id)
+ end
+ end
+
+
+ def blob_data(credentials, bucket_id, blob_id, opts={})
+ s3_client = new_client(credentials, :s3)
+ safely do
+ s3_client.interface.get(bucket_id, blob_id) do |chunk|
+ yield chunk
+ end
+ end
+ end
+
+ def storage_volumes(credentials, opts={})
+ ec2 = new_client( credentials )
+ volume_list = (opts and opts[:id]) ? opts[:id] : nil
+ safely do
+ ec2.describe_volumes(volume_list).collect do |volume|
+ convert_volume(volume)
+ end
+ end
+ end
+
+ def create_storage_volume(credentials, opts=nil)
+ ec2 = new_client(credentials)
+ opts ||= {}
+ opts[:snapshot_id] ||= ""# -> this causes signature validation error with Euca backend
+ opts[:capacity] ||= "1"
+ opts[:realm_id] ||= realms(credentials).first.id
+ safely do
+ convert_volume(ec2.create_volume(opts[:snapshot_id], opts[:capacity], opts[:realm_id]))
+
+ end
+ end
+
+ def destroy_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ unless ec2.delete_volume(opts[:id])
+ raise Deltacloud::BackendError.new(500, "StorageVolume", "Cannot delete storage volume")
+ end
+ storage_volume(credentials, opts[:id])
+ end
+ end
+
+ def attach_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_volume(ec2.attach_volume(opts[:id], opts[:instance_id], opts[:device]))
+ end
+ end
+
+ def detach_storage_volume(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_volume(ec2.detach_volume(opts[:id], opts[:instance_id], opts[:device], true))
+ end
+ end
+
+ def storage_snapshots(credentials, opts={})
+ ec2 = new_client(credentials)
+ snapshot_list = (opts and opts[:id]) ? opts[:id] : []
+ safely do
+ ec2.describe_snapshots(snapshot_list).collect do |snapshot|
+ convert_snapshot(snapshot)
+ end
+ end
+ end
+
+ def create_storage_snapshot(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ convert_snapshot(ec2.try_create_snapshot(opts[:volume_id]))
+ end
+ end
+
+ def destroy_storage_snapshot(credentials, opts={})
+ ec2 = new_client(credentials)
+ safely do
+ unless ec2.delete_snapshot(opts[:id])
+ raise Deltacloud::BackendError.new(500, "StorageSnapshot", "Cannot destroy this snapshot")
+ end
+ end
+ end
+
+ def valid_credentials?(credentials)
+ retval = true
+ begin
+ realms(credentials)
+ rescue Deltacloud::BackendError
+ retval = false
+ end
+ retval
+ end
+
+ private
+
+ def new_client(credentials, type = :ec2)
+ klass = case type
+ when :elb then Aws::Elb
+ when :ec2 then Aws::Ec2
+ when :s3 then Aws::S3
+ end
+ klass.new(credentials.user, credentials.password, eucalyptus_endpoint)
+#:server => endpoint_for_service(type))
+ end
+
+ def eucalyptus_endpoint
+ endpoint = (Thread.current[:provider] || ENV['API_PROVIDER'])
+ #parse endpoint string into server, port, service, and protocol
+ if endpoint
+ {:server => URI.parse(endpoint).host, :port => URI.parse(endpoint).port, :service => URI.parse(endpoint).path, :protocol => URI.parse(endpoint).scheme}
+ else
+ { } #EC2_URL env variable will be used by AWS
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def tag_instance(credentials, instance, name)
+ ec2 = new_client(credentials)
+ safely do
+ ec2.create_tag(instance.id, 'name', name)
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def untag_instance(credentials, instance_id)
+ ec2 = new_client(credentials)
+ safely do
+ ec2.delete_tag(instance_id, 'name')
+ end
+ end
+
+ # shouldn't be called now; eucalyptus doens't support tagging yet
+ def delete_unused_tags(credentials, inst_ids)
+ ec2 = new_client(credentials)
+ tags = []
+ safely do
+ tags = ec2.describe_tags('Filter.1.Name' => 'resource-type', 'Filter.1.Value' => 'instance')
+ tags.collect! { |t| t[:aws_resource_id] }
+ inst_ids.each do |inst_id|
+ unless tags.include?(inst_id)
+ ec2.delete_tag(inst_id, 'name')
+ end
+ end
+ end
+ end
+
+ def convert_bucket(s3_bucket)
+ #get blob list:
+ blob_list = []
+ s3_bucket.keys.each do |s3_object|
+ blob_list << s3_object.name
+ end
+
+ #can use AWS::S3::Owner.current.display_name or current.id
+ Bucket.new(
+ :id => s3_bucket.name,
+ :name => s3_bucket.name,
+ :size => s3_bucket.keys.length,
+ :blob_list => blob_list
+ )
+ end
+
+ def convert_object(s3_object)
+ Blob.new(
+ :id => s3_object.name,
+ :bucket => s3_object.bucket.name.to_s,
+ :content_length => s3_object.headers['content-length'],
+ :content_type => s3_object.headers['content-type'],
+ :last_modified => s3_object.last_modified,
+ :user_metadata => s3_object.meta_headers
+ )
+ end
+
+ def convert_realm(realm)
+ Realm.new(
+ :id => realm[:zone_name],
+ :name => realm[:zone_name],
+ :state => realm[:zone_state],
+ :limit => realm[:zone_state].eql?('available') ? :unlimited : 0
+ )
+ end
+
+ def convert_image(image)
+ # There is not support for 'name' for now
+ Image.new(
+ :id => image[:aws_id],
+ :name => image[:aws_name] || image[:aws_id],
+ :description => image[:aws_description] || image[:aws_location],
+ :owner_id => image[:aws_owner],
+ :architecture => image[:aws_architecture],
+ :state => image[:state]
+ )
+ end
+
+ def convert_instance(instance)
+ Instance.new(
+ :id => instance[:aws_instance_id],
+ :name => instance[:aws_image_id],
+ :state => convert_state(instance[:aws_state]),
+ :image_id => instance[:aws_image_id],
+ :owner_id => instance[:aws_owner],
+ :actions => instance_actions_for(convert_state(instance[:aws_state])),
+ :key_name => instance[:ssh_key_name],
+ :launch_time => instance[:aws_launch_time],
+ :instance_profile => InstanceProfile.new(instance[:aws_instance_type]),
+ :realm_id => instance[:aws_availability_zone],
+ :private_addresses => instance[:private_dns_name],
+ :public_addresses => instance[:dns_name]
+ )
+ end
+
+ def convert_key(key)
+ Key.new(
+ :id => key[:aws_key_name],
+ :fingerprint => key[:aws_fingerprint],
+ :credential_type => :key,
+ :pem_rsa_key => key[:aws_material],
+ :state => "AVAILABLE"
+ )
+ end
+
+ def convert_volume(volume)
+ StorageVolume.new(
+ :id => volume[:aws_id],
+ :created => volume[:aws_created_at],
+ :state => volume[:aws_status] ? volume[:aws_status].upcase : 'unknown',
+ :capacity => volume[:aws_size],
+ :instance_id => volume[:aws_instance_id],
+ :realm_id => volume[:zone],
+ :device => volume[:aws_device],
+ # TODO: the available actions should be tied to the current
+ # volume state
+ :actions => [:attach, :detach, :destroy]
+ )
+ end
+
+ def convert_snapshot(snapshot)
+ StorageSnapshot.new(
+ :id => snapshot[:aws_id],
+ :state => snapshot[:aws_status],
+ :storage_volume_id => snapshot[:aws_volume_id],
+ :created => snapshot[:aws_started_at]
+ )
+ end
+
+ def convert_load_balancer(credentials, loadbalancer)
+ puts loadbalancer.inspect
+ realms = []
+ balancer_realms = loadbalancer[:availability_zones].each do |zone|
+ realms << realm(credentials, zone)
+ end
+ balancer = LoadBalancer.new({
+ :id => loadbalancer[:name],
+ :created_at => loadbalancer[:created_time],
+ :public_addresses => [loadbalancer[:dns_name]],
+ :realms => realms
+ })
+ balancer.listeners = []
+ balancer.instances = []
+ loadbalancer[:listeners].each do |listener|
+ balancer.add_listener(listener)
+ end
+ loadbalancer[:instances].each do |instance|
+ balancer.instances << instance(credentials, :id => instance[:id])
+ end
+ balancer
+ end
+
+ def convert_state(ec2_state)
+ case ec2_state
+ when "terminated"
+ "STOPPED"
+ when "stopped"
+ "STOPPED"
+ when "running"
+ "RUNNING"
+ when "pending"
+ "PENDING"
+ when "shutting-down"
+ "STOPPED"
+ end
+ end
+
+ def catched_exceptions_list
+ {
+ :auth => [], # [ ::Aws::AuthFailure ],
+ :error => [ ::Aws::AwsError ],
+ :glob => [ /AWS::(\w+)/, /Deltacloud::Runner::(\w+)/ ]
+ }
+ end
+
+ end
+ end
+ end
+end
diff --git a/server/server.rb b/server/server.rb
index fe723c5..ff7c988 100644
--- a/server/server.rb
+++ b/server/server.rb
@@ -478,8 +478,8 @@ collection :storage_snapshots do
with_capability :destroy_storage_snapshot
param :id, :string, :required
control do
- driver.create_storage_snapshot(credentials, params)
- redirect(storage_snapshot_url(params[:id]))
+ driver.destroy_storage_snapshot(credentials, params)
+ redirect(storage_snapshots_url)
end
end
--
1.7.1