You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by "marios@redhat.com" <ma...@redhat.com> on 2011/08/31 18:22:50 UTC
Deltacloud dependencies
Writing docs and unless I've missed something I think we have issues
with our dependencies. We have a .gemspec as well as a Gemfile (this
isn't the problem per say...).
The Gemfile is fine and lists dependencies comprehensively, including
for all the external gems required to talk to the cloud providers (like
'aws', 'cloudfiles' etc). If you are setting up a dev environment and
building the gem, you can use 'bundle install --system' and you get
everything. All good.
However, the .gemspec only lists dependencies for the gems required to
run the deltacloud server (like 'sinatra', 'thin' etc). This means that
if you are just looking to download/use deltacloud and do a 'sudo gem
install deltacloud-core' you don't get the gems for talking to the cloud
providers. This will mean that 'deltacloudd -i ec2' gives a scary error
message about missing stuff. Not good for someone just trying out
deltacloud to see if its worth looking into.
Resolutions:
A) Add Gemfile dependencies to the .gemspec:
require 'bundler'
Gem::Specification.new do |s|
... (all the other dependencies and author, name etc)
s.add_bundler_dependencies
Problem with this approach is: 1) it adds a dependency for bundler,
before you can build/install the gem (no biggie really since our gemspec
already requires 'rake'). 2) 'add_bundler_dependencies' seems to have
been deprecated https://github.com/carlhuda/bundler/issues/614 3)
Gemfile has cloud provider dependencies in groups, and
add_bundler_dependencies doesn't include those
B) Add a full list of dependencies to the .gemspec like here
http://jeffkreeftmeijer.com/2010/lets-not-add_bundler_dependencies-anymore/
but then does that make Gemfile redundant?
I like B. What do other people think? Perhaps there isn't a problem? For
example, we can augment the Download instructions with text along the
lines of 'once you've installed deltacloud, you need to install gem X to
talk to cloud provider Y' but I think thats really counterproductive -
the whole point is to make getting and using deltacloud easy.
marios
Re: Deltacloud dependencies
Posted by Chris Lalancette <cl...@redhat.com>.
On 08/31/11 - 10:14:48AM, David Lutterkort wrote:
> > B) Add a full list of dependencies to the .gemspec like here
> > http://jeffkreeftmeijer.com/2010/lets-not-add_bundler_dependencies-anymore/
> > but then does that make Gemfile redundant?
> >
> > I like B. What do other people think? Perhaps there isn't a problem? For
> > example, we can augment the Download instructions with text along the
> > lines of 'once you've installed deltacloud, you need to install gem X to
> > talk to cloud provider Y' but I think thats really counterproductive -
> > the whole point is to make getting and using deltacloud easy.
>
> Yeah, I wouldn't want people to have to think too hard what drivers they
> need and then install the supporting gems manually. I would have
> preferred (A), since bundler is more precise about versions, but since
> that's deprecated upstream, we should go with (B).
Yeah, I also like B). If I'm understanding that webpage above, it should
simply be a matter of listing all of your dependencies in the .gemspec, and
then adding "gemspec" into your Gemfile. That way you get the best of both
worlds; you can "gem install deltacloud-core" and get all of the dependencies,
or if you are a developer you can "bundle install --system" to get them.
--
Chris Lalancette
Re: DeltaCloud Haml use
Posted by Tong Li <li...@us.ibm.com>.
David,
Thanks a lot for your information. I've tried simply using sinatra
and haml, I was able to direct the request to any format I would like to,
but when I started using DC framework (Rabbit), for some reason, haml
always looks for template with the following pattern
xml haml :"index" => index.xml.haml
json haml :"index" => index.json.haml
vs. pure Sinatra and Haml
xml haml :"index" => index.haml
json haml :"index" => index.haml
As I indicated in an earlier post, I would like to use xml as a base
to convert the xml response to other format requested, to be able to do
that, I need to be able to for request other than xml , make haml to deal
with xml template, then work with the response to produce other format.
Where and how can I change this behavior in DC?
Thanks.
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
From: David Lutterkort <lu...@redhat.com>
To: deltacloud-dev@incubator.apache.org
Date: 09/02/2011 06:25 PM
Subject: Re: DeltaCloud Haml use
On Fri, 2011-09-02 at 11:02 -0400, Tong Li wrote:
> When implementing RESTful APIs, often you have to support both XML and
JSON
> (sometimes also ATOM Feed), Ruby has nice functions to convert between
XML
> and JSON. In DeltaCloud, very often see code like this.
We used to convert directly from model objects to XML/JSON; that's kinda
dangerous, because any change in a model object would lead to a change
in the output XML, i.e. it was too easy to break the API with an
innocuous-looking code change.
That's why we switched to explicit templates, so that output changes
require a more conscious developer action.
Frankly, I hadn't thought about the route of generating JSON from XML;
it would require that you generate the XML, parse it into a DOM, and
then transform it to JSON. We haven't established fixed rules for the
XML -> JSON conversion, but it seems there's a natural mapping that
converts the DOM to a Hash:
dom_to_json(<elt attrs>body</elt>) =
{ "elt" => attrs.inject({}) { |h, k,v| h[k] = v }.merge
(convert(body)) }
dom_to_json(<elt1 ...>...</elt1> rest) =
h = convert(rest)
e = convert(<elt1 ...>...</elt1>)
if h["elt1"]
if h["elt1"].is_a?(Array)
h["elt1"] << e["elt1"]
else
h["elt1"] = [ h["elt1"], e["elt1"] ]
end
end
That of course only works if there's no conflict between attribute names
and the names of child elements; but that should be the case for the
Deltacloud XML.
With that, your typical respond_to block would then look like
respond_to do |format|
format.xml { haml :"index" }
format.html { haml :"index" }
format.json { xml_to_json(xml_parse(haml :"index", :format
=> :xml)) }
end
Of course, all that repetitive json mumbo jumbo should be hidden in a
helper.
BTW, be careful with Rails' Hash.from_xml - it drops attributes, e.g.:
>> Hash.from_xml("<elt attr='1'>body</elt>")
=> {"elt"=>"body"}
> This is to ensure that the json format contains absolutely
same
> information as in xml format and only provide one xml haml file, the
> problem is that now I do not know how to change the request so that the
> template searches for index.xml.haml rather than index.json.haml when
> request for json. I think there should be a way to do this, but within DC
> rabbit, I do not know if there is an easy way.
The 'haml' function actually comes from Sinatra[1]; you can specify a
format explicitly with 'haml :template, :format => :xml' etc.
> of course, hash data can be
> directly converted to json, but sometimes, the xml haml may take in data
> from more than one objects, it will be a lot easier to simply direct the
> request to xml and get the xml object then convert it to json or even
feed,
> Any pointer will be appreciated.
Yes, I really like this idea; do you think you could write a patch for
that ?
David
[1] http://www.sinatrarb.com/intro#Haml%20Templates
Re: syslog in DeltaCloud 0.4.0
Posted by "marios@redhat.com" <ma...@redhat.com>.
On 13/09/11 01:15, David Lutterkort wrote:
> On Mon, 2011-09-12 at 12:11 -0600, Tong Li wrote:
>> there were two screen captures inline in my email, can not see them in your
>> response (probably because your email system does not support inline
>> pictures), let me send as attachment.
>>
>> (See attached file: slow performance screen capture.png)
>
> The attachments didn't make it either - I wonder if ezmlm strips them
> out.
>
I think the apache list doesn't allow attachments; afaik the best way to
do this other than sending direct to recipients, is to open a jira and
attach them there,
marios
> David
>
>
Re: syslog in DeltaCloud 0.4.0
Posted by David Lutterkort <lu...@redhat.com>.
On Mon, 2011-09-12 at 12:11 -0600, Tong Li wrote:
> there were two screen captures inline in my email, can not see them in your
> response (probably because your email system does not support inline
> pictures), let me send as attachment.
>
> (See attached file: slow performance screen capture.png)
The attachments didn't make it either - I wonder if ezmlm strips them
out.
David
Re: syslog in DeltaCloud 0.4.0
Posted by Tong Li <li...@us.ibm.com>.
there were two screen captures inline in my email, can not see them in your
response (probably because your email system does not support inline
pictures), let me send as attachment.
(See attached file: slow performance screen capture.png)
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
From: David Lutterkort <lu...@redhat.com>
To: deltacloud-dev@incubator.apache.org
Date: 09/12/2011 01:43 PM
Subject: Re: syslog in DeltaCloud 0.4.0
On Mon, 2011-09-12 at 08:34 -0400, Tong Li wrote:
> I do not think using "lib" to reference anything is a good practice. It
> just causes problems. if not using lib, DC artifacts can be used by other
> gems. when some reference lib, then it actually won't work, problem
> described in item 2 is very easy to fix, simply remove "lib/", then it
will
> work just fine. If it is not an issue for DC, but it is not even a good
> practice beside the point of reuse.
Yes, I agree; and that will be fixed in due time. I just don't want to
spin another release candidate and hold another vote just for minor
fixes like this.
> for item3, when I used dc as a gem, used in my own gem, it did not work,
As I said, there is zero guarantee that using the dc server gem as a
library works.
> the method is undefined, even though integer.rb was required already, not
> sure what went wrong.
Hmm .. weird.
> for item4, the singularize method actually require i18n, can you check if
> your env actually has i18n, if you do, then you are probably using it
> without knowing you are.
This is from lib/deltacloud/core_ext/string.rb:
def singularize
return self.gsub(/es$/, '') if self =~ /sses$/
self.gsub(/s$/, '')
end
As you can tell, there's no dependency on i18n. As I said in my previous
mail, we do not use active_support or i18n, even though we have method
names in core_ext that follow commonly known method names from Rails.
> item 1 described in previous email was a bug. Please take a look.
As Michal explained, it's redundant code with no adverse effects. For an
array a, 'a.map' behaves the same as 'a.map { |x| x }', which is
perfectly fine for the use in views/instances/show.html.haml - certainly
something that should be cleaned up, but not a release stopper.
> for the slow performance, I can not be sure what is causing it. please
take
> a look at this firebug capture, not sure why the API GET instances took
> 25.07s, not sure why it references include.js from tb.adurr.com.
That is very strange - there is no such load for me; there's also no
reference to either include.js or adurr.com. It seems this isn't coming
from Deltacloud.
If you google for tb.adurr.com, you'll find various posts. I didn't
follow them enough to determine if this is benign or malware, but would
probably warrant closer follow-up.
> I am very
> sure that 0.3.0 won't take that long. Can not say it was because of thin
or
> webrick, or browser version, when I took the snap shot of firebug, I was
> using latest firebug and firefox.
Very strange.
> this is to get all the instances.
>
>
> Another one for rendering a particular instance.
Did you mean to attach more details ?
thanks,
David
Re: syslog in DeltaCloud 0.4.0
Posted by David Lutterkort <lu...@redhat.com>.
On Mon, 2011-09-12 at 08:34 -0400, Tong Li wrote:
> I do not think using "lib" to reference anything is a good practice. It
> just causes problems. if not using lib, DC artifacts can be used by other
> gems. when some reference lib, then it actually won't work, problem
> described in item 2 is very easy to fix, simply remove "lib/", then it will
> work just fine. If it is not an issue for DC, but it is not even a good
> practice beside the point of reuse.
Yes, I agree; and that will be fixed in due time. I just don't want to
spin another release candidate and hold another vote just for minor
fixes like this.
> for item3, when I used dc as a gem, used in my own gem, it did not work,
As I said, there is zero guarantee that using the dc server gem as a
library works.
> the method is undefined, even though integer.rb was required already, not
> sure what went wrong.
Hmm .. weird.
> for item4, the singularize method actually require i18n, can you check if
> your env actually has i18n, if you do, then you are probably using it
> without knowing you are.
This is from lib/deltacloud/core_ext/string.rb:
def singularize
return self.gsub(/es$/, '') if self =~ /sses$/
self.gsub(/s$/, '')
end
As you can tell, there's no dependency on i18n. As I said in my previous
mail, we do not use active_support or i18n, even though we have method
names in core_ext that follow commonly known method names from Rails.
> item 1 described in previous email was a bug. Please take a look.
As Michal explained, it's redundant code with no adverse effects. For an
array a, 'a.map' behaves the same as 'a.map { |x| x }', which is
perfectly fine for the use in views/instances/show.html.haml - certainly
something that should be cleaned up, but not a release stopper.
> for the slow performance, I can not be sure what is causing it. please take
> a look at this firebug capture, not sure why the API GET instances took
> 25.07s, not sure why it references include.js from tb.adurr.com.
That is very strange - there is no such load for me; there's also no
reference to either include.js or adurr.com. It seems this isn't coming
from Deltacloud.
If you google for tb.adurr.com, you'll find various posts. I didn't
follow them enough to determine if this is benign or malware, but would
probably warrant closer follow-up.
> I am very
> sure that 0.3.0 won't take that long. Can not say it was because of thin or
> webrick, or browser version, when I took the snap shot of firebug, I was
> using latest firebug and firefox.
Very strange.
> this is to get all the instances.
>
>
> Another one for rendering a particular instance.
Did you mean to attach more details ?
thanks,
David
Re: syslog in DeltaCloud 0.4.0
Posted by Tong Li <li...@us.ibm.com>.
I do not think using "lib" to reference anything is a good practice. It
just causes problems. if not using lib, DC artifacts can be used by other
gems. when some reference lib, then it actually won't work, problem
described in item 2 is very easy to fix, simply remove "lib/", then it will
work just fine. If it is not an issue for DC, but it is not even a good
practice beside the point of reuse.
for item3, when I used dc as a gem, used in my own gem, it did not work,
the method is undefined, even though integer.rb was required already, not
sure what went wrong.
for item4, the singularize method actually require i18n, can you check if
your env actually has i18n, if you do, then you are probably using it
without knowing you are.
item 1 described in previous email was a bug. Please take a look.
for the slow performance, I can not be sure what is causing it. please take
a look at this firebug capture, not sure why the API GET instances took
25.07s, not sure why it references include.js from tb.adurr.com. I am very
sure that 0.3.0 won't take that long. Can not say it was because of thin or
webrick, or browser version, when I took the snap shot of firebug, I was
using latest firebug and firefox.
this is to get all the instances.
Another one for rendering a particular instance.
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
From: David Lutterkort <lu...@redhat.com>
To: deltacloud-dev@incubator.apache.org
Date: 09/09/2011 02:22 PM
Subject: Re: syslog in DeltaCloud 0.4.0
On Wed, 2011-09-07 at 17:09 -0400, Tong Li wrote:
> 2. incorrectly use "lib" in the require path, this causes big problem for
> others to use DC gem. There are two places
>
> base_driver/base_driver.rb
> sinatra/rack_syslog.rb
>
> both files have some thing like this
>
> require 'lib/xxxxx'
>
> They should be changed to
>
> require 'xxxxx'
The Deltacloud server is _not_ meant to be used as a library. There is
absolutely no guarantee about any sort of API stability for any of the
ruby code; the stable API for Deltacloud is the RESTful web API.
Therefore, the require above is not an issue.
> 3. In base_driver/features.rb, it still uses ordinalize method from an
> integer, which has been removed in newer active_support package,
We do not depend on active_support. Integer#ordinalize is defined in
lib/deltacloud/core_ext/integer.rb - for some of these core_ext's we do
use names familiar from active_support/Rails, but define them in our own
code.
> 4. i18n is a dependency , but was not stated in the spec.
Where do you see that dependency ? In the public instance I just set up
from scratch, i18n is not installed, and everything works. I can also
not find any reference to i18n in the code.
> 5. nokogiri requires some other dependency which is not available on 64
> bits platform, forgot the name of it, when I tried to gem install, it did
a
> local build, then blows up because it was trying to use 32 libraries on
> windows.
I am unfortunately not familiar with the intricacies of multilib on
Windows. Could you dig up what exactly blows up ? Could there be a
problem if you already have 32 bit libxml2/libxslt installed and then
try to build 64 bit nokogiri ?
> 6. syslog is not available on windows platform, both item 5 and 6 really
> hurts. I had to setup an Ubuntu 11.04 machine to do my development.
Yes, that needs to be addressed.
> 7. new RC4 is extremely slow. I run 0.4.0 and 0.3.0 on the same machine,
> 0.4.0 took a lot longer from user point view. not sure if it has anything
> to do with the newer UI or because of the thin server which is now
> required. Any reason why switch it from WEBrick?
When using native ruby, thin has always been the default web server.
WEBrick is generally not meant for production uses since it's very slow,
single-threaded etc.
Is there any way you can quantify the slow down you see in the UI, e.g.
with Firebug's 'Net' view - the jquery UI introduces a good bit of
Javascript; that's also the biggest change that might affect
performance.
> is there any way these problems can be fixed? or they have been fixed in
> newer builds? If I can access the source, I do not mind to fix them. I
> think that problems described in 1 to 4 should really be fixed ASAP.
I think none of 1 through 4 are actual issues. Definitely, we need to
address the syslog issue, and the slowness you see. But I think that can
be done in the next release.
David
Re: syslog in DeltaCloud 0.4.0
Posted by David Lutterkort <lu...@redhat.com>.
On Wed, 2011-09-07 at 17:09 -0400, Tong Li wrote:
> 2. incorrectly use "lib" in the require path, this causes big problem for
> others to use DC gem. There are two places
>
> base_driver/base_driver.rb
> sinatra/rack_syslog.rb
>
> both files have some thing like this
>
> require 'lib/xxxxx'
>
> They should be changed to
>
> require 'xxxxx'
The Deltacloud server is _not_ meant to be used as a library. There is
absolutely no guarantee about any sort of API stability for any of the
ruby code; the stable API for Deltacloud is the RESTful web API.
Therefore, the require above is not an issue.
> 3. In base_driver/features.rb, it still uses ordinalize method from an
> integer, which has been removed in newer active_support package,
We do not depend on active_support. Integer#ordinalize is defined in
lib/deltacloud/core_ext/integer.rb - for some of these core_ext's we do
use names familiar from active_support/Rails, but define them in our own
code.
> 4. i18n is a dependency , but was not stated in the spec.
Where do you see that dependency ? In the public instance I just set up
from scratch, i18n is not installed, and everything works. I can also
not find any reference to i18n in the code.
> 5. nokogiri requires some other dependency which is not available on 64
> bits platform, forgot the name of it, when I tried to gem install, it did a
> local build, then blows up because it was trying to use 32 libraries on
> windows.
I am unfortunately not familiar with the intricacies of multilib on
Windows. Could you dig up what exactly blows up ? Could there be a
problem if you already have 32 bit libxml2/libxslt installed and then
try to build 64 bit nokogiri ?
> 6. syslog is not available on windows platform, both item 5 and 6 really
> hurts. I had to setup an Ubuntu 11.04 machine to do my development.
Yes, that needs to be addressed.
> 7. new RC4 is extremely slow. I run 0.4.0 and 0.3.0 on the same machine,
> 0.4.0 took a lot longer from user point view. not sure if it has anything
> to do with the newer UI or because of the thin server which is now
> required. Any reason why switch it from WEBrick?
When using native ruby, thin has always been the default web server.
WEBrick is generally not meant for production uses since it's very slow,
single-threaded etc.
Is there any way you can quantify the slow down you see in the UI, e.g.
with Firebug's 'Net' view - the jquery UI introduces a good bit of
Javascript; that's also the biggest change that might affect
performance.
> is there any way these problems can be fixed? or they have been fixed in
> newer builds? If I can access the source, I do not mind to fix them. I
> think that problems described in 1 to 4 should really be fixed ASAP.
I think none of 1 through 4 are actual issues. Definitely, we need to
address the syslog issue, and the slowness you see. But I think that can
be done in the next release.
David
Re: syslog in DeltaCloud 0.4.0
Posted by Francesco Vollero <ra...@gmail.com>.
Hey Tong,
Tomorrow morning i gonna fix those problems, the ones that still in bug state. I gonna give back here on ml asap with the status.
Cheers,
Francesco
Sent from my iPad
On 07/set/2011, at 23:09, Tong Li <li...@us.ibm.com> wrote:
> Tried DC RC 4 with ruby 1.9.1 and latest dependencies and found quite few problems.
>
> The RC4 was downloaded here and seemed it was posted on 8/24.
> http://people.apache.org/~lutter/deltacloud/0.4.0/rc4/
>
> 1. This should be just a bug.
> views/instances/show.html.haml has errors, it should have never worked.
>
> both line 31 and 34 has code like this.
>
> %p{ :'data-role' => 'fieldcontain'}=@instance.public_addresses.map.join(',')
>
> the map.join should not have worked.
>
> it needs to be changed to
>
> %p{ :'data-role' => 'fieldcontain'}=@instance.public_addresses.join(',')
>
> 2. incorrectly use "lib" in the require path, this causes big problem for others to use DC gem. There are two places
>
> base_driver/base_driver.rb
> sinatra/rack_syslog.rb
>
> both files have some thing like this
>
> require 'lib/xxxxx'
>
> They should be changed to
>
> require 'xxxxx'
> 3. In base_driver/features.rb, it still uses ordinalize method from an integer, which has been removed in newer active_support package,
> 4. i18n is a dependency , but was not stated in the spec.
> 5. nokogiri requires some other dependency which is not available on 64 bits platform, forgot the name of it, when I tried to gem install, it did a local build, then blows up because it was trying to use 32 libraries on windows.
> 6. syslog is not available on windows platform, both item 5 and 6 really hurts. I had to setup an Ubuntu 11.04 machine to do my development.
> 7. new RC4 is extremely slow. I run 0.4.0 and 0.3.0 on the same machine, 0.4.0 took a lot longer from user point view. not sure if it has anything to do with the newer UI or because of the thin server which is now required. Any reason why switch it from WEBrick?
>
> is there any way these problems can be fixed? or they have been fixed in newer builds? If I can access the source, I do not mind to fix them. I think that problems described in 1 to 4 should really be fixed ASAP.
>
> Thanks.
>
> Tong Li
> Emerging Technologies & Standards
> B062/K317
> litong01@us.ibm.com
>
> David Lutterkort ---09/06/2011 07:17:45 PM---On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote: > finally had time to try this rc4, however, syslo
>
> From: David Lutterkort <lu...@redhat.com>
> To: deltacloud-dev@incubator.apache.org
> Date: 09/06/2011 07:17 PM
> Subject: Re: syslog in DeltaCloud 0.4.0
>
>
>
> On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
> > finally had time to try this rc4, however, syslog is now needed by few
> > files in DC. unfortunately syslog is not available on windows platform, so
> > somehow this release becomes platform specific. is this really intended? am
> > I missing something?
>
> No, the intent is not to make it platform specific - we need to figure
> out how to replace syslog on Windows. Should probably fall back to
> writing to stdout. Or do you know of a more elegant fallback ?
>
> David
>
>
>
Re: syslog in DeltaCloud 0.4.0
Posted by Michal Fojtik <mf...@redhat.com>.
On Sep 7, 2011, at 11:09 PM, Tong Li wrote:
> Tried DC RC 4 with ruby 1.9.1 and latest dependencies and found quite few problems.
>
> The RC4 was downloaded here and seemed it was posted on 8/24.
> http://people.apache.org/~lutter/deltacloud/0.4.0/rc4/
>
> 1. This should be just a bug.
> views/instances/show.html.haml has errors, it should have never worked.
>
> both line 31 and 34 has code like this.
>
> %p{ :'data-role' => 'fieldcontain'}=@instance.public_addresses.map.join(',')
>
> the map.join should not have worked.
It works:
irb(main):001:0> [1,2,3,4].map.join(',')
=> "1,2,3,4"
However you raised a valid point, because the map there is completely meaningless in this
context and should be removed.
> it needs to be changed to
>
> %p{ :'data-role' => 'fieldcontain'}=@instance.public_addresses.join(',')
>
> 2. incorrectly use "lib" in the require path, this causes big problem for others to use DC gem. There are two places
>
> base_driver/base_driver.rb
> sinatra/rack_syslog.rb
>
> both files have some thing like this
>
> require 'lib/xxxxx'
>
> They should be changed to
>
> require 'xxxxx'
> 3. In base_driver/features.rb, it still uses ordinalize method from an integer, which has been removed in newer active_support package,
AFAIK we should be 'active_support' free. If we're using this method it must be defined locally or be part
of integer class in 1.8.7.
> 4. i18n is a dependency , but was not stated in the spec.
This is a 'active_support' dependency which again, should be not required anymore. If it is, it's a bug.
Btw. what driver you're using? EC2?
> 5. nokogiri requires some other dependency which is not available on 64 bits platform, forgot the name of it, when I tried to gem install, it did a local build, then blows up because it was trying to use 32 libraries on windows.
> 6. syslog is not available on windows platform, both item 5 and 6 really hurts. I had to setup an Ubuntu 11.04 machine to do my development.
Sorry for this dependency, I did not realized that it should be a problem on Windows. I think we should subclass
syslog class and add fallback to windows registry instead. However for development I though syslog is turned off
by default.
> 7. new RC4 is extremely slow. I run 0.4.0 and 0.3.0 on the same machine, 0.4.0 took a lot longer from user point view. not sure if it has anything to do with the newer UI or because of the thin server which is now required. Any reason why switch it from WEBrick?
Thin server was required from the start. If deltacloud doesn't have this gem installed it fall back to Webrick.
Could you be more descriptive what is slow? From 0.3.0 we have special HTTP header 'X-Runtime' in response headers,
that hold information about how long did it take to generate a response.
If those times differs a lot from 0.3.0 it looks like a bug.
> is there any way these problems can be fixed? or they have been fixed in newer builds? If I can access the source, I do not mind to fix them. I think that problems described in 1 to 4 should really be fixed ASAP.
I'll look on them ASAP. Probably best way how to report bugs is our JIRA (https://issues.apache.org/jira)
where you can then track status of the bug.
-- Michal
>
>
> David Lutterkort ---09/06/2011 07:17:45 PM---On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote: > finally had time to try this rc4, however, syslo
>
> From: David Lutterkort <lu...@redhat.com>
> To: deltacloud-dev@incubator.apache.org
> Date: 09/06/2011 07:17 PM
> Subject: Re: syslog in DeltaCloud 0.4.0
>
>
>
> On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
> > finally had time to try this rc4, however, syslog is now needed by few
> > files in DC. unfortunately syslog is not available on windows platform, so
> > somehow this release becomes platform specific. is this really intended? am
> > I missing something?
>
> No, the intent is not to make it platform specific - we need to figure
> out how to replace syslog on Windows. Should probably fall back to
> writing to stdout. Or do you know of a more elegant fallback ?
>
> David
>
>
>
------------------------------------------------------
Michal Fojtik, mfojtik@redhat.com
Deltacloud API: http://deltacloud.org
Re: syslog in DeltaCloud 0.4.0
Posted by Tong Li <li...@us.ibm.com>.
Tried DC RC 4 with ruby 1.9.1 and latest dependencies and found quite few
problems.
The RC4 was downloaded here and seemed it was posted on 8/24.
http://people.apache.org/~lutter/deltacloud/0.4.0/rc4/
1. This should be just a bug.
views/instances/show.html.haml has errors, it should have never
worked.
both line 31 and 34 has code like this.
%p{ :'data-role' => 'fieldcontain'
}=@instance.public_addresses.map.join(',')
the map.join should not have worked.
it needs to be changed to
%p{ :'data-role' => 'fieldcontain'}=@instance.public_addresses.join
(',')
2. incorrectly use "lib" in the require path, this causes big problem for
others to use DC gem. There are two places
base_driver/base_driver.rb
sinatra/rack_syslog.rb
both files have some thing like this
require 'lib/xxxxx'
They should be changed to
require 'xxxxx'
3. In base_driver/features.rb, it still uses ordinalize method from an
integer, which has been removed in newer active_support package,
4. i18n is a dependency , but was not stated in the spec.
5. nokogiri requires some other dependency which is not available on 64
bits platform, forgot the name of it, when I tried to gem install, it did a
local build, then blows up because it was trying to use 32 libraries on
windows.
6. syslog is not available on windows platform, both item 5 and 6 really
hurts. I had to setup an Ubuntu 11.04 machine to do my development.
7. new RC4 is extremely slow. I run 0.4.0 and 0.3.0 on the same machine,
0.4.0 took a lot longer from user point view. not sure if it has anything
to do with the newer UI or because of the thin server which is now
required. Any reason why switch it from WEBrick?
is there any way these problems can be fixed? or they have been fixed in
newer builds? If I can access the source, I do not mind to fix them. I
think that problems described in 1 to 4 should really be fixed ASAP.
Thanks.
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
From: David Lutterkort <lu...@redhat.com>
To: deltacloud-dev@incubator.apache.org
Date: 09/06/2011 07:17 PM
Subject: Re: syslog in DeltaCloud 0.4.0
On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
> finally had time to try this rc4, however, syslog is now needed by few
> files in DC. unfortunately syslog is not available on windows platform,
so
> somehow this release becomes platform specific. is this really intended?
am
> I missing something?
No, the intent is not to make it platform specific - we need to figure
out how to replace syslog on Windows. Should probably fall back to
writing to stdout. Or do you know of a more elegant fallback ?
David
Re: syslog in DeltaCloud 0.4.0
Posted by David Lutterkort <lu...@redhat.com>.
That would be great.
David
On Wed, 2011-09-07 at 02:08 +0200, Francesco Vollero wrote:
> We(I) can create a module that log to windows log registry as a daemon
> should do. This are my two cents.
>
> Francesco
> On Sep 7, 2011 1:15 AM, "David Lutterkort" <lu...@redhat.com> wrote:
> > On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
> >> finally had time to try this rc4, however, syslog is now needed by few
> >> files in DC. unfortunately syslog is not available on windows platform,
> so
> >> somehow this release becomes platform specific. is this really intended?
> am
> >> I missing something?
> >
> > No, the intent is not to make it platform specific - we need to figure
> > out how to replace syslog on Windows. Should probably fall back to
> > writing to stdout. Or do you know of a more elegant fallback ?
> >
> > David
> >
> >
Re: syslog in DeltaCloud 0.4.0
Posted by Francesco Vollero <ra...@gmail.com>.
We(I) can create a module that log to windows log registry as a daemon
should do. This are my two cents.
Francesco
On Sep 7, 2011 1:15 AM, "David Lutterkort" <lu...@redhat.com> wrote:
> On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
>> finally had time to try this rc4, however, syslog is now needed by few
>> files in DC. unfortunately syslog is not available on windows platform,
so
>> somehow this release becomes platform specific. is this really intended?
am
>> I missing something?
>
> No, the intent is not to make it platform specific - we need to figure
> out how to replace syslog on Windows. Should probably fall back to
> writing to stdout. Or do you know of a more elegant fallback ?
>
> David
>
>
Re: syslog in DeltaCloud 0.4.0
Posted by David Lutterkort <lu...@redhat.com>.
On Tue, 2011-09-06 at 15:40 -0400, Tong Li wrote:
> finally had time to try this rc4, however, syslog is now needed by few
> files in DC. unfortunately syslog is not available on windows platform, so
> somehow this release becomes platform specific. is this really intended? am
> I missing something?
No, the intent is not to make it platform specific - we need to figure
out how to replace syslog on Windows. Should probably fall back to
writing to stdout. Or do you know of a more elegant fallback ?
David
syslog in DeltaCloud 0.4.0
Posted by Tong Li <li...@us.ibm.com>.
finally had time to try this rc4, however, syslog is now needed by few
files in DC. unfortunately syslog is not available on windows platform, so
somehow this release becomes platform specific. is this really intended? am
I missing something?
Thanks.
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
Re: DeltaCloud Haml use
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2011-09-02 at 11:02 -0400, Tong Li wrote:
> When implementing RESTful APIs, often you have to support both XML and JSON
> (sometimes also ATOM Feed), Ruby has nice functions to convert between XML
> and JSON. In DeltaCloud, very often see code like this.
We used to convert directly from model objects to XML/JSON; that's kinda
dangerous, because any change in a model object would lead to a change
in the output XML, i.e. it was too easy to break the API with an
innocuous-looking code change.
That's why we switched to explicit templates, so that output changes
require a more conscious developer action.
Frankly, I hadn't thought about the route of generating JSON from XML;
it would require that you generate the XML, parse it into a DOM, and
then transform it to JSON. We haven't established fixed rules for the
XML -> JSON conversion, but it seems there's a natural mapping that
converts the DOM to a Hash:
dom_to_json(<elt attrs>body</elt>) =
{ "elt" => attrs.inject({}) { |h, k,v| h[k] = v }.merge(convert(body)) }
dom_to_json(<elt1 ...>...</elt1> rest) =
h = convert(rest)
e = convert(<elt1 ...>...</elt1>)
if h["elt1"]
if h["elt1"].is_a?(Array)
h["elt1"] << e["elt1"]
else
h["elt1"] = [ h["elt1"], e["elt1"] ]
end
end
That of course only works if there's no conflict between attribute names
and the names of child elements; but that should be the case for the
Deltacloud XML.
With that, your typical respond_to block would then look like
respond_to do |format|
format.xml { haml :"index" }
format.html { haml :"index" }
format.json { xml_to_json(xml_parse(haml :"index", :format => :xml)) }
end
Of course, all that repetitive json mumbo jumbo should be hidden in a
helper.
BTW, be careful with Rails' Hash.from_xml - it drops attributes, e.g.:
>> Hash.from_xml("<elt attr='1'>body</elt>")
=> {"elt"=>"body"}
> This is to ensure that the json format contains absolutely same
> information as in xml format and only provide one xml haml file, the
> problem is that now I do not know how to change the request so that the
> template searches for index.xml.haml rather than index.json.haml when
> request for json. I think there should be a way to do this, but within DC
> rabbit, I do not know if there is an easy way.
The 'haml' function actually comes from Sinatra[1]; you can specify a
format explicitly with 'haml :template, :format => :xml' etc.
> of course, hash data can be
> directly converted to json, but sometimes, the xml haml may take in data
> from more than one objects, it will be a lot easier to simply direct the
> request to xml and get the xml object then convert it to json or even feed,
> Any pointer will be appreciated.
Yes, I really like this idea; do you think you could write a patch for
that ?
David
[1] http://www.sinatrarb.com/intro#Haml%20Templates
DeltaCloud Haml use
Posted by Tong Li <li...@us.ibm.com>.
When implementing RESTful APIs, often you have to support both XML and JSON
(sometimes also ATOM Feed), Ruby has nice functions to convert between XML
and JSON. In DeltaCloud, very often see code like this.
respond_to do |format|
format.html {haml :"index" }
format.xml {haml :"index" }
format.json { haml :"index" }
end
is there anyway to use some thing like this.
respond_to do |format|
format.html {haml :"index" }
format.xml {haml :"index" }
format.json do
get the xml data,
then convert it to json, by using Hash.from_xml(the
stream).to_json
end
end
This is to ensure that the json format contains absolutely same
information as in xml format and only provide one xml haml file, the
problem is that now I do not know how to change the request so that the
template searches for index.xml.haml rather than index.json.haml when
request for json. I think there should be a way to do this, but within DC
rabbit, I do not know if there is an easy way. of course, hash data can be
directly converted to json, but sometimes, the xml haml may take in data
from more than one objects, it will be a lot easier to simply direct the
request to xml and get the xml object then convert it to json or even feed,
Any pointer will be appreciated.
Thanks.
Tong Li
Emerging Technologies & Standards
B062/K317
litong01@us.ibm.com
Re: Deltacloud dependencies
Posted by David Lutterkort <lu...@redhat.com>.
On Wed, 2011-08-31 at 19:22 +0300, marios@redhat.com wrote:
> However, the .gemspec only lists dependencies for the gems required to
> run the deltacloud server (like 'sinatra', 'thin' etc). This means that
> if you are just looking to download/use deltacloud and do a 'sudo gem
> install deltacloud-core' you don't get the gems for talking to the cloud
> providers. This will mean that 'deltacloudd -i ec2' gives a scary error
> message about missing stuff. Not good for someone just trying out
> deltacloud to see if its worth looking into.
What a pickle. I agree that we want people to only have to run 'gem
install deltacloud-core' and be able to run their favorite driver; we
could do what we do for the rpm's and have sub-gems for each of the
driver, but that might be overkill.
> Resolutions:
>
> A) Add Gemfile dependencies to the .gemspec:
>
> require 'bundler'
> Gem::Specification.new do |s|
> ... (all the other dependencies and author, name etc)
> s.add_bundler_dependencies
>
> Problem with this approach is: 1) it adds a dependency for bundler,
> before you can build/install the gem (no biggie really since our gemspec
> already requires 'rake'). 2) 'add_bundler_dependencies' seems to have
> been deprecated https://github.com/carlhuda/bundler/issues/614 3)
> Gemfile has cloud provider dependencies in groups, and
> add_bundler_dependencies doesn't include those
>
> B) Add a full list of dependencies to the .gemspec like here
> http://jeffkreeftmeijer.com/2010/lets-not-add_bundler_dependencies-anymore/
> but then does that make Gemfile redundant?
>
> I like B. What do other people think? Perhaps there isn't a problem? For
> example, we can augment the Download instructions with text along the
> lines of 'once you've installed deltacloud, you need to install gem X to
> talk to cloud provider Y' but I think thats really counterproductive -
> the whole point is to make getting and using deltacloud easy.
Yeah, I wouldn't want people to have to think too hard what drivers they
need and then install the supporting gems manually. I would have
preferred (A), since bundler is more precise about versions, but since
that's deprecated upstream, we should go with (B).
David