You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by lu...@redhat.com on 2011/11/09 22:18:22 UTC
Model layer for CIMI
This is a model layer for CIMI that does both serialization and
deserialization of JSON and XML based on a simple metadata description of
the CIMI model expressed in a custom DSL. In particular, we won't need HAML
templates to format objects, and I am reasonably sure that the metadata
makes sure we serialize in a predictable manner to XML.
Patch 1/2 contains the basics, including the DSL, 2/2 shows how this can be
used for a specific class.
There's a few convenience additions that should be made to the DSL, in
particular making it possible to reuse nested structures that are used in
several places, something like
array :operations, :class => Operation
but we can add that later.
One small fly in the ointment is that the use of XmlSimple causes child
elements in XML to be rendered in an unpredictable order; the CIMI standard
doesn't say anything about reordering elements, though with a strict
interpretation of XML, the rendered XML does not conform to the XML schema
implicitly defined in the CIMI standard.
David
[PATCH 2/2] CIMI: model for machineTemplate
Posted by lu...@redhat.com.
From: David Lutterkort <lu...@redhat.com>
Signed-off-by: David Lutterkort <lu...@redhat.com>
---
server/lib/cimi/model/machine_template.rb | 36 +++++++++
server/spec/cimi/data/machine_template.json | 30 ++++++++
server/spec/cimi/data/machine_template.xml | 24 ++++++
server/spec/cimi/model/machine_template_spec.rb | 51 +++++++++++++
server/spec/spec_helper.rb | 92 +++++++++++++++++++++++
5 files changed, 233 insertions(+), 0 deletions(-)
create mode 100644 server/lib/cimi/model/machine_template.rb
create mode 100644 server/spec/cimi/data/machine_template.json
create mode 100644 server/spec/cimi/data/machine_template.xml
create mode 100644 server/spec/cimi/model/machine_template_spec.rb
diff --git a/server/lib/cimi/model/machine_template.rb b/server/lib/cimi/model/machine_template.rb
new file mode 100644
index 0000000..458f365
--- /dev/null
+++ b/server/lib/cimi/model/machine_template.rb
@@ -0,0 +1,36 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+class CIMI::Model::MachineTemplate < CIMI::Model::Base
+ href :machine_config
+ href :machine_image
+ href :machine_admin
+ array :volumes do
+ scalar :href
+ scalar :protocol
+ scalar :attachment_point
+ end
+ array :volume_templates do
+ scalar :href, :attachment_point, :protocol
+ end
+ array :network_interfaces do
+ href :vsp
+ text :hostname, :mac_address, :state, :protocol, :allocation
+ text :address, :default_gateway, :dns, :max_transmission_unit
+ end
+ array :operations do
+ scalar :rel, :href
+ end
+end
diff --git a/server/spec/cimi/data/machine_template.json b/server/spec/cimi/data/machine_template.json
new file mode 100644
index 0000000..54de58d
--- /dev/null
+++ b/server/spec/cimi/data/machine_template.json
@@ -0,0 +1,30 @@
+{
+ "uri": "http://cimi.example.org/machine_templates/1",
+ "name": "My First Template",
+ "description": "A template for testing",
+ "created": "2011-11-01",
+ "machineConfig": { "href": "http://cimi.example.org/machine_configs/1" },
+ "machineImage": { "href": "http://cimi.example.org/machine_images/1" },
+ "volumes": [{
+ "href": "http://cimi.example.org/volumes/1",
+ "attachmentPoint": "/dev/sda",
+ "protocol": "nfs"
+ }],
+ "networkInterfaces": [{
+ "vsp": { "href": "http://cimi.example.org/vsps/1" },
+ "hostname": "host.cimi.example.org",
+ "macAddress": "00:11:22:33:44:55",
+ "state": "UP",
+ "protocol": "TCP",
+ "allocation": "static",
+ "address": "192.168.0.17",
+ "defaultGateway": "192.168.0.1",
+ "dns": "192.168.0.1",
+ "maxTransmissionUnit": "1500"
+ }],
+ "operations": [
+ { "rel": "edit",
+ "href": "http://cimi.example.org/machine_templates/1/edit" },
+ { "rel": "delete",
+ "href": "http://cimi.example.org/machine_templates/1/delete" }]
+}
diff --git a/server/spec/cimi/data/machine_template.xml b/server/spec/cimi/data/machine_template.xml
new file mode 100644
index 0000000..160b311
--- /dev/null
+++ b/server/spec/cimi/data/machine_template.xml
@@ -0,0 +1,24 @@
+<MachineTemplate xmlns="http://www.dmtf.org/cimi">
+ <uri>http://cimi.example.org/machine_templates/1</uri>
+ <name>My First Template</name>
+ <description>A template for testing</description>
+ <created>2011-11-01</created>
+ <machineConfig href="http://cimi.example.org/machine_configs/1"/>
+ <machineImage href="http://cimi.example.org/machine_images/1"/>
+ <volume href="http://cimi.example.org/volumes/1"
+ attachmentPoint="/dev/sda" protocol="nfs" />
+ <networkInterface>
+ <vsp href="http://cimi.example.org/vsps/1"/>
+ <hostname>host.cimi.example.org</hostname>
+ <macAddress>00:11:22:33:44:55</macAddress>
+ <state>UP</state>
+ <protocol>TCP</protocol>
+ <allocation>static</allocation>
+ <address>192.168.0.17</address>
+ <defaultGateway>192.168.0.1</defaultGateway>
+ <dns>192.168.0.1</dns>
+ <maxTransmissionUnit>1500</maxTransmissionUnit>
+ </networkInterface>
+ <operation rel="edit" href="http://cimi.example.org/machine_templates/1/edit"/>
+ <operation rel="delete" href="http://cimi.example.org/machine_templates/1/delete"/>
+</MachineTemplate>
diff --git a/server/spec/cimi/model/machine_template_spec.rb b/server/spec/cimi/model/machine_template_spec.rb
new file mode 100644
index 0000000..d7028a6
--- /dev/null
+++ b/server/spec/cimi/model/machine_template_spec.rb
@@ -0,0 +1,51 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+require 'spec_helper'
+
+require 'cimi/model'
+
+describe "MachineTemplate model" do
+
+ DATA_DIR = File::join(File::dirname(__FILE__), '..', 'data')
+
+ before(:all) do
+ @xml = IO::read(File::join(DATA_DIR, "machine_template.xml"))
+ @json = IO::read(File::join(DATA_DIR, "machine_template.json"))
+ end
+
+ it "can be constructed from XML" do
+ templ = CIMI::Model::MachineTemplate.from_xml(@xml)
+ templ.should_not be_nil
+ templ.created.should == "2011-11-01"
+ templ.volumes.size.should == 1
+ templ.should serialize_to @xml, :fmt => :xml
+ end
+
+ it "should convert strings in keys to symbols when contructed from XML" do
+ templ = CIMI::Model::MachineTemplate.from_xml(@xml)
+ templ.should_not be_nil
+ templ.attribute_values.keys.each { |key| key.should be_a(Symbol) }
+ end
+
+ it "can be constructed from JSON" do
+ templ = CIMI::Model::MachineTemplate.from_json(@json)
+ templ.should_not be_nil
+
+ templ.created.should == "2011-11-01"
+ templ.should serialize_to @json, :fmt => :json
+ end
+end
diff --git a/server/spec/spec_helper.rb b/server/spec/spec_helper.rb
index 1e0dd87..d9a51f5 100644
--- a/server/spec/spec_helper.rb
+++ b/server/spec/spec_helper.rb
@@ -25,3 +25,95 @@ def parse_xml(xml, opts = {})
opts[:keep_root] = true unless opts.has_key?(:keep_root)
XmlSimple.xml_in(xml, opts)
end
+
+class HashCmp
+ def initialize(exp, act)
+ @exp = exp
+ @act = act
+ @io = StringIO.new
+ end
+
+ def match?
+ @equal = true
+ compare_values(@exp, @act, [])
+ @equal
+ end
+
+ def errors
+ @io.string
+ end
+
+ private
+ def compare_values(exp, act, path)
+ if exp.is_a?(String)
+ mismatch("entries differ", exp, act, path) unless exp == act
+ elsif exp.is_a?(Array)
+ mismatch("expected array", exp, act, path) unless act.is_a?(Array)
+ unless act.size == exp.size
+ mismatch("different array lengths", exp, act, path)
+ end
+ name = path.pop
+ 0.upto(exp.size-1) do |i|
+ compare_values(exp[i], act[i], path + [ "#{name}[#{i}]" ])
+ end
+ elsif exp.is_a?(Hash)
+ unless act.is_a?(Hash)
+ mismatch("expected Hash", exp, act, path)
+ return
+ end
+ unless (missing = exp.keys - act.keys).empty?
+ error "Missing key(s) at /#{path.join("/")}: #{missing.inspect}"
+ end
+ unless (excess = act.keys - exp.keys).empty?
+ error "Excess key(s) at /#{path.join("/")}: #{excess.inspect}"
+ end
+ (exp.keys - missing - excess).each do |k|
+ compare_values(exp[k], act[k], path + [ k ])
+ end
+ end
+ end
+
+ def mismatch(msg, exp, act, path)
+ error "#{msg}[#{fmt(path)}]: #{exp.inspect} != #{act.inspect}"
+ end
+
+ def error(msg)
+ @equal = false
+ @io.puts msg
+ end
+
+ def fmt(path)
+ "/#{path.join("/")}"
+ end
+end
+
+RSpec::Matchers.define :serialize_to do |exp, opts|
+ match do |act|
+ matcher(exp, act, opts[:fmt]).match?
+ end
+
+ failure_message_for_should do |act|
+ m = matcher(exp, act, opts[:fmt])
+ m.match?
+ "#{opts[:fmt].to_s.upcase} documents do not match\n" + m.errors
+ end
+
+ def matcher(exp, act, fmt)
+ raise "missing format; use :fmt => [:xml || :json]" if fmt.nil?
+ exp, act = [exp, act].map { |x| convert(x, fmt) }
+ HashCmp.new(exp, act)
+ end
+
+ def convert(x, fmt)
+ if fmt == :json
+ x = x.to_json if x.is_a?(CIMI::Model::Base)
+ x = JSON.parse(x) if x.is_a?(String)
+ elsif fmt == :xml
+ x = x.to_xml if x.is_a?(CIMI::Model::Base)
+ x = parse_xml(x) if x.is_a?(String)
+ else
+ raise "Invalid format #{fmt}"
+ end
+ x
+ end
+end
--
1.7.6.4
[PATCH 1/2] CIMI: model layer
Posted by lu...@redhat.com.
From: David Lutterkort <lu...@redhat.com>
A way to describe the schema of CIMI objects so that we can serialize to
and from XML
Signed-off-by: David Lutterkort <lu...@redhat.com>
---
server/Rakefile | 5 +
server/lib/cimi/model.rb | 23 +++
server/lib/cimi/model/base.rb | 169 ++++++++++++++++++++
server/lib/cimi/model/schema.rb | 257 ++++++++++++++++++++++++++++++
server/lib/deltacloud/core_ext.rb | 1 +
server/lib/deltacloud/core_ext/array.rb | 25 +++
server/lib/deltacloud/core_ext/hash.rb | 7 +
server/lib/deltacloud/core_ext/string.rb | 3 +
server/spec/cimi/model/schema_spec.rb | 245 ++++++++++++++++++++++++++++
server/spec/spec_helper.rb | 27 +++
10 files changed, 762 insertions(+), 0 deletions(-)
create mode 100644 server/lib/cimi/model.rb
create mode 100644 server/lib/cimi/model/base.rb
create mode 100644 server/lib/cimi/model/schema.rb
create mode 100644 server/lib/deltacloud/core_ext/array.rb
create mode 100644 server/spec/cimi/model/schema_spec.rb
create mode 100644 server/spec/spec_helper.rb
diff --git a/server/Rakefile b/server/Rakefile
index f07a6ce..273469c 100644
--- a/server/Rakefile
+++ b/server/Rakefile
@@ -20,6 +20,7 @@
require 'rake'
require 'rake/testtask'
require 'rubygems/package_task'
+require 'spec/rake/spectask'
begin
require 'ci/reporter/rake/test_unit'
@@ -89,6 +90,10 @@ task :cucumber do
end
end
+Spec::Rake::SpecTask.new('spec') do |t|
+ t.spec_files = FileList['spec/**/*_spec.rb']
+end
+
begin
require 'yard'
YARD::Rake::YardocTask.new do |t|
diff --git a/server/lib/cimi/model.rb b/server/lib/cimi/model.rb
new file mode 100644
index 0000000..c1d7f02
--- /dev/null
+++ b/server/lib/cimi/model.rb
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+module CIMI
+ module Model; end
+end
+
+require 'cimi/model/schema'
+require 'cimi/model/base'
+require 'cimi/model/machine_template'
diff --git a/server/lib/cimi/model/base.rb b/server/lib/cimi/model/base.rb
new file mode 100644
index 0000000..0ef2b27
--- /dev/null
+++ b/server/lib/cimi/model/base.rb
@@ -0,0 +1,169 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+require 'xmlsimple'
+require 'json'
+
+# The base class for any CIMI object that we either read from a request or
+# write as a response. This class handles serializing/deserializing XML and
+# JSON into a common form.
+#
+# == Defining the schema
+#
+# The conversion of XML and JSON into internal objects is based on a schema
+# that is defined through a DSL:
+#
+# class Machine < CIMI::Model::Base
+# text :status
+# href :meter
+# array :volumes do
+# scalar :href, :attachment_point, :protocol
+# end
+# end
+#
+# The DSL automatically takes care of converting identifiers from their
+# underscored form to the camel-cased form used by CIMI. The above class
+# can be used in the following way:
+#
+# machine = Machine.from_xml(some_xml)
+# if machine.status == "UP"
+# ...
+# end
+# sda = machine.volumes.find { |v| v.attachment_point == "/dev/sda" }
+# handle_meter(machine.meter.href)
+#
+# The keywords for the DSL are
+# [scalar(names, ...)]
+# Define a scalar attribute; in JSON, this is represented as a string
+# property. In XML, this can be represented in a number of ways,
+# depending on whether the option :text is set:
+# * :text not set: attribute on the enclosing element
+# * :text == :direct: the text content of the enclosing element
+# * :text == :nested: the text content of an element +<name>...</name>+
+# [text(names)]
+# A shorthand for +scalar(names, :text => :nested)+, i.e., for
+# attributes that in XML are represented by their own tags
+# [href(name)]
+# A shorthand for +struct name { scalar :href }+; in JSON, this is
+# represented as +{ name: { "href": string } }+, and in XML as +<name
+# href="..."/>+
+# [struct(name, opts, &block)]
+# A structured subobject; the block defines the schema of the
+# subobject. The +:content+ option can be used to specify the attribute
+# that should receive the content of hte corresponding XML element
+# [array(name, opts, &block)]
+# An array of structured subobjects; the block defines the schema of
+# the subobjects.
+class CIMI::Model::Base
+
+ #
+ # We keep the values of the attributes in a hash
+ #
+ attr_reader :attribute_values
+
+ # Keep the list of all attributes in an array +attributes+; for each
+ # attribute, we also define a getter and a setter to access/change the
+ # value for that attribute
+ class << self
+ def schema
+ @schema ||= CIMI::Model::Schema.new
+ end
+
+ def schema=(s)
+ @schema = s
+ end
+
+ def inherited(child)
+ child.schema = self.schema.dup
+ end
+
+ def add_attributes!(names, attr_klass, &block)
+ schema.add_attributes!(names, attr_klass, &block)
+ names.each do |name|
+ define_method(name) { @attribute_values[name] }
+ define_method(:"#{name}=") { |newval| @attribute_values[name] = newval }
+ end
+ end
+ end
+
+ extend CIMI::Model::Schema::DSL
+
+ def [](a)
+ @attribute_values[a]
+ end
+
+ def []=(a, v)
+ @attribute_values[a] = v
+ end
+
+ #
+ # Factory methods
+ #
+ def initialize(values = {})
+ @attribute_values = values
+ end
+
+ # Construct a new object from the XML representation +xml+
+ def self.from_xml(text)
+ xml = XmlSimple.xml_in(text, :force_content => true)
+ model = self.new
+ @schema.from_xml(xml, model)
+ model
+ end
+
+ # Construct a new object
+ def self.from_json(text)
+ json = JSON::parse(text)
+ model = self.new
+ @schema.from_json(json, model)
+ model
+ end
+
+ #
+ # Serialize
+ #
+
+ def self.xml_tag_name
+ self.name.split("::").last
+ end
+
+ def self.to_json(model)
+ @schema.to_json(model)
+ end
+
+ def self.to_xml(model)
+ xml = @schema.to_xml(model)
+ xml["xmlns"] = "http://www.dmtf.org/cimi"
+ XmlSimple.xml_out(xml, :root_name => xml_tag_name)
+ end
+
+ def to_json
+ self.class.to_json(self)
+ end
+
+ def to_xml
+ self.class.to_xml(self)
+ end
+
+ #
+ # Common attributes for all resources
+ #
+ text :uri, :name, :description, :created
+
+ # FIXME: this doesn't match with JSON
+ array :properties, :content => :value do
+ scalar :key
+ end
+end
diff --git a/server/lib/cimi/model/schema.rb b/server/lib/cimi/model/schema.rb
new file mode 100644
index 0000000..e470121
--- /dev/null
+++ b/server/lib/cimi/model/schema.rb
@@ -0,0 +1,257 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+# The smarts of converting from XML and JSON into internal objects
+class CIMI::Model::Schema
+
+ #
+ # Attributes describe how we extract values from XML/JSON
+ #
+ class Attribute
+ attr_reader :name, :xml_name, :json_name
+
+ def initialize(name, opts = {})
+ @name = name
+ @xml_name = (opts[:xml_name] || name).to_s.camelize(true)
+ @json_name = (opts[:json_name] || name).to_s.camelize(true)
+ end
+
+ def from_xml(xml, model)
+ model[@name] = xml[@xml_name].first if xml.has_key?(@xml_name)
+ end
+
+ def from_json(json, model)
+ model[@name] = json[@json_name]
+ end
+
+ def to_xml(model, xml)
+ xml[@xml_name] = [model[@name]] if model[@name]
+ end
+
+ def to_json(model, json)
+ json[@json_name] = model[@name] if model[@name]
+ end
+ end
+
+ class Scalar < Attribute
+ def initialize(name, opts)
+ @text = opts[:text]
+ if ! [nil, :nested, :direct].include?(@text)
+ raise "text option for scalar must be :nested or :direct"
+ end
+ super(name, opts)
+ end
+
+ def text?; @text; end
+
+ def nested_text?; @text == :nested; end
+
+ def from_xml(xml, model)
+ if @text == :nested
+ model[@name] = xml[@xml_name].first["content"] if xml[@xml_name]
+ elsif @text == :direct
+ model[@name] = xml["content"]
+ else
+ model[@name] = xml[@xml_name]
+ end
+ end
+
+ def to_xml(model, xml)
+ return unless model[@name]
+ if @text == :nested
+ xml[@xml_name] = [{ "content" => model[@name] }]
+ elsif @text == :direct
+ xml["content"] = model[@name]
+ else
+ xml[@xml_name] = model[@name]
+ end
+ end
+ end
+
+ class Struct < Attribute
+ def initialize(name, opts, &block)
+ content = opts[:content]
+ super(name)
+ @schema = CIMI::Model::Schema.new
+ @schema.instance_eval(&block) if block_given?
+ @schema.scalar(content, :text => :direct) if content
+ end
+
+ def from_xml(xml, model)
+ xml = xml.has_key?(xml_name) ? xml[xml_name].first : {}
+ model[name] = convert_from_xml(xml)
+ end
+
+ def from_json(json, model)
+ json = json.has_key?(json_name) ? json[json_name] : {}
+ model[name] = convert_from_json(json)
+ end
+
+ def to_xml(model, xml)
+ conv = convert_to_xml(model[name])
+ xml[xml_name] = [conv] unless conv.empty?
+ end
+
+ def to_json(model, json)
+ conv = convert_to_json(model[name])
+ json[json_name] = conv unless conv.empty?
+ end
+
+ def convert_from_xml(xml)
+ sub = struct.new
+ @schema.from_xml(xml, sub)
+ sub
+ end
+
+ def convert_from_json(json)
+ sub = struct.new
+ @schema.from_json(json, sub)
+ sub
+ end
+
+ def convert_to_xml(model)
+ xml = {}
+ @schema.to_xml(model, xml)
+ xml
+ end
+
+ def convert_to_json(model)
+ json = {}
+ @schema.to_json(model, json)
+ json
+ end
+
+ private
+ def struct
+ cname = "CIMI_#{json_name.capitalize}"
+ if ::Struct.const_defined?(cname)
+ ::Struct.const_get(cname)
+ else
+ ::Struct.new("CIMI_#{json_name.capitalize}",
+ *@schema.attribute_names)
+ end
+ end
+ end
+
+ class Array < Attribute
+ # For an array :things, we collect all <thing/> elements (XmlSimple
+ # actually does the collecting)
+ def initialize(name, opts = {}, &block)
+ opts[:xml_name] = name.to_s.singularize unless opts[:xml_name]
+ super(name, opts)
+ @struct = Struct.new(name, opts, &block)
+ end
+
+ def from_xml(xml, model)
+ model[name] = (xml[xml_name] || []).map do |elt|
+ @struct.convert_from_xml(elt)
+ end
+ end
+
+ def from_json(json, model)
+ model[name] = (json[json_name] || []).map do |elt|
+ @struct.convert_from_json(elt)
+ end
+ end
+
+ def to_xml(model, xml)
+ ary = model[name].map { |elt| @struct.convert_to_xml(elt) }
+ xml[xml_name] = ary unless ary.empty?
+ end
+
+ def to_json(model, json)
+ ary = model[name].map { |elt| @struct.convert_to_json(elt) }
+ json[json_name] = ary unless ary.empty?
+ end
+ end
+
+ #
+ # The actual Schema class
+ #
+ def initialize
+ @attributes = []
+ end
+
+ def from_xml(xml, model = {})
+ @attributes.freeze
+ @attributes.each { |attr| attr.from_xml(xml, model) }
+ model
+ end
+
+ def from_json(json, model = {})
+ @attributes.freeze
+ @attributes.each { |attr| attr.from_json(json, model) }
+ model
+ end
+
+ def to_xml(model, xml = {})
+ @attributes.freeze
+ @attributes.each { |attr| attr.to_xml(model, xml) }
+ xml
+ end
+
+ def to_json(model, json = {})
+ @attributes.freeze
+ @attributes.each { |attr| attr.to_json(model, json) }
+ json
+ end
+
+ def attribute_names
+ @attributes.map { |a| a.name }
+ end
+
+ #
+ # The DSL
+ #
+ # Requires that the class into which this is included has a
+ # +add_attributes!+ method
+ module DSL
+ def href(*args)
+ args.each do |arg|
+ struct(arg) { scalar :href }
+ end
+ end
+
+ def text(*args)
+ args.expand_opts!(:text => :nested)
+ scalar(*args)
+ end
+
+ def scalar(*args)
+ add_attributes!(args, Scalar)
+ end
+
+ def array(name, opts={}, &block)
+ add_attributes!([name, opts], Array, &block)
+ end
+
+ def struct(name, opts={}, &block)
+ add_attributes!([name, opts], Struct, &block)
+ end
+ end
+
+ include DSL
+
+ def add_attributes!(args, attr_klass, &block)
+ if @attributes.frozen?
+ raise "The schema has already been used to convert objects"
+ end
+ opts = args.extract_opts!
+ args.each do |arg|
+ @attributes << attr_klass.new(arg, opts, &block)
+ end
+ end
+end
diff --git a/server/lib/deltacloud/core_ext.rb b/server/lib/deltacloud/core_ext.rb
index 43b3b25..042bffc 100644
--- a/server/lib/deltacloud/core_ext.rb
+++ b/server/lib/deltacloud/core_ext.rb
@@ -17,3 +17,4 @@
require 'deltacloud/core_ext/string'
require 'deltacloud/core_ext/integer'
require 'deltacloud/core_ext/hash'
+require 'deltacloud/core_ext/array'
diff --git a/server/lib/deltacloud/core_ext/array.rb b/server/lib/deltacloud/core_ext/array.rb
new file mode 100644
index 0000000..620b1f4
--- /dev/null
+++ b/server/lib/deltacloud/core_ext/array.rb
@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+class Array
+ def expand_opts!(more_opts)
+ self << {} unless last.is_a?(Hash)
+ last.update(more_opts)
+ end
+
+ def extract_opts!
+ last.is_a?(Hash) ? pop : {}
+ end
+end
diff --git a/server/lib/deltacloud/core_ext/hash.rb b/server/lib/deltacloud/core_ext/hash.rb
index 9cdba50..fb12045 100644
--- a/server/lib/deltacloud/core_ext/hash.rb
+++ b/server/lib/deltacloud/core_ext/hash.rb
@@ -26,4 +26,11 @@ class Hash
#remove the original keys
self.delete_if{|k,v| remove.include?(k)}
end
+
+ # Method copied from https://github.com/rails/rails/blob/77efc20a54708ba37ba679ffe90021bf8a8d3a8a/activesupport/lib/active_support/core_ext/hash/keys.rb#L23
+ def symbolize_keys
+ keys.each { |key| self[(key.to_sym rescue key) || key] = delete(key) }
+ self
+ end
+
end
diff --git a/server/lib/deltacloud/core_ext/string.rb b/server/lib/deltacloud/core_ext/string.rb
index 70d0df6..c5d9bf3 100644
--- a/server/lib/deltacloud/core_ext/string.rb
+++ b/server/lib/deltacloud/core_ext/string.rb
@@ -59,4 +59,7 @@ class String
self[0, 1].downcase + self[1..-1]
end
+ def capitalize
+ self[0, 1].upcase + self[1..-1]
+ end
end
diff --git a/server/spec/cimi/model/schema_spec.rb b/server/spec/cimi/model/schema_spec.rb
new file mode 100644
index 0000000..0b262f4
--- /dev/null
+++ b/server/spec/cimi/model/schema_spec.rb
@@ -0,0 +1,245 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+require 'spec_helper'
+
+require 'cimi/model'
+
+describe "Schema" do
+ before(:each) do
+ @schema = CIMI::Model::Schema.new
+ end
+
+ it "does not allow adding attributes after being used for conversion" do
+ @schema.scalar(:before)
+ @schema.from_json({})
+ lambda { @schema.scalar(:after) }.should raise_error
+ end
+
+ describe "scalars" do
+ before(:each) do
+ @schema.scalar(:attr)
+ @schema.text(:camel_hump)
+
+ @schema.attribute_names.should == [:attr, :camel_hump]
+ end
+
+ let :sample_xml do
+ parse_xml("<camelHump>bumpy</camelHump>", :keep_root => true)
+ end
+
+ it "should camel case attribute names for JSON" do
+ obj = @schema.from_json("camelHump" => "bumpy")
+ obj.should_not be_nil
+ obj[:camel_hump].should == "bumpy"
+
+ json = @schema.to_json(obj)
+ json.should == { "camelHump" => "bumpy" }
+ end
+
+ it "should camel case attribute names for XML" do
+ obj = @schema.from_xml(sample_xml)
+
+ obj.should_not be_nil
+ obj[:camel_hump].should == "bumpy"
+
+ xml = @schema.to_xml(obj)
+
+ xml.should == { "camelHump" => [{ "content" => "bumpy" }] }
+ end
+
+ it "should allow aliasing the XML and JSON name" do
+ @schema.scalar :aliased, :xml_name => :xml, :json_name => :json
+ obj = @schema.from_xml({"aliased" => "no", "xml" => "yes"}, {})
+ obj[:aliased].should == "yes"
+
+ obj = @schema.from_json({"aliased" => "no", "json" => "yes"}, {})
+ obj[:aliased].should == "yes"
+ end
+ end
+
+ describe "hrefs" do
+ before(:each) do
+ @schema.href(:meter)
+ end
+
+ it "should extract the href attribute from XML" do
+ xml = parse_xml("<meter href='http://example.org/'/>")
+
+ obj = @schema.from_xml(xml)
+ check obj
+ @schema.to_xml(obj).should == xml
+ end
+
+ it "should extract the href attribute from JSON" do
+ json = { "meter" => { "href" => "http://example.org/" } }
+
+ obj = @schema.from_json(json)
+ check obj
+ @schema.to_json(obj).should == json
+ end
+
+ def check(obj)
+ obj.should_not be_nil
+ obj[:meter].href.should == 'http://example.org/'
+ end
+ end
+
+ describe "structs" do
+ before(:each) do
+ @schema.struct(:struct, :content => :scalar) do
+ scalar :href
+ end
+ @schema.attribute_names.should == [:struct]
+ end
+
+ let(:sample_json) do
+ { "struct" => { "scalar" => "v1", "href" => "http://example.org/" } }
+ end
+
+ let (:sample_xml) do
+ parse_xml("<struct href='http://example.org/'>v1</struct>")
+ end
+
+ let (:sample_xml_no_href) do
+ parse_xml("<struct>v1</struct>")
+ end
+
+ describe "JSON conversion" do
+ it "should convert empty hash" do
+ model = @schema.from_json({ })
+ check_empty_struct model
+ @schema.to_json(model).should == {}
+ end
+
+ it "should convert empty body" do
+ model = @schema.from_json({ "struct" => { } })
+ check_empty_struct model
+ @schema.to_json(model).should == {}
+ end
+
+ it "should convert values" do
+ model = @schema.from_json(sample_json)
+ check_struct model
+ @schema.to_json(model).should == sample_json
+ end
+ end
+
+ describe "XML conversion" do
+ it "should convert empty hash" do
+ model = @schema.from_xml({ })
+ check_empty_struct model
+ @schema.to_xml(model).should == {}
+ end
+
+ it "should convert empty body" do
+ model = @schema.from_json({ "struct" => { } })
+ check_empty_struct model
+ @schema.to_xml(model).should == {}
+ end
+
+ it "should convert values" do
+ model = @schema.from_xml(sample_xml)
+ check_struct model
+ @schema.to_xml(model).should == sample_xml
+ end
+
+ it "should handle missing attributes" do
+ model = @schema.from_xml(sample_xml_no_href)
+ check_struct model, :nil_href => true
+ @schema.to_xml(model).should == sample_xml_no_href
+ end
+ end
+
+ def check_struct(obj, opts = {})
+ obj.should_not be_nil
+ obj[:struct].should_not be_nil
+ obj[:struct].scalar.should == "v1"
+ if opts[:nil_href]
+ obj[:struct].href.should be_nil
+ else
+ obj[:struct].href.should == "http://example.org/"
+ end
+ end
+
+ def check_empty_struct(obj)
+ obj.should_not be_nil
+ obj[:struct].should_not be_nil
+ obj[:struct].scalar.should be_nil
+ obj[:struct].href.should be_nil
+ end
+ end
+
+ describe "arrays" do
+ before(:each) do
+ @schema.array(:structs, :content => :scalar) do
+ scalar :href
+ end
+ end
+
+ let(:sample_json) do
+ { "structs" => [{ "scalar" => "v1", "href" => "http://example.org/1" },
+ { "scalar" => "v2", "href" => "http://example.org/2" }] }
+ end
+
+ let (:sample_xml) do
+ parse_xml("<wrapper>
+ <struct href='http://example.org/1'>v1</struct>
+ <struct href='http://example.org/2'>v2</struct>
+</wrapper>", :keep_root => false)
+ end
+
+ it "should convert missing array from JSON" do
+ obj = @schema.from_json({})
+
+ obj.should_not be_nil
+ obj[:structs].should == []
+ @schema.to_json(obj).should == {}
+ end
+
+ it "should convert empty array from JSON" do
+ obj = @schema.from_json("structs" => [])
+
+ obj.should_not be_nil
+ obj[:structs].should == []
+ @schema.to_json(obj).should == {}
+ end
+
+ it "should convert arrays from JSON" do
+ obj = @schema.from_json(sample_json)
+
+ check_structs(obj)
+ @schema.to_json(obj).should == sample_json
+ end
+
+ it "should convert arrays from XML" do
+ obj = @schema.from_xml(sample_xml)
+
+ check_structs(obj)
+ @schema.to_xml(obj).should == sample_xml
+ end
+
+ def check_structs(obj)
+ obj.should_not be_nil
+ obj[:structs].size.should == 2
+ obj[:structs][0].scalar.should == "v1"
+ obj[:structs][0].href.should == "http://example.org/1"
+ obj[:structs][1].scalar.should == "v2"
+ obj[:structs][1].href.should == "http://example.org/2"
+ end
+ end
+
+end
diff --git a/server/spec/spec_helper.rb b/server/spec/spec_helper.rb
new file mode 100644
index 0000000..1e0dd87
--- /dev/null
+++ b/server/spec/spec_helper.rb
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership. The
+# ASF licenses this file to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the
+# License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+require 'rubygems'
+require 'pp'
+
+require 'deltacloud/core_ext'
+require 'xmlsimple'
+
+def parse_xml(xml, opts = {})
+ opts[:force_content] = true
+ opts[:keep_root] = true unless opts.has_key?(:keep_root)
+ XmlSimple.xml_in(xml, opts)
+end
--
1.7.6.4
Re: Model layer for CIMI
Posted by David Lutterkort <lu...@redhat.com>.
On Thu, 2011-11-10 at 08:36 -0500, Tong Li wrote:
> I do not think DMTF cimi spec strongly suggested the sequence of the
> elements. At least it did not say in the spec. although most of the samples
> do show a patten of the elements.
That's one of the implicit rules of XML: order matters, DOM's are
ordered trees etc.
> When I used XmlSimple (), it actually
> keeps the right order with the xml it reads in. The difficulty is when you
> have a hash (produced by XmlSimple from an XML doc), you most likely will
> add or remove things off of the hash, that is where it sequence really get
> messed up, I do not consider it has anything to do with the XmlSimple, it
> is just when hash get serialized to xml or json, probably the serializer
> does not know how to deal with the sequence since the hash certainly does
> not have the sequence information of its members.
Either way, we'll have to write our own XML serialization to make sure
element order is correct; the schema that is declared with the DSL has
the element order (the order in which attributes are declared)
David
Re: Model layer for CIMI
Posted by David Lutterkort <lu...@redhat.com>.
On Thu, 2011-11-10 at 09:02 -0500, Doug Davis wrote:
> I think its safe to assume that when the xsd for the cimi spec is written
> it'll be written such that the xml elements will be required to be in the
> order specified by the xsd.
Yeah, I wasn't expecting otherwise. Shouldn't be a huge issue to take
care of.
BTW, I _think_ we'll have enough metadata in the Ruby code to generate
rng (or maybe even xsd) if that should be useful.
David
Re: Model layer for CIMI
Posted by Doug Davis <du...@us.ibm.com>.
I think its safe to assume that when the xsd for the cimi spec is written
it'll be written such that the xml elements will be required to be in the
order specified by the xsd.
thanks
-Doug
______________________________________________________
STSM | Standards Architect | IBM Software Group
(919) 254-6905 | IBM 444-6905 | dug@us.ibm.com
The more I'm around some people, the more I like my dog.
Tong Li/Raleigh/IBM@IBMUS
11/10/2011 08:36 AM
Please respond to
deltacloud-dev@incubator.apache.org
To
deltacloud-dev@incubator.apache.org
cc
deltacloud-dev@incubator.apache.org
Subject
Re: Model layer for CIMI
I do not think DMTF cimi spec strongly suggested the sequence of the
elements. At least it did not say in the spec. although most of the
samples do show a patten of the elements. When I used XmlSimple (), it
actually keeps the right order with the xml it reads in. The difficulty is
when you have a hash (produced by XmlSimple from an XML doc), you most
likely will add or remove things off of the hash, that is where it
sequence really get messed up, I do not consider it has anything to do
with the XmlSimple, it is just when hash get serialized to xml or json,
probably the serializer does not know how to deal with the sequence since
the hash certainly does not have the sequence information of its members.
just my 2cents.
Tong Li
Emerging Technologies & Standards
Building 501/B205
litong01@us.ibm.com
lutter---11/09/2011 04:19:20 PM---This is a model layer for CIMI that does
both serialization and deserialization of JSON and XML base
From: lutter@redhat.com
To: deltacloud-dev@incubator.apache.org
Date: 11/09/2011 04:19 PM
Subject: Model layer for CIMI
This is a model layer for CIMI that does both serialization and
deserialization of JSON and XML based on a simple metadata description of
the CIMI model expressed in a custom DSL. In particular, we won't need
HAML
templates to format objects, and I am reasonably sure that the metadata
makes sure we serialize in a predictable manner to XML.
Patch 1/2 contains the basics, including the DSL, 2/2 shows how this can
be
used for a specific class.
There's a few convenience additions that should be made to the DSL, in
particular making it possible to reuse nested structures that are used in
several places, something like
array :operations, :class => Operation
but we can add that later.
One small fly in the ointment is that the use of XmlSimple causes child
elements in XML to be rendered in an unpredictable order; the CIMI
standard
doesn't say anything about reordering elements, though with a strict
interpretation of XML, the rendered XML does not conform to the XML schema
implicitly defined in the CIMI standard.
David
Re: Model layer for CIMI
Posted by Tong Li <li...@us.ibm.com>.
I do not think DMTF cimi spec strongly suggested the sequence of the
elements. At least it did not say in the spec. although most of the samples
do show a patten of the elements. When I used XmlSimple (), it actually
keeps the right order with the xml it reads in. The difficulty is when you
have a hash (produced by XmlSimple from an XML doc), you most likely will
add or remove things off of the hash, that is where it sequence really get
messed up, I do not consider it has anything to do with the XmlSimple, it
is just when hash get serialized to xml or json, probably the serializer
does not know how to deal with the sequence since the hash certainly does
not have the sequence information of its members.
just my 2cents.
Tong Li
Emerging Technologies & Standards
Building 501/B205
litong01@us.ibm.com
From: lutter@redhat.com
To: deltacloud-dev@incubator.apache.org
Date: 11/09/2011 04:19 PM
Subject: Model layer for CIMI
This is a model layer for CIMI that does both serialization and
deserialization of JSON and XML based on a simple metadata description of
the CIMI model expressed in a custom DSL. In particular, we won't need HAML
templates to format objects, and I am reasonably sure that the metadata
makes sure we serialize in a predictable manner to XML.
Patch 1/2 contains the basics, including the DSL, 2/2 shows how this can be
used for a specific class.
There's a few convenience additions that should be made to the DSL, in
particular making it possible to reuse nested structures that are used in
several places, something like
array :operations, :class => Operation
but we can add that later.
One small fly in the ointment is that the use of XmlSimple causes child
elements in XML to be rendered in an unpredictable order; the CIMI standard
doesn't say anything about reordering elements, though with a strict
interpretation of XML, the rendered XML does not conform to the XML schema
implicitly defined in the CIMI standard.
David
Re: Model layer for CIMI
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2011-11-11 at 17:57 +0100, Michal Fojtik wrote:
> ACK. We should integrate this into our code base so we can start
> improving this and adding more collections.
Thanks. Pushed.
David
Re: Model layer for CIMI
Posted by Michal Fojtik <mf...@redhat.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
lutter@redhat.com wrote:
ACK. We should integrate this into our code base so we can start
improving this and adding more collections.
All tests run smoothly, applied without whitespace errors and the code
looks beautiful :-)
-- Michal
> This is a model layer for CIMI that does both serialization and
> deserialization of JSON and XML based on a simple metadata
> description of the CIMI model expressed in a custom DSL. In
> particular, we won't need HAML templates to format objects, and I am
> reasonably sure that the metadata makes sure we serialize in a
> predictable manner to XML.
>
> Patch 1/2 contains the basics, including the DSL, 2/2 shows how this
> can be used for a specific class.
>
> There's a few convenience additions that should be made to the DSL,
> in particular making it possible to reuse nested structures that are
> used in several places, something like
>
> array :operations, :class => Operation
>
> but we can add that later.
>
> One small fly in the ointment is that the use of XmlSimple causes
> child elements in XML to be rendered in an unpredictable order; the
> CIMI standard doesn't say anything about reordering elements, though
> with a strict interpretation of XML, the rendered XML does not
> conform to the XML schema implicitly defined in the CIMI standard.
>
> David
- --
- --
Michal Fojtik, mfojtik@redhat.com
Deltacloud API: http://deltacloud.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJOvVP9AAoJEEnFpmY+LvFRgnIH/1yKjQkRH0wndFHZzNIuudmG
rEy1V5kmtSApieuWawVSnyqGGRbLZxdk/tLY/Lkks/3wrOsQs9iosV7OL5LNYrEw
VrqVjCLMmd4JxKk6AoEiVTUjXMlmR1x11prsAHRIf42usTtatZvj6cfMeUQTW8/c
G+RjYTxqXpXt0R19pUqGzJqNwQqDhPtnazELo4TUupbxYw3d4wig/RMBKnoC9nbV
JO37IbsrXCK1dSoPtruPiOYIzSF8fHoFY1rtagJxxils6q3iZq3H+Sx+7aOTMmky
CevQjZL4Rca8YiU9FOyvbVyYBK3uqgsQbk7/Rhpn4d+5gdnp5uZoTgL157LSG9k=
=yPVU
-----END PGP SIGNATURE-----
Re: Model layer for CIMI
Posted by Tong Li <li...@us.ibm.com>.
BTW, I am using xml-simple 1.1.1, json 1.6.1.
Thanks.
Tong Li
Emerging Technologies & Standards
Building 501/B205
litong01@us.ibm.com
From: lutter@redhat.com
To: deltacloud-dev@incubator.apache.org
Date: 11/09/2011 04:19 PM
Subject: Model layer for CIMI
This is a model layer for CIMI that does both serialization and
deserialization of JSON and XML based on a simple metadata description of
the CIMI model expressed in a custom DSL. In particular, we won't need HAML
templates to format objects, and I am reasonably sure that the metadata
makes sure we serialize in a predictable manner to XML.
Patch 1/2 contains the basics, including the DSL, 2/2 shows how this can be
used for a specific class.
There's a few convenience additions that should be made to the DSL, in
particular making it possible to reuse nested structures that are used in
several places, something like
array :operations, :class => Operation
but we can add that later.
One small fly in the ointment is that the use of XmlSimple causes child
elements in XML to be rendered in an unpredictable order; the CIMI standard
doesn't say anything about reordering elements, though with a strict
interpretation of XML, the rendered XML does not conform to the XML schema
implicitly defined in the CIMI standard.
David