You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "Han,Meng" <me...@ufl.edu> on 2013/09/18 04:14:18 UTC
conflicting dependencies between CloudStack and Whirr
Dear all,
I am adding an API to CloudStack which utilizes Whirr to launch various
clusters on CloudStack. Now I am facing a dependency conflicting issue.
Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
1.7.1. If I use gson 1.7.1 for Whirr, the following error will happen:
com.google.common.util.concurrent.ExecutionError:
java.lang.NoClassDefFoundError: com/google/gson/TypeAdapter
This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
modify CloudStack to use gson 2.2.2 CloudStack will not build
successfully due to the following test error.
[INFO] Building Apache CloudStack Core 4.1.1
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
---
[INFO] Deleting /home/meng/cloudstack/core/target
[INFO]
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @
cloud-core ---
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
cloud-core ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/meng/cloudstack/core/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
cloud-core ---
[INFO] Compiling 156 source files to
/home/meng/cloudstack/core/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources
(default-testResources) @ cloud-core ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/meng/cloudstack/core/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile
(default-testCompile) @ cloud-core ---
[INFO] Compiling 1 source file to
/home/meng/cloudstack/core/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ cloud-core
---
[INFO] Surefire report directory:
/home/meng/cloudstack/core/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.cloud.agent.transport.RequestTest
log4j:WARN No appenders could be found for logger
(com.cloud.agent.transport.RequestTest).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
for more info.
Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
sec <<< FAILURE!
Results :
Failed tests: testSerDeser(com.cloud.agent.transport.RequestTest)
Tests in error:
testLogging(com.cloud.agent.transport.RequestTest)
Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
I ran "mvn -P developer clean install -DskipTests" , the CloudStack
building process went through. But when I issued an API --launchCluster
to CloudStack, the following error related to gson popped up on
CloudStack Management Server:
ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
problem with
[{"StartupRoutingCommand":{"cpus":2,"speed":3000,"memory":3844370432,"dom0MinMemory":384437043,"poolSync":false,"vms":{"v-2-VM":{"state":"Running"},"s-1-VM":{"state":"Running"},"r-4-VM":{"state":"Running"}},"caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS":"CentOS","Host.OS.Kernel.Version":"2.6.32-358.el6.x86_64","Host.OS.Version":"6.4"},"type":"Routing","dataCenter":"1","pod":"1","cluster":"1","guid":"a7320748-6346-3c9a-975e-90ac4ae4a986-LibvirtComputingResource","name":"meng.acis.ufl.edu","id":2,"version":"4.1.1","publicIpAddress":"10.244.18.55","publicNetmask":"255.0.0.0","publicMacAddress":"00:23:ae:94:f7:22","privateIpAddress":"10.244.18.55","privateMacAddress":"00:23:ae:94:f7:22","privateNetmask":"255.0.0.0","storageIpAddress":"10.244.18.55","storageNetmask":"255.0.0.0","storageMacAddress":"00:23:ae:94:f7:22","resourceName":"LibvirtComputingResource","gatewayIpAddress":"!
10.244.18
.1","contextMap":{},"wait":0}},{"StartupStorageCommand":{"totalSize":0,"poolInfo":{"uuid":"9447c0b1-cc3f-439f-85f6-13d35539a9ed","host":"10.244.18.55","localPath":"/var/lib/libvirt/images/","hostPath":"/var/lib/libvirt/images/","poolType":"Filesystem","capacityBytes":52844687360,"availableBytes":41535332352},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"1","pod":"1","guid":"a7320748-6346-3c9a-975e-90ac4ae4a986-LibvirtComputingResource","name":"meng.acis.ufl.edu","id":2,"version":"4.1.1","resourceName":"LibvirtComputingResource","contextMap":{},"wait":0}}]
java.lang.ClassCastException:
com.google.gson.internal.$Gson$Types$GenericArrayTypeImpl cannot be cast
to java.lang.Class
at
com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:84)
at
com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:37)
at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
at com.google.gson.Gson.fromJson(Gson.java:795)
at com.cloud.agent.transport.Request.getCommands(Request.java:235)
at
com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1221)
at
com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1374)
at
com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:659)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
exception but pushing on
On the CS Managment console I can see the cluster was started then
quickly died.
Running on provider cloudstack using identity
h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ
INFO [whirr.actions.BootstrapClusterAction]
(729061754@qtp-385354117-0:) Bootstrapping cluster
INFO [whirr.compute.BootstrapTemplate] (729061754@qtp-385354117-0:)
Configuring template for bootstrap-hadoop-datanode_hadoop-tasktracker
INFO [whirr.compute.BootstrapTemplate] (729061754@qtp-385354117-0:)
Configuring template for bootstrap-hadoop-namenode_hadoop-jobtracker
INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
with roles [hadoop-datanode, hadoop-tasktracker]
INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
with roles [hadoop-namenode, hadoop-jobtracker]
...
INFO [whirr.actions.DestroyClusterAction] (729061754@qtp-385354117-0:)
Cluster hadoop destroyed
I attached the debuging log to this email.
The launchCluster API is simple, it calls the LaunchClusterCommand in
Whirr to launch a cluster.
LaunchClusterResponse response = new LaunchClusterResponse();
response.setObjectName("launchCluster");
LaunchClusterCommand command = null;
try {
command = new LaunchClusterCommand();
} catch (Exception ex) {
Logger.getLogger(LaunchClusterCmd.class.getName()).log(Level.SEVERE,
null, ex);
}
String[] args = new String[2];
args[0] = "--config";
args[1] = config;
try {
command.run(System.in, System.out, System.err,
Arrays.asList(args));
} catch (Exception ex) {
Logger.getLogger(LaunchClusterCmd.class.getName()).log(Level.SEVERE,
null, ex);
}
response.setResponseName(getCommandName());
output = "successfully launched the cluster.";
response.setOutPut(output);
response.setAsync(Boolean.FALSE);
this.setResponseObject(response);
Could someone help me out here? What would be a proper gson version for
CloudStack and Whirr to run at the same time?
Thanks loads.
Best Regards,
Meng
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
I think I agree, but I hate to shoot down a GSoC project. Where is
the code for this currently? I don't really know much about this
enhancement. Regardless, I still want jackson.
Darren
On Mon, Sep 23, 2013 at 11:54 AM, Chiradeep Vittal
<Ch...@citrix.com> wrote:
> I still think this is the wrong way to implement the Whirr integration.
>
> On 9/23/13 6:45 AM, "sebgoa" <ru...@gmail.com> wrote:
>
>>
>>On Sep 23, 2013, at 7:12 AM, Darren Shepherd
>><da...@gmail.com> wrote:
>>
>>> The only way this will get committed is if we either move to latest
>>>gson or
>>> jackson. Regardless, the outcome will be that Meng can assume this gson
>>> conflict won't happen. The problem is just how fast can we move all of
>>>ACS
>>> to a new json library. Is it possible for Meng to commit to a branch
>>>that
>>> just blindly updates gson to the latest (in /pom.xml).
>>
>>Yes I will talk to Meng about this. thanks
>>
>>> When we get ACS on
>>> a newer json library, we can merge in that branch. Tomorrow (9/23), I
>>>can
>>> attempt to put together a patch to move to jackson. I think I ironed
>>>out
>>> most of the technical issues so I just have to globally apply the
>>>changes.
>>>
>>> Darren
>>>
>>>
>>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen
>>><ru...@gmail.com>wrote:
>>>
>>>>
>>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>
>>>>>
>>>>> The Nicira NVP plugin is also depending on gson. If we make any
>>>>>changes
>>>> we need to validate against their api to ensure that the interface
>>>>still
>>>> works.
>>>>>
>>>>> If we put the changes in a separate branch i'd be happy to run the
>>>> changes against the api.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Hugo
>>>>
>>>> So how can Meng get out of this bind, she is trying to complete her
>>>>google
>>>> summer of code project.
>>>>
>>>> thanks,
>>>>
>>>> -Sebastien
>>>>
>>>>>
>>>>>
>>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>
>>>>>> After much searching I found the two lines of code I needed to get it
>>>> to serialize right. I'll keep looking and see if I find other snags.
>>>> Maybe we can just move to jackson.... we'll see, haven't tried to
>>>> deserialize yet.
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com>
>>>>>>>wrote:
>>>>>>>
>>>>>>> Darren,
>>>>>>>
>>>>>>> When I looked into updating to the latest gson, the problem, IIRC,
>>>>>>>is
>>>> things that weren't considered cyclical dependencies are suddenly
>>>> considered cyclical with the latest. I don't recall where though.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>>
>>>>>>>> Alright, I looked into this and it will take a bit more work to
>>>> switch to Jackson.
>>>>>>>> The snag I hit was how we serialize commands for the agents. We
>>>>>>>>use a
>>>>>>>> customer typeadatper in gson to produce a format like {
>>>>>>>>"org.MyClass"
>>>>>>>> : {...} }. In jackson I don't see producing a similar format as
>>>> being so straight
>>>>>>>> forward. I'm sure there's an elegant solution, but I didn't
>>>> immediately find it.
>>>>>>>> The problem is if you use a custom serializer in jackson and do
>>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>>>> loop
>>>>>>>> because it will call your serializer again. So if somebody can
>>>> figure out how to
>>>>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>>>>
>>>>>>>> So, being that it is not an obvious switch to using jackson, what
>>>>>>>>is
>>>> the impact
>>>>>>>> of moving to the latest gson? What were the issue encountered when
>>>>>>>> people tried it before?
>>>>>>>>
>>>>>>>> Darren
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I can do some analysis on this. I'm always up for a terribly
>>>>>>>>>painful
>>>>>>>>> refactor :)
>>>>>>>>>
>>>>>>>>> Darren
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang
>>>>>>>>>><Al...@citrix.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I actually did a quick try to update cloudstack to use newest
>>>>>>>>>>gson
>>>>>>>>>> version about 3 months back. Had to roll it back but I didn't
>>>>>>>>>>try
>>>>>>>>>> very hard though. Part of the reason why I decided to rollback
>>>>>>>>>>is
>>>>>>>>>> due to gson is used differently by various components in
>>>>>>>>>>CloudStack
>>>>>>>>>> and I didn't have time to get into each component to see if I
>>>>>>>>>>break
>>>>>>>>>> anything. The other is also I thought using Jackson would be
>>>>>>>>>>much
>>>>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>>>>
>>>>>>>>>> Not that any of these helps you. Just know updating CloudStack
>>>>>>>>>>to
>>>>>>>>>> latest gson is not simple.
>>>>>>>>>>
>>>>>>>>>> --Alex
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>>>>> To: dev@whirr.apache.org
>>>>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and
>>>>>>>>>>>Whirr
>>>>>>>>>>>
>>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container -
>>>>>>>>>>>just
>>>>>>>>>>> add
>>>>>>>>>> the
>>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface
>>>>>>>>>>>(part
>>>>>>>>>>> of
>>>>>>>>>> jclouds cli).
>>>>>>>>>>>
>>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>>>> version.
>>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>
>>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>>>>> conflicting
>>>>>>>>>>> issue.
>>>>>>>>>>>>
>>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>>>>> gson
>>>>>>>>>> 1.7.1.
>>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>>>>
>>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>>>>
>>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack
>>>>>>>>>>>>will
>>>>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>>>>
>>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>>>>>
>>>>>>>>>>>>------------------------------**------------------------------**
>>>>>>>>>>>> ------------
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>>>>> cloud-core
>>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process
>>>>>>>>>>>>(default)
>>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using
>>>>>>>>>>>>'UTF-8'
>>>>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>>>>> 3
>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>>maven-compiler-plugin:2.5.1:**compile
>>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>>>> resources.
>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>>>>> 3
>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1
>>>>>>>>>>>>source
>>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes
>>>>>>>>>>>>[INFO]
>>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>>>>> cloud-core
>>>>>>>>>>>> ---
>>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>>>>> target/surefire-reports
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>> T E S T S
>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>>>> log4j:WARN See
>>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>>>>> http://logging.apa
>>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>>>>
>>>>>>>>>>>> Results :
>>>>>>>>>>>>
>>>>>>>>>>>> Failed tests:
>>>> testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>
>>>>>>>>>>>> Tests in error:
>>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>
>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>>>>> API --launchCluster to CloudStack, the following error related
>>>>>>>>>>>>to
>>>>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>>>>
>>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:)
>>>>>>>>>>>>Caught
>>>>>>>>>>>> problem with
>>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>>>>
>>>>>>>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg
>>>>>>>>y"
>>>>>>>>>>>> :"**
>>>>>>>>>>>>
>>>>>>>>>>>>HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>>>>> *
>>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>>>>> 10.244.18
>>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>>>>>
>>>>>>>>>>>>STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>>>>>
>>>>>>>>>>>>:"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>>>*
>>>>>>>>>>>> *
>>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>>>>>
>>>>>>>>>>>>:2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc
>>>>>>>>>>>>e
>>>>>>>>>>>> ","**
>>>>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>>>>> java:235)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>>>>> )
>>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>>>>> following exception but pushing on
>>>>>>>>>>>>
>>>>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>>>>> then quickly died.
>>>>>>>>>>>> Running on provider cloudstack using identity
>>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>>>>> ZRGXVO1VApG
>>>>>>>>>>>> 6q0a
>>>>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>>>>> [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>>(729061754@qtp-385354117-0:)
>>>>>>>>>>>> Configuring template for
>>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>>>>> 0:)
>>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>>>>> jobtracker
>>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1
>>>>>>>>>>>>node(s)
>>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>>>>
>>>>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>>>>
>>>>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>>>>
>>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>>>>> try {
>>>>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>> VER
>>>>>>>>>>>> E,
>>>>>>>>>>>> null, ex);
>>>>>>>>>>>> }
>>>>>>>>>>>> String[] args = new String[2];
>>>>>>>>>>>> args[0] = "--config";
>>>>>>>>>>>> args[1] = config;
>>>>>>>>>>>>
>>>>>>>>>>>> try {
>>>>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>>>>> Arrays.asList(args));
>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>> VER
>>>>>>>>>>>> E,
>>>>>>>>>>>> null, ex);
>>>>>>>>>>>> }
>>>>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>>>>> response.setOutPut(output);
>>>>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>>>>
>>>>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks loads.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Meng
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>>>
>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
Okay, I officially throw in the towel on this one. I made really good
progress on getting jackson into place, but after I was about 75% done
I just felt it was WAY too big of a change to do at once. If we want
to move to jackson, I would say the best approach would be more
incremental and not big bang style. So gson is here to stay for
awhile. Anybody else want to try upgrading to latest gson?
Darren
On Mon, Sep 23, 2013 at 2:00 PM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
> On Sep 23, 2013, at 2:54 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
>
>> I still think this is the wrong way to implement the Whirr integration.
>
> Probably, but it's too late now.
>
> We can just get it in a branch and look at it
>
>>
>> On 9/23/13 6:45 AM, "sebgoa" <ru...@gmail.com> wrote:
>>
>>>
>>> On Sep 23, 2013, at 7:12 AM, Darren Shepherd
>>> <da...@gmail.com> wrote:
>>>
>>>> The only way this will get committed is if we either move to latest
>>>> gson or
>>>> jackson. Regardless, the outcome will be that Meng can assume this gson
>>>> conflict won't happen. The problem is just how fast can we move all of
>>>> ACS
>>>> to a new json library. Is it possible for Meng to commit to a branch
>>>> that
>>>> just blindly updates gson to the latest (in /pom.xml).
>>>
>>> Yes I will talk to Meng about this. thanks
>>>
>>>> When we get ACS on
>>>> a newer json library, we can merge in that branch. Tomorrow (9/23), I
>>>> can
>>>> attempt to put together a patch to move to jackson. I think I ironed
>>>> out
>>>> most of the technical issues so I just have to globally apply the
>>>> changes.
>>>>
>>>> Darren
>>>>
>>>>
>>>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen
>>>> <ru...@gmail.com>wrote:
>>>>
>>>>>
>>>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>>
>>>>>>
>>>>>> The Nicira NVP plugin is also depending on gson. If we make any
>>>>>> changes
>>>>> we need to validate against their api to ensure that the interface
>>>>> still
>>>>> works.
>>>>>>
>>>>>> If we put the changes in a separate branch i'd be happy to run the
>>>>> changes against the api.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Hugo
>>>>>
>>>>> So how can Meng get out of this bind, she is trying to complete her
>>>>> google
>>>>> summer of code project.
>>>>>
>>>>> thanks,
>>>>>
>>>>> -Sebastien
>>>>>
>>>>>>
>>>>>>
>>>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>
>>>>>>> After much searching I found the two lines of code I needed to get it
>>>>> to serialize right. I'll keep looking and see if I find other snags.
>>>>> Maybe we can just move to jackson.... we'll see, haven't tried to
>>>>> deserialize yet.
>>>>>>>
>>>>>>> Darren
>>>>>>>
>>>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Darren,
>>>>>>>>
>>>>>>>> When I looked into updating to the latest gson, the problem, IIRC,
>>>>>>>> is
>>>>> things that weren't considered cyclical dependencies are suddenly
>>>>> considered cyclical with the latest. I don't recall where though.
>>>>>>>>
>>>>>>>> --Alex
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>>>
>>>>>>>>> Alright, I looked into this and it will take a bit more work to
>>>>> switch to Jackson.
>>>>>>>>> The snag I hit was how we serialize commands for the agents. We
>>>>>>>>> use a
>>>>>>>>> customer typeadatper in gson to produce a format like {
>>>>>>>>> "org.MyClass"
>>>>>>>>> : {...} }. In jackson I don't see producing a similar format as
>>>>> being so straight
>>>>>>>>> forward. I'm sure there's an elegant solution, but I didn't
>>>>> immediately find it.
>>>>>>>>> The problem is if you use a custom serializer in jackson and do
>>>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>>>>> loop
>>>>>>>>> because it will call your serializer again. So if somebody can
>>>>> figure out how to
>>>>>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>>>>>
>>>>>>>>> So, being that it is not an obvious switch to using jackson, what
>>>>>>>>> is
>>>>> the impact
>>>>>>>>> of moving to the latest gson? What were the issue encountered when
>>>>>>>>> people tried it before?
>>>>>>>>>
>>>>>>>>> Darren
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I can do some analysis on this. I'm always up for a terribly
>>>>>>>>>> painful
>>>>>>>>>> refactor :)
>>>>>>>>>>
>>>>>>>>>> Darren
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang
>>>>>>>>>>> <Al...@citrix.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I actually did a quick try to update cloudstack to use newest
>>>>>>>>>>> gson
>>>>>>>>>>> version about 3 months back. Had to roll it back but I didn't
>>>>>>>>>>> try
>>>>>>>>>>> very hard though. Part of the reason why I decided to rollback
>>>>>>>>>>> is
>>>>>>>>>>> due to gson is used differently by various components in
>>>>>>>>>>> CloudStack
>>>>>>>>>>> and I didn't have time to get into each component to see if I
>>>>>>>>>>> break
>>>>>>>>>>> anything. The other is also I thought using Jackson would be
>>>>>>>>>>> much
>>>>>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>>>>>
>>>>>>>>>>> Not that any of these helps you. Just know updating CloudStack
>>>>>>>>>>> to
>>>>>>>>>>> latest gson is not simple.
>>>>>>>>>>>
>>>>>>>>>>> --Alex
>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>>>>>> To: dev@whirr.apache.org
>>>>>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and
>>>>>>>>>>>> Whirr
>>>>>>>>>>>>
>>>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container -
>>>>>>>>>>>> just
>>>>>>>>>>>> add
>>>>>>>>>>> the
>>>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface
>>>>>>>>>>>> (part
>>>>>>>>>>>> of
>>>>>>>>>>> jclouds cli).
>>>>>>>>>>>>
>>>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>>>>> version.
>>>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>>>>>> conflicting
>>>>>>>>>>>> issue.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>>>>>> gson
>>>>>>>>>>> 1.7.1.
>>>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>>>>>
>>>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>>>>>
>>>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack
>>>>>>>>>>>>> will
>>>>>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>>>>>> ------------
>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>>>>>> cloud-core
>>>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process
>>>>>>>>>>>>> (default)
>>>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using
>>>>>>>>>>>>> 'UTF-8'
>>>>>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>>>>>> 3
>>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**compile
>>>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>>>>> resources.
>>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>>>>>> 3
>>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1
>>>>>>>>>>>>> source
>>>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes
>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>>>>>> cloud-core
>>>>>>>>>>>>> ---
>>>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>>>>>> target/surefire-reports
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>>> T E S T S
>>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>>>>> log4j:WARN See
>>>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>>>>>> http://logging.apa
>>>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Results :
>>>>>>>>>>>>>
>>>>>>>>>>>>> Failed tests:
>>>>> testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tests in error:
>>>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>>>>>> API --launchCluster to CloudStack, the following error related
>>>>>>>>>>>>> to
>>>>>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>>>>>
>>>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:)
>>>>>>>>>>>>> Caught
>>>>>>>>>>>>> problem with
>>>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>>>>>
>>>>>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg
>>>>>>>>> y"
>>>>>>>>>>>>> :"**
>>>>>>>>>>>>>
>>>>>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>>>>>> *
>>>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>>>>>> 10.244.18
>>>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>>>>>>
>>>>>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>>>>>>
>>>>>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>>>> *
>>>>>>>>>>>>> *
>>>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>>>>>>
>>>>>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc
>>>>>>>>>>>>> e
>>>>>>>>>>>>> ","**
>>>>>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>>>>>> java:235)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>>>>>> )
>>>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>>>>>> following exception but pushing on
>>>>>>>>>>>>>
>>>>>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>>>>>> then quickly died.
>>>>>>>>>>>>> Running on provider cloudstack using identity
>>>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>>>>>> ZRGXVO1VApG
>>>>>>>>>>>>> 6q0a
>>>>>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>>>>>> [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>>> (729061754@qtp-385354117-0:)
>>>>>>>>>>>>> Configuring template for
>>>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>>>>>> 0:)
>>>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>>>>>> jobtracker
>>>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1
>>>>>>>>>>>>> node(s)
>>>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>>>>>
>>>>>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>>>>>
>>>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>>>>>> try {
>>>>>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>>> VER
>>>>>>>>>>>>> E,
>>>>>>>>>>>>> null, ex);
>>>>>>>>>>>>> }
>>>>>>>>>>>>> String[] args = new String[2];
>>>>>>>>>>>>> args[0] = "--config";
>>>>>>>>>>>>> args[1] = config;
>>>>>>>>>>>>>
>>>>>>>>>>>>> try {
>>>>>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>>>>>> Arrays.asList(args));
>>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>>> VER
>>>>>>>>>>>>> E,
>>>>>>>>>>>>> null, ex);
>>>>>>>>>>>>> }
>>>>>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>>>>>> response.setOutPut(output);
>>>>>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>>>>>
>>>>>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks loads.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Meng
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Sep 23, 2013, at 2:54 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> I still think this is the wrong way to implement the Whirr integration.
Probably, but it's too late now.
We can just get it in a branch and look at it
>
> On 9/23/13 6:45 AM, "sebgoa" <ru...@gmail.com> wrote:
>
>>
>> On Sep 23, 2013, at 7:12 AM, Darren Shepherd
>> <da...@gmail.com> wrote:
>>
>>> The only way this will get committed is if we either move to latest
>>> gson or
>>> jackson. Regardless, the outcome will be that Meng can assume this gson
>>> conflict won't happen. The problem is just how fast can we move all of
>>> ACS
>>> to a new json library. Is it possible for Meng to commit to a branch
>>> that
>>> just blindly updates gson to the latest (in /pom.xml).
>>
>> Yes I will talk to Meng about this. thanks
>>
>>> When we get ACS on
>>> a newer json library, we can merge in that branch. Tomorrow (9/23), I
>>> can
>>> attempt to put together a patch to move to jackson. I think I ironed
>>> out
>>> most of the technical issues so I just have to globally apply the
>>> changes.
>>>
>>> Darren
>>>
>>>
>>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen
>>> <ru...@gmail.com>wrote:
>>>
>>>>
>>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>
>>>>>
>>>>> The Nicira NVP plugin is also depending on gson. If we make any
>>>>> changes
>>>> we need to validate against their api to ensure that the interface
>>>> still
>>>> works.
>>>>>
>>>>> If we put the changes in a separate branch i'd be happy to run the
>>>> changes against the api.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Hugo
>>>>
>>>> So how can Meng get out of this bind, she is trying to complete her
>>>> google
>>>> summer of code project.
>>>>
>>>> thanks,
>>>>
>>>> -Sebastien
>>>>
>>>>>
>>>>>
>>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>
>>>>>> After much searching I found the two lines of code I needed to get it
>>>> to serialize right. I'll keep looking and see if I find other snags.
>>>> Maybe we can just move to jackson.... we'll see, haven't tried to
>>>> deserialize yet.
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Darren,
>>>>>>>
>>>>>>> When I looked into updating to the latest gson, the problem, IIRC,
>>>>>>> is
>>>> things that weren't considered cyclical dependencies are suddenly
>>>> considered cyclical with the latest. I don't recall where though.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>>
>>>>>>>> Alright, I looked into this and it will take a bit more work to
>>>> switch to Jackson.
>>>>>>>> The snag I hit was how we serialize commands for the agents. We
>>>>>>>> use a
>>>>>>>> customer typeadatper in gson to produce a format like {
>>>>>>>> "org.MyClass"
>>>>>>>> : {...} }. In jackson I don't see producing a similar format as
>>>> being so straight
>>>>>>>> forward. I'm sure there's an elegant solution, but I didn't
>>>> immediately find it.
>>>>>>>> The problem is if you use a custom serializer in jackson and do
>>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>>>> loop
>>>>>>>> because it will call your serializer again. So if somebody can
>>>> figure out how to
>>>>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>>>>
>>>>>>>> So, being that it is not an obvious switch to using jackson, what
>>>>>>>> is
>>>> the impact
>>>>>>>> of moving to the latest gson? What were the issue encountered when
>>>>>>>> people tried it before?
>>>>>>>>
>>>>>>>> Darren
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I can do some analysis on this. I'm always up for a terribly
>>>>>>>>> painful
>>>>>>>>> refactor :)
>>>>>>>>>
>>>>>>>>> Darren
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang
>>>>>>>>>> <Al...@citrix.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I actually did a quick try to update cloudstack to use newest
>>>>>>>>>> gson
>>>>>>>>>> version about 3 months back. Had to roll it back but I didn't
>>>>>>>>>> try
>>>>>>>>>> very hard though. Part of the reason why I decided to rollback
>>>>>>>>>> is
>>>>>>>>>> due to gson is used differently by various components in
>>>>>>>>>> CloudStack
>>>>>>>>>> and I didn't have time to get into each component to see if I
>>>>>>>>>> break
>>>>>>>>>> anything. The other is also I thought using Jackson would be
>>>>>>>>>> much
>>>>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>>>>
>>>>>>>>>> Not that any of these helps you. Just know updating CloudStack
>>>>>>>>>> to
>>>>>>>>>> latest gson is not simple.
>>>>>>>>>>
>>>>>>>>>> --Alex
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>>>>> To: dev@whirr.apache.org
>>>>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and
>>>>>>>>>>> Whirr
>>>>>>>>>>>
>>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container -
>>>>>>>>>>> just
>>>>>>>>>>> add
>>>>>>>>>> the
>>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface
>>>>>>>>>>> (part
>>>>>>>>>>> of
>>>>>>>>>> jclouds cli).
>>>>>>>>>>>
>>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>>>> version.
>>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>
>>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>>>>> conflicting
>>>>>>>>>>> issue.
>>>>>>>>>>>>
>>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>>>>> gson
>>>>>>>>>> 1.7.1.
>>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>>>>
>>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>>>>
>>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack
>>>>>>>>>>>> will
>>>>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>>>>
>>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>>>>> ------------
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>>>>> cloud-core
>>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process
>>>>>>>>>>>> (default)
>>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using
>>>>>>>>>>>> 'UTF-8'
>>>>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>>>>> 3
>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**compile
>>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>>>> resources.
>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>>>>> 3
>>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1
>>>>>>>>>>>> source
>>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>>>>> cloud-core
>>>>>>>>>>>> ---
>>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>>>>> target/surefire-reports
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>> T E S T S
>>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>>>> log4j:WARN See
>>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>>>>> http://logging.apa
>>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>>>>
>>>>>>>>>>>> Results :
>>>>>>>>>>>>
>>>>>>>>>>>> Failed tests:
>>>> testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>
>>>>>>>>>>>> Tests in error:
>>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>>
>>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>>>>> API --launchCluster to CloudStack, the following error related
>>>>>>>>>>>> to
>>>>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>>>>
>>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:)
>>>>>>>>>>>> Caught
>>>>>>>>>>>> problem with
>>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>>>>
>>>>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg
>>>>>>>> y"
>>>>>>>>>>>> :"**
>>>>>>>>>>>>
>>>>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>>>>> *
>>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>>>>> 10.244.18
>>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>>>>>
>>>>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>>>>>
>>>>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>>> *
>>>>>>>>>>>> *
>>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>>>>>
>>>>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc
>>>>>>>>>>>> e
>>>>>>>>>>>> ","**
>>>>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>>>>> java:235)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>>>>> )
>>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>>>>> at
>>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>>>>> following exception but pushing on
>>>>>>>>>>>>
>>>>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>>>>> then quickly died.
>>>>>>>>>>>> Running on provider cloudstack using identity
>>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>>>>> ZRGXVO1VApG
>>>>>>>>>>>> 6q0a
>>>>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>>>>> [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>> (729061754@qtp-385354117-0:)
>>>>>>>>>>>> Configuring template for
>>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>>>>> 0:)
>>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>>>>> jobtracker
>>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1
>>>>>>>>>>>> node(s)
>>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>>>>
>>>>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>>>>
>>>>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>>>>
>>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>>>>> try {
>>>>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>> VER
>>>>>>>>>>>> E,
>>>>>>>>>>>> null, ex);
>>>>>>>>>>>> }
>>>>>>>>>>>> String[] args = new String[2];
>>>>>>>>>>>> args[0] = "--config";
>>>>>>>>>>>> args[1] = config;
>>>>>>>>>>>>
>>>>>>>>>>>> try {
>>>>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>>>>> Arrays.asList(args));
>>>>>>>>>>>> } catch (Exception ex) {
>>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>>> VER
>>>>>>>>>>>> E,
>>>>>>>>>>>> null, ex);
>>>>>>>>>>>> }
>>>>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>>>>> response.setOutPut(output);
>>>>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>>>>
>>>>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks loads.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Meng
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>>>
>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I still think this is the wrong way to implement the Whirr integration.
On 9/23/13 6:45 AM, "sebgoa" <ru...@gmail.com> wrote:
>
>On Sep 23, 2013, at 7:12 AM, Darren Shepherd
><da...@gmail.com> wrote:
>
>> The only way this will get committed is if we either move to latest
>>gson or
>> jackson. Regardless, the outcome will be that Meng can assume this gson
>> conflict won't happen. The problem is just how fast can we move all of
>>ACS
>> to a new json library. Is it possible for Meng to commit to a branch
>>that
>> just blindly updates gson to the latest (in /pom.xml).
>
>Yes I will talk to Meng about this. thanks
>
>> When we get ACS on
>> a newer json library, we can merge in that branch. Tomorrow (9/23), I
>>can
>> attempt to put together a patch to move to jackson. I think I ironed
>>out
>> most of the technical issues so I just have to globally apply the
>>changes.
>>
>> Darren
>>
>>
>> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen
>><ru...@gmail.com>wrote:
>>
>>>
>>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>
>>>>
>>>> The Nicira NVP plugin is also depending on gson. If we make any
>>>>changes
>>> we need to validate against their api to ensure that the interface
>>>still
>>> works.
>>>>
>>>> If we put the changes in a separate branch i'd be happy to run the
>>> changes against the api.
>>>>
>>>> Cheers,
>>>>
>>>> Hugo
>>>
>>> So how can Meng get out of this bind, she is trying to complete her
>>>google
>>> summer of code project.
>>>
>>> thanks,
>>>
>>> -Sebastien
>>>
>>>>
>>>>
>>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
>>> darren.s.shepherd@gmail.com> wrote:
>>>>
>>>>> After much searching I found the two lines of code I needed to get it
>>> to serialize right. I'll keep looking and see if I find other snags.
>>> Maybe we can just move to jackson.... we'll see, haven't tried to
>>> deserialize yet.
>>>>>
>>>>> Darren
>>>>>
>>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com>
>>>>>>wrote:
>>>>>>
>>>>>> Darren,
>>>>>>
>>>>>> When I looked into updating to the latest gson, the problem, IIRC,
>>>>>>is
>>> things that weren't considered cyclical dependencies are suddenly
>>> considered cyclical with the latest. I don't recall where though.
>>>>>>
>>>>>> --Alex
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>
>>>>>>> Alright, I looked into this and it will take a bit more work to
>>> switch to Jackson.
>>>>>>> The snag I hit was how we serialize commands for the agents. We
>>>>>>>use a
>>>>>>> customer typeadatper in gson to produce a format like {
>>>>>>>"org.MyClass"
>>>>>>> : {...} }. In jackson I don't see producing a similar format as
>>> being so straight
>>>>>>> forward. I'm sure there's an elegant solution, but I didn't
>>> immediately find it.
>>>>>>> The problem is if you use a custom serializer in jackson and do
>>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>>> loop
>>>>>>> because it will call your serializer again. So if somebody can
>>> figure out how to
>>>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>>>
>>>>>>> So, being that it is not an obvious switch to using jackson, what
>>>>>>>is
>>> the impact
>>>>>>> of moving to the latest gson? What were the issue encountered when
>>>>>>> people tried it before?
>>>>>>>
>>>>>>> Darren
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>>
>>>>>>>> I can do some analysis on this. I'm always up for a terribly
>>>>>>>>painful
>>>>>>>> refactor :)
>>>>>>>>
>>>>>>>> Darren
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang
>>>>>>>>><Al...@citrix.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I actually did a quick try to update cloudstack to use newest
>>>>>>>>>gson
>>>>>>>>> version about 3 months back. Had to roll it back but I didn't
>>>>>>>>>try
>>>>>>>>> very hard though. Part of the reason why I decided to rollback
>>>>>>>>>is
>>>>>>>>> due to gson is used differently by various components in
>>>>>>>>>CloudStack
>>>>>>>>> and I didn't have time to get into each component to see if I
>>>>>>>>>break
>>>>>>>>> anything. The other is also I thought using Jackson would be
>>>>>>>>>much
>>>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>>>
>>>>>>>>> Not that any of these helps you. Just know updating CloudStack
>>>>>>>>>to
>>>>>>>>> latest gson is not simple.
>>>>>>>>>
>>>>>>>>> --Alex
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>>>> To: dev@whirr.apache.org
>>>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and
>>>>>>>>>>Whirr
>>>>>>>>>>
>>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container -
>>>>>>>>>>just
>>>>>>>>>> add
>>>>>>>>> the
>>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface
>>>>>>>>>>(part
>>>>>>>>>> of
>>>>>>>>> jclouds cli).
>>>>>>>>>>
>>>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>>> version.
>>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Dear all,
>>>>>>>>>>>
>>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>>>> conflicting
>>>>>>>>>> issue.
>>>>>>>>>>>
>>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>>>> gson
>>>>>>>>> 1.7.1.
>>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>>>
>>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>>>
>>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack
>>>>>>>>>>>will
>>>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>>>
>>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>>>>
>>>>>>>>>>>------------------------------**------------------------------**
>>>>>>>>>>> ------------
>>>>>>>>>>> [INFO]
>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>>>> cloud-core
>>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>>>> [INFO]
>>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process
>>>>>>>>>>>(default)
>>>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using
>>>>>>>>>>>'UTF-8'
>>>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>>>> 3
>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>>maven-compiler-plugin:2.5.1:**compile
>>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>>> resources.
>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>>>> 3
>>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1
>>>>>>>>>>>source
>>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes
>>>>>>>>>>>[INFO]
>>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>>>> cloud-core
>>>>>>>>>>> ---
>>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>>>> target/surefire-reports
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>> T E S T S
>>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>>> log4j:WARN See
>>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>>>> http://logging.apa
>>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>>>
>>>>>>>>>>> Results :
>>>>>>>>>>>
>>>>>>>>>>> Failed tests:
>>> testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>
>>>>>>>>>>> Tests in error:
>>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>>
>>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>>>> API --launchCluster to CloudStack, the following error related
>>>>>>>>>>>to
>>>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>>>
>>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:)
>>>>>>>>>>>Caught
>>>>>>>>>>> problem with
>>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>>>
>>>>>>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrateg
>>>>>>>y"
>>>>>>>>>>> :"**
>>>>>>>>>>>
>>>>>>>>>>>HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>>>> *
>>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>>>> 10.244.18
>>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>>>>
>>>>>>>>>>>STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>>>>
>>>>>>>>>>>:"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>>>>*
>>>>>>>>>>> *
>>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>>>>
>>>>>>>>>>>:2,"version":"4.1.1","**resourceName":"**LibvirtComputingResourc
>>>>>>>>>>>e
>>>>>>>>>>> ","**
>>>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>>>> at
>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>>>> at
>>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>>>> at
>>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>>>> java:235)
>>>>>>>>>>> at
>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>>>> at
>>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>>>> )
>>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>>>> following exception but pushing on
>>>>>>>>>>>
>>>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>>>> then quickly died.
>>>>>>>>>>> Running on provider cloudstack using identity
>>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>>>> ZRGXVO1VApG
>>>>>>>>>>> 6q0a
>>>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>>>> [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>>(729061754@qtp-385354117-0:)
>>>>>>>>>>> Configuring template for
>>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>>>> 0:)
>>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>>>> jobtracker
>>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1
>>>>>>>>>>>node(s)
>>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>>>
>>>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>>>
>>>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>>>
>>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>>>> try {
>>>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>>>> } catch (Exception ex) {
>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>> VER
>>>>>>>>>>> E,
>>>>>>>>>>> null, ex);
>>>>>>>>>>> }
>>>>>>>>>>> String[] args = new String[2];
>>>>>>>>>>> args[0] = "--config";
>>>>>>>>>>> args[1] = config;
>>>>>>>>>>>
>>>>>>>>>>> try {
>>>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>>>> Arrays.asList(args));
>>>>>>>>>>> } catch (Exception ex) {
>>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>>> VER
>>>>>>>>>>> E,
>>>>>>>>>>> null, ex);
>>>>>>>>>>> }
>>>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>>>> response.setOutPut(output);
>>>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>>>
>>>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>>>
>>>>>>>>>>> Thanks loads.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Meng
>>>>>>>>
>>>>>>>>
>>>>
>>>
>>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by sebgoa <ru...@gmail.com>.
On Sep 23, 2013, at 7:12 AM, Darren Shepherd <da...@gmail.com> wrote:
> The only way this will get committed is if we either move to latest gson or
> jackson. Regardless, the outcome will be that Meng can assume this gson
> conflict won't happen. The problem is just how fast can we move all of ACS
> to a new json library. Is it possible for Meng to commit to a branch that
> just blindly updates gson to the latest (in /pom.xml).
Yes I will talk to Meng about this. thanks
> When we get ACS on
> a newer json library, we can merge in that branch. Tomorrow (9/23), I can
> attempt to put together a patch to move to jackson. I think I ironed out
> most of the technical issues so I just have to globally apply the changes.
>
> Darren
>
>
> On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen <ru...@gmail.com>wrote:
>
>>
>> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>
>>>
>>> The Nicira NVP plugin is also depending on gson. If we make any changes
>> we need to validate against their api to ensure that the interface still
>> works.
>>>
>>> If we put the changes in a separate branch i'd be happy to run the
>> changes against the api.
>>>
>>> Cheers,
>>>
>>> Hugo
>>
>> So how can Meng get out of this bind, she is trying to complete her google
>> summer of code project.
>>
>> thanks,
>>
>> -Sebastien
>>
>>>
>>>
>>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
>> darren.s.shepherd@gmail.com> wrote:
>>>
>>>> After much searching I found the two lines of code I needed to get it
>> to serialize right. I'll keep looking and see if I find other snags.
>> Maybe we can just move to jackson.... we'll see, haven't tried to
>> deserialize yet.
>>>>
>>>> Darren
>>>>
>>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
>>>>>
>>>>> Darren,
>>>>>
>>>>> When I looked into updating to the latest gson, the problem, IIRC, is
>> things that weren't considered cyclical dependencies are suddenly
>> considered cyclical with the latest. I don't recall where though.
>>>>>
>>>>> --Alex
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>
>>>>>> Alright, I looked into this and it will take a bit more work to
>> switch to Jackson.
>>>>>> The snag I hit was how we serialize commands for the agents. We use a
>>>>>> customer typeadatper in gson to produce a format like { "org.MyClass"
>>>>>> : {...} }. In jackson I don't see producing a similar format as
>> being so straight
>>>>>> forward. I'm sure there's an elegant solution, but I didn't
>> immediately find it.
>>>>>> The problem is if you use a custom serializer in jackson and do
>>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>> loop
>>>>>> because it will call your serializer again. So if somebody can
>> figure out how to
>>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>>
>>>>>> So, being that it is not an obvious switch to using jackson, what is
>> the impact
>>>>>> of moving to the latest gson? What were the issue encountered when
>>>>>> people tried it before?
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>>
>>>>>>> I can do some analysis on this. I'm always up for a terribly painful
>>>>>>> refactor :)
>>>>>>>
>>>>>>> Darren
>>>>>>>
>>>>>>>
>>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I actually did a quick try to update cloudstack to use newest gson
>>>>>>>> version about 3 months back. Had to roll it back but I didn't try
>>>>>>>> very hard though. Part of the reason why I decided to rollback is
>>>>>>>> due to gson is used differently by various components in CloudStack
>>>>>>>> and I didn't have time to get into each component to see if I break
>>>>>>>> anything. The other is also I thought using Jackson would be much
>>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>>
>>>>>>>> Not that any of these helps you. Just know updating CloudStack to
>>>>>>>> latest gson is not simple.
>>>>>>>>
>>>>>>>> --Alex
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>>> To: dev@whirr.apache.org
>>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>>>
>>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
>>>>>>>>> add
>>>>>>>> the
>>>>>>>>> feature url. Bonus: you can also use jclouds shell interface (part
>>>>>>>>> of
>>>>>>>> jclouds cli).
>>>>>>>>>
>>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>> version.
>>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>>
>>>>>>>>>> Dear all,
>>>>>>>>>>
>>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>>> conflicting
>>>>>>>>> issue.
>>>>>>>>>>
>>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>>> gson
>>>>>>>> 1.7.1.
>>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>>
>>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>>
>>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
>>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>>
>>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>>> ------------
>>>>>>>>>> [INFO]
>>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>>> cloud-core
>>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>>> [INFO]
>>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
>>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>>> 3
>>>>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>> resources.
>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>>> 3
>>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
>>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
>>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>>> cloud-core
>>>>>>>>>> ---
>>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>>> target/surefire-reports
>>>>>>>>>>
>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>> T E S T S
>>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>> log4j:WARN See
>>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>>> http://logging.apa
>>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>>
>>>>>>>>>> Results :
>>>>>>>>>>
>>>>>>>>>> Failed tests:
>> testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>
>>>>>>>>>> Tests in error:
>>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>>
>>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>>> API --launchCluster to CloudStack, the following error related to
>>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>>
>>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>>>>>>>>>> problem with
>>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>>>>>>>>>> :"**
>>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>>> *
>>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>>> 10.244.18
>>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
>>>>>>>>>> *
>>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
>>>>>>>>>> ","**
>>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>>> at
>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>>> at
>>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>>> at
>>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>>> java:235)
>>>>>>>>>> at
>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>>> at
>>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>>> )
>>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>>> following exception but pushing on
>>>>>>>>>>
>>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>>> then quickly died.
>>>>>>>>>> Running on provider cloudstack using identity
>>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>>> ZRGXVO1VApG
>>>>>>>>>> 6q0a
>>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>>>>>>>>>> Configuring template for
>>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>>> 0:)
>>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>>> jobtracker
>>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>>
>>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>>
>>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>>
>>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>>> try {
>>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>>> } catch (Exception ex) {
>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>> VER
>>>>>>>>>> E,
>>>>>>>>>> null, ex);
>>>>>>>>>> }
>>>>>>>>>> String[] args = new String[2];
>>>>>>>>>> args[0] = "--config";
>>>>>>>>>> args[1] = config;
>>>>>>>>>>
>>>>>>>>>> try {
>>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>>> Arrays.asList(args));
>>>>>>>>>> } catch (Exception ex) {
>>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>>> VER
>>>>>>>>>> E,
>>>>>>>>>> null, ex);
>>>>>>>>>> }
>>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>>> response.setOutPut(output);
>>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>>
>>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>>
>>>>>>>>>> Thanks loads.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Meng
>>>>>>>
>>>>>>>
>>>
>>
>>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
The only way this will get committed is if we either move to latest gson or
jackson. Regardless, the outcome will be that Meng can assume this gson
conflict won't happen. The problem is just how fast can we move all of ACS
to a new json library. Is it possible for Meng to commit to a branch that
just blindly updates gson to the latest (in /pom.xml). When we get ACS on
a newer json library, we can merge in that branch. Tomorrow (9/23), I can
attempt to put together a patch to move to jackson. I think I ironed out
most of the technical issues so I just have to globally apply the changes.
Darren
On Fri, Sep 20, 2013 at 1:09 AM, Sebastien Goasguen <ru...@gmail.com>wrote:
>
> On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>
> >
> > The Nicira NVP plugin is also depending on gson. If we make any changes
> we need to validate against their api to ensure that the interface still
> works.
> >
> > If we put the changes in a separate branch i'd be happy to run the
> changes against the api.
> >
> > Cheers,
> >
> > Hugo
>
> So how can Meng get out of this bind, she is trying to complete her google
> summer of code project.
>
> thanks,
>
> -Sebastien
>
> >
> >
> > On Sep 20, 2013, at 12:08 PM, Darren Shepherd <
> darren.s.shepherd@gmail.com> wrote:
> >
> >> After much searching I found the two lines of code I needed to get it
> to serialize right. I'll keep looking and see if I find other snags.
> Maybe we can just move to jackson.... we'll see, haven't tried to
> deserialize yet.
> >>
> >> Darren
> >>
> >>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
> >>>
> >>> Darren,
> >>>
> >>> When I looked into updating to the latest gson, the problem, IIRC, is
> things that weren't considered cyclical dependencies are suddenly
> considered cyclical with the latest. I don't recall where though.
> >>>
> >>> --Alex
> >>>
> >>>> -----Original Message-----
> >>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> >>>> Sent: Thursday, September 19, 2013 1:33 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
> >>>>
> >>>> Alright, I looked into this and it will take a bit more work to
> switch to Jackson.
> >>>> The snag I hit was how we serialize commands for the agents. We use a
> >>>> customer typeadatper in gson to produce a format like { "org.MyClass"
> >>>> : {...} }. In jackson I don't see producing a similar format as
> being so straight
> >>>> forward. I'm sure there's an elegant solution, but I didn't
> immediately find it.
> >>>> The problem is if you use a custom serializer in jackson and do
> >>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
> loop
> >>>> because it will call your serializer again. So if somebody can
> figure out how to
> >>>> do this format easily in jackson, the rest looks simple enough.
> >>>>
> >>>> So, being that it is not an obvious switch to using jackson, what is
> the impact
> >>>> of moving to the latest gson? What were the issue encountered when
> >>>> people tried it before?
> >>>>
> >>>> Darren
> >>>>
> >>>>
> >>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
> >>>> darren.s.shepherd@gmail.com> wrote:
> >>>>
> >>>>> I can do some analysis on this. I'm always up for a terribly painful
> >>>>> refactor :)
> >>>>>
> >>>>> Darren
> >>>>>
> >>>>>
> >>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
> >>>>> wrote:
> >>>>>
> >>>>>> I actually did a quick try to update cloudstack to use newest gson
> >>>>>> version about 3 months back. Had to roll it back but I didn't try
> >>>>>> very hard though. Part of the reason why I decided to rollback is
> >>>>>> due to gson is used differently by various components in CloudStack
> >>>>>> and I didn't have time to get into each component to see if I break
> >>>>>> anything. The other is also I thought using Jackson would be much
> >>>>>> better and thought our next attempt should be with Jackson.
> >>>>>>
> >>>>>> Not that any of these helps you. Just know updating CloudStack to
> >>>>>> latest gson is not simple.
> >>>>>>
> >>>>>> --Alex
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
> >>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
> >>>>>>> To: dev@whirr.apache.org
> >>>>>>> Cc: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
> >>>>>>>
> >>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
> >>>>>>> add
> >>>>>> the
> >>>>>>> feature url. Bonus: you can also use jclouds shell interface (part
> >>>>>>> of
> >>>>>> jclouds cli).
> >>>>>>>
> >>>>>>> Another option is to upgrade the CloudStack API to use the new
> version.
> >>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
> >>>>>>>>
> >>>>>>>> Dear all,
> >>>>>>>>
> >>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
> >>>>>>>> various clusters on CloudStack. Now I am facing a dependency
> >>>>>> conflicting
> >>>>>>> issue.
> >>>>>>>>
> >>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
> >>>>>>>> gson
> >>>>>> 1.7.1.
> >>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
> >>>>>>>>
> >>>>>>>> com.google.common.util.**concurrent.ExecutionError:
> >>>>>>> java.lang.**NoClassDefFoundError:
> >>>>>>>> com/google/gson/TypeAdapter
> >>>>>>>>
> >>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
> >>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
> >>>>>>>> not build successfully due to the following test error.
> >>>>>>>>
> >>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> >>>>>>>> ------------------------------**------------------------------**
> >>>>>>>> ------------
> >>>>>>>> [INFO]
> >>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
> >>>>>>>> cloud-core
> >>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
> >>>>>>>> [INFO]
> >>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
> >>>>>>>> @ cloud-core --- [INFO] [INFO] ---
> >>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
> >>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> >>>>>>>> encoding to copy filtered resources.
> >>>>>>>> [INFO] skip non existing resourceDirectory
> >>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
> >>>> 3
> >>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> >>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
> >>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
> >>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
> >>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
> >>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
> resources.
> >>>>>>>> [INFO] skip non existing resourceDirectory
> >>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
> >>>> 3
> >>>>>>>> resources [INFO] [INFO] ---
> >>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
> >>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
> >>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
> >>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
> >>>>>>>> cloud-core
> >>>>>>>> ---
> >>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> >>>>>>>> target/surefire-reports
> >>>>>>>>
> >>>>>>>> ------------------------------**-------------------------
> >>>>>>>> T E S T S
> >>>>>>>> ------------------------------**-------------------------
> >>>>>>>> Running com.cloud.agent.transport.**RequestTest
> >>>>>>>> log4j:WARN No appenders could be found for logger
> >>>>>>>> (com.cloud.agent.transport.**RequestTest).
> >>>>>>>> log4j:WARN Please initialize the log4j system properly.
> >>>>>>>> log4j:WARN See
> >>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
> >>>>>> http://logging.apa
> >>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
> >>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
> >>>>>>>> 3.054 sec <<< FAILURE!
> >>>>>>>>
> >>>>>>>> Results :
> >>>>>>>>
> >>>>>>>> Failed tests:
> testSerDeser(com.cloud.agent.**transport.RequestTest)
> >>>>>>>>
> >>>>>>>> Tests in error:
> >>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
> >>>>>>>>
> >>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
> >>>>>>>> CloudStack building process went through. But when I issued an
> >>>>>>>> API --launchCluster to CloudStack, the following error related to
> >>>>>>>> gson popped up on CloudStack Management Server:
> >>>>>>>>
> >>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> >>>>>>>> problem with
> >>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> >>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> >>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> >>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> >>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
> >>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
> >>>>>>>> :"**
> >>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> >>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> >>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> >>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
> >>>>>>> **LibvirtComputingResource","*
> >>>>>>>> *
> >>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> >>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> >>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> >>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> >>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> >>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> >>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> >>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
> >>>>>>>> 10.244.18
> >>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> >>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> >>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> >>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> >>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
> >>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> >>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> >>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
> >>>>>>>> *
> >>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> >>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
> >>>>>>>> ","**
> >>>>>>>> contextMap":{},"wait":0}}]
> >>>>>>>> java.lang.ClassCastException:
> >>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> >>>>>>>> cannot be cast to java.lang.Class
> >>>>>>>> at
> >>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> >>>>>>>> ArrayTypeAdaptor.java:84)
> >>>>>>>> at
> >>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> >>>>>>>> ArrayTypeAdaptor.java:37)
> >>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
> >>>>>>>> TreeTypeAdapter.java:58)
> >>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
> >>>>>>>> at
> >>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
> >>>>>>>> java:235)
> >>>>>>>> at
> >>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> >>>>>>>> processRequest(**AgentManagerImpl.java:1221)
> >>>>>>>> at
> >>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> >>>>>>>> doTask(AgentManagerImpl.java:**1374)
> >>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> >>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
> >>>>>>> )
> >>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
> >>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> >>>>>>>> ThreadPoolExecutor.java:1110)
> >>>>>>>> at
> >>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> >>>>>>>> ThreadPoolExecutor.java:603)
> >>>>>>>> at java.lang.Thread.run(Thread.**java:722)
> >>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
> >>>>>>>> following exception but pushing on
> >>>>>>>>
> >>>>>>>> On the CS Managment console I can see the cluster was started
> >>>>>>>> then quickly died.
> >>>>>>>> Running on provider cloudstack using identity
> >>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
> >>>>>>> ZRGXVO1VApG
> >>>>>>>> 6q0a
> >>>>>>>> **K-A-tQRQsZFwnOXQ
> >>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
> >>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
> >>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> >>>>>>>> Configuring template for
> >>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
> >>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
> >>>>>>>> (729061754@qtp-385354117-
> >>>>>>> 0:)
> >>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
> >>>>>>> jobtracker
> >>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
> >>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
> >>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> >>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
> >>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
> >>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
> >>>>>>>>
> >>>>>>>> I attached the debuging log to this email.
> >>>>>>>>
> >>>>>>>> The launchCluster API is simple, it calls the
> >>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
> >>>>>>>>
> >>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
> >>>>>>>> response.setObjectName("**launchCluster");
> >>>>>>>> LaunchClusterCommand command = null;
> >>>>>>>> try {
> >>>>>>>> command = new LaunchClusterCommand();
> >>>>>>>> } catch (Exception ex) {
> >>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
> >>>>>>> VER
> >>>>>>>> E,
> >>>>>>>> null, ex);
> >>>>>>>> }
> >>>>>>>> String[] args = new String[2];
> >>>>>>>> args[0] = "--config";
> >>>>>>>> args[1] = config;
> >>>>>>>>
> >>>>>>>> try {
> >>>>>>>> command.run(System.in, System.out, System.err,
> >>>>>>>> Arrays.asList(args));
> >>>>>>>> } catch (Exception ex) {
> >>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
> >>>>>>> VER
> >>>>>>>> E,
> >>>>>>>> null, ex);
> >>>>>>>> }
> >>>>>>>> response.setResponseName(**getCommandName());
> >>>>>>>> output = "successfully launched the cluster.";
> >>>>>>>> response.setOutPut(output);
> >>>>>>>> response.setAsync(Boolean.**FALSE);
> >>>>>>>> this.setResponseObject(**response);
> >>>>>>>>
> >>>>>>>> Could someone help me out here? What would be a proper gson
> >>>>>>>> version for CloudStack and Whirr to run at the same time?
> >>>>>>>>
> >>>>>>>> Thanks loads.
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Meng
> >>>>>
> >>>>>
> >
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I don't think adding an API to CloudStack is particularly appropriate way
to integrate with whirr? Whirr should *use* the CloudStack API, not add to
it?
On 9/20/13 1:09 AM, "Sebastien Goasguen" <ru...@gmail.com> wrote:
>
>On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>
>>
>> The Nicira NVP plugin is also depending on gson. If we make any changes
>>we need to validate against their api to ensure that the interface still
>>works.
>>
>> If we put the changes in a separate branch i'd be happy to run the
>>changes against the api.
>>
>> Cheers,
>>
>> Hugo
>
>So how can Meng get out of this bind, she is trying to complete her
>google summer of code project.
>
>thanks,
>
>-Sebastien
>
>>
>>
>> On Sep 20, 2013, at 12:08 PM, Darren Shepherd
>><da...@gmail.com> wrote:
>>
>>> After much searching I found the two lines of code I needed to get it
>>>to serialize right. I'll keep looking and see if I find other snags.
>>>Maybe we can just move to jackson.... we'll see, haven't tried to
>>>deserialize yet.
>>>
>>> Darren
>>>
>>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
>>>>
>>>> Darren,
>>>>
>>>> When I looked into updating to the latest gson, the problem, IIRC, is
>>>>things that weren't considered cyclical dependencies are suddenly
>>>>considered cyclical with the latest. I don't recall where though.
>>>>
>>>> --Alex
>>>>
>>>>> -----Original Message-----
>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>
>>>>> Alright, I looked into this and it will take a bit more work to
>>>>>switch to Jackson.
>>>>> The snag I hit was how we serialize commands for the agents. We use
>>>>>a
>>>>> customer typeadatper in gson to produce a format like { "org.MyClass"
>>>>> : {...} }. In jackson I don't see producing a similar format as
>>>>>being so straight
>>>>> forward. I'm sure there's an elegant solution, but I didn't
>>>>>immediately find it.
>>>>> The problem is if you use a custom serializer in jackson and do
>>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a
>>>>>loop
>>>>> because it will call your serializer again. So if somebody can
>>>>>figure out how to
>>>>> do this format easily in jackson, the rest looks simple enough.
>>>>>
>>>>> So, being that it is not an obvious switch to using jackson, what is
>>>>>the impact
>>>>> of moving to the latest gson? What were the issue encountered when
>>>>> people tried it before?
>>>>>
>>>>> Darren
>>>>>
>>>>>
>>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>>
>>>>>> I can do some analysis on this. I'm always up for a terribly
>>>>>>painful
>>>>>> refactor :)
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>>
>>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I actually did a quick try to update cloudstack to use newest gson
>>>>>>> version about 3 months back. Had to roll it back but I didn't try
>>>>>>> very hard though. Part of the reason why I decided to rollback is
>>>>>>> due to gson is used differently by various components in CloudStack
>>>>>>> and I didn't have time to get into each component to see if I break
>>>>>>> anything. The other is also I thought using Jackson would be much
>>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>>
>>>>>>> Not that any of these helps you. Just know updating CloudStack to
>>>>>>> latest gson is not simple.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>>> To: dev@whirr.apache.org
>>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>>
>>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
>>>>>>>> add
>>>>>>> the
>>>>>>>> feature url. Bonus: you can also use jclouds shell interface (part
>>>>>>>> of
>>>>>>> jclouds cli).
>>>>>>>>
>>>>>>>> Another option is to upgrade the CloudStack API to use the new
>>>>>>>>version.
>>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>>
>>>>>>>>> Dear all,
>>>>>>>>>
>>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>>> conflicting
>>>>>>>> issue.
>>>>>>>>>
>>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>>> gson
>>>>>>> 1.7.1.
>>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>>
>>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>>
>>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
>>>>>>>>> not build successfully due to the following test error.
>>>>>>>>>
>>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>> ------------
>>>>>>>>> [INFO]
>>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>>> cloud-core
>>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>>> [INFO]
>>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
>>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>>>>>>>>> encoding to copy filtered resources.
>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>>> 3
>>>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered
>>>>>>>>>resources.
>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>>> 3
>>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
>>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
>>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>>> cloud-core
>>>>>>>>> ---
>>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>>> target/surefire-reports
>>>>>>>>>
>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>> T E S T S
>>>>>>>>> ------------------------------**-------------------------
>>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>> log4j:WARN See
>>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>>> http://logging.apa
>>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>>
>>>>>>>>> Results :
>>>>>>>>>
>>>>>>>>> Failed tests:
>>>>>>>>>testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>
>>>>>>>>> Tests in error:
>>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>>
>>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>>> API --launchCluster to CloudStack, the following error related to
>>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>>
>>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>>>>>>>>> problem with
>>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>>>
>>>>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>>>>>>>>> :"**
>>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>>> **LibvirtComputingResource","*
>>>>>>>>> *
>>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>>> 10.244.18
>>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
>>>>>>>>> *
>>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
>>>>>>>>> ","**
>>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>>> java.lang.ClassCastException:
>>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>>> at
>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>>> at
>>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>>> at
>>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>>> java:235)
>>>>>>>>> at
>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>>> at
>>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>>> )
>>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>>> following exception but pushing on
>>>>>>>>>
>>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>>> then quickly died.
>>>>>>>>> Running on provider cloudstack using identity
>>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>>> ZRGXVO1VApG
>>>>>>>>> 6q0a
>>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>>>>>>>>> Configuring template for
>>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>>> (729061754@qtp-385354117-
>>>>>>>> 0:)
>>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>>> jobtracker
>>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>>
>>>>>>>>> I attached the debuging log to this email.
>>>>>>>>>
>>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>>
>>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>>> try {
>>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>>> } catch (Exception ex) {
>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>> VER
>>>>>>>>> E,
>>>>>>>>> null, ex);
>>>>>>>>> }
>>>>>>>>> String[] args = new String[2];
>>>>>>>>> args[0] = "--config";
>>>>>>>>> args[1] = config;
>>>>>>>>>
>>>>>>>>> try {
>>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>>> Arrays.asList(args));
>>>>>>>>> } catch (Exception ex) {
>>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>>> VER
>>>>>>>>> E,
>>>>>>>>> null, ex);
>>>>>>>>> }
>>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>>> response.setOutPut(output);
>>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>>> this.setResponseObject(**response);
>>>>>>>>>
>>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>>
>>>>>>>>> Thanks loads.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Meng
>>>>>>
>>>>>>
>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Sep 20, 2013, at 12:45 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>
> The Nicira NVP plugin is also depending on gson. If we make any changes we need to validate against their api to ensure that the interface still works.
>
> If we put the changes in a separate branch i'd be happy to run the changes against the api.
>
> Cheers,
>
> Hugo
So how can Meng get out of this bind, she is trying to complete her google summer of code project.
thanks,
-Sebastien
>
>
> On Sep 20, 2013, at 12:08 PM, Darren Shepherd <da...@gmail.com> wrote:
>
>> After much searching I found the two lines of code I needed to get it to serialize right. I'll keep looking and see if I find other snags. Maybe we can just move to jackson.... we'll see, haven't tried to deserialize yet.
>>
>> Darren
>>
>>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
>>>
>>> Darren,
>>>
>>> When I looked into updating to the latest gson, the problem, IIRC, is things that weren't considered cyclical dependencies are suddenly considered cyclical with the latest. I don't recall where though.
>>>
>>> --Alex
>>>
>>>> -----Original Message-----
>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>> Sent: Thursday, September 19, 2013 1:33 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>
>>>> Alright, I looked into this and it will take a bit more work to switch to Jackson.
>>>> The snag I hit was how we serialize commands for the agents. We use a
>>>> customer typeadatper in gson to produce a format like { "org.MyClass"
>>>> : {...} }. In jackson I don't see producing a similar format as being so straight
>>>> forward. I'm sure there's an elegant solution, but I didn't immediately find it.
>>>> The problem is if you use a custom serializer in jackson and do
>>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop
>>>> because it will call your serializer again. So if somebody can figure out how to
>>>> do this format easily in jackson, the rest looks simple enough.
>>>>
>>>> So, being that it is not an obvious switch to using jackson, what is the impact
>>>> of moving to the latest gson? What were the issue encountered when
>>>> people tried it before?
>>>>
>>>> Darren
>>>>
>>>>
>>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>>> darren.s.shepherd@gmail.com> wrote:
>>>>
>>>>> I can do some analysis on this. I'm always up for a terribly painful
>>>>> refactor :)
>>>>>
>>>>> Darren
>>>>>
>>>>>
>>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
>>>>> wrote:
>>>>>
>>>>>> I actually did a quick try to update cloudstack to use newest gson
>>>>>> version about 3 months back. Had to roll it back but I didn't try
>>>>>> very hard though. Part of the reason why I decided to rollback is
>>>>>> due to gson is used differently by various components in CloudStack
>>>>>> and I didn't have time to get into each component to see if I break
>>>>>> anything. The other is also I thought using Jackson would be much
>>>>>> better and thought our next attempt should be with Jackson.
>>>>>>
>>>>>> Not that any of these helps you. Just know updating CloudStack to
>>>>>> latest gson is not simple.
>>>>>>
>>>>>> --Alex
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>>> To: dev@whirr.apache.org
>>>>>>> Cc: dev@cloudstack.apache.org
>>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>>
>>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
>>>>>>> add
>>>>>> the
>>>>>>> feature url. Bonus: you can also use jclouds shell interface (part
>>>>>>> of
>>>>>> jclouds cli).
>>>>>>>
>>>>>>> Another option is to upgrade the CloudStack API to use the new version.
>>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>>> conflicting
>>>>>>> issue.
>>>>>>>>
>>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>>> gson
>>>>>> 1.7.1.
>>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>>
>>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>>> com/google/gson/TypeAdapter
>>>>>>>>
>>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
>>>>>>>> not build successfully due to the following test error.
>>>>>>>>
>>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>>> ------------------------------**------------------------------**
>>>>>>>> ------------
>>>>>>>> [INFO]
>>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>>> cloud-core
>>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>>> [INFO]
>>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
>>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>>>>>>>> encoding to copy filtered resources.
>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>>> 3
>>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>>> 3
>>>>>>>> resources [INFO] [INFO] ---
>>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
>>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
>>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>>> cloud-core
>>>>>>>> ---
>>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>>> target/surefire-reports
>>>>>>>>
>>>>>>>> ------------------------------**-------------------------
>>>>>>>> T E S T S
>>>>>>>> ------------------------------**-------------------------
>>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>> log4j:WARN See
>>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>>> http://logging.apa
>>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>>
>>>>>>>> Results :
>>>>>>>>
>>>>>>>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>>
>>>>>>>> Tests in error:
>>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>>
>>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>>
>>>>>>>>
>>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>>> API --launchCluster to CloudStack, the following error related to
>>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>>
>>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>>>>>>>> problem with
>>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>>>>>>>> :"**
>>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>>> **LibvirtComputingResource","*
>>>>>>>> *
>>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>>> 10.244.18
>>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
>>>>>>>> *
>>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
>>>>>>>> ","**
>>>>>>>> contextMap":{},"wait":0}}]
>>>>>>>> java.lang.ClassCastException:
>>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>>> cannot be cast to java.lang.Class
>>>>>>>> at
>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>>> at
>>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>>> TreeTypeAdapter.java:58)
>>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>>> at
>>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>>> java:235)
>>>>>>>> at
>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>>> at
>>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>>> )
>>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>>> at
>>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>>> following exception but pushing on
>>>>>>>>
>>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>>> then quickly died.
>>>>>>>> Running on provider cloudstack using identity
>>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>>> ZRGXVO1VApG
>>>>>>>> 6q0a
>>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>>>>>>>> Configuring template for
>>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>>> (729061754@qtp-385354117-
>>>>>>> 0:)
>>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>>> jobtracker
>>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>>
>>>>>>>> I attached the debuging log to this email.
>>>>>>>>
>>>>>>>> The launchCluster API is simple, it calls the
>>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>>
>>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>>> response.setObjectName("**launchCluster");
>>>>>>>> LaunchClusterCommand command = null;
>>>>>>>> try {
>>>>>>>> command = new LaunchClusterCommand();
>>>>>>>> } catch (Exception ex) {
>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>> VER
>>>>>>>> E,
>>>>>>>> null, ex);
>>>>>>>> }
>>>>>>>> String[] args = new String[2];
>>>>>>>> args[0] = "--config";
>>>>>>>> args[1] = config;
>>>>>>>>
>>>>>>>> try {
>>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>>> Arrays.asList(args));
>>>>>>>> } catch (Exception ex) {
>>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>>> VER
>>>>>>>> E,
>>>>>>>> null, ex);
>>>>>>>> }
>>>>>>>> response.setResponseName(**getCommandName());
>>>>>>>> output = "successfully launched the cluster.";
>>>>>>>> response.setOutPut(output);
>>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>>> this.setResponseObject(**response);
>>>>>>>>
>>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>>
>>>>>>>> Thanks loads.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Meng
>>>>>
>>>>>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Hugo Trippaers <hu...@trippaers.nl>.
The Nicira NVP plugin is also depending on gson. If we make any changes we need to validate against their api to ensure that the interface still works.
If we put the changes in a separate branch i'd be happy to run the changes against the api.
Cheers,
Hugo
On Sep 20, 2013, at 12:08 PM, Darren Shepherd <da...@gmail.com> wrote:
> After much searching I found the two lines of code I needed to get it to serialize right. I'll keep looking and see if I find other snags. Maybe we can just move to jackson.... we'll see, haven't tried to deserialize yet.
>
> Darren
>
>> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
>>
>> Darren,
>>
>> When I looked into updating to the latest gson, the problem, IIRC, is things that weren't considered cyclical dependencies are suddenly considered cyclical with the latest. I don't recall where though.
>>
>> --Alex
>>
>>> -----Original Message-----
>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>> Sent: Thursday, September 19, 2013 1:33 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>
>>> Alright, I looked into this and it will take a bit more work to switch to Jackson.
>>> The snag I hit was how we serialize commands for the agents. We use a
>>> customer typeadatper in gson to produce a format like { "org.MyClass"
>>> : {...} }. In jackson I don't see producing a similar format as being so straight
>>> forward. I'm sure there's an elegant solution, but I didn't immediately find it.
>>> The problem is if you use a custom serializer in jackson and do
>>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop
>>> because it will call your serializer again. So if somebody can figure out how to
>>> do this format easily in jackson, the rest looks simple enough.
>>>
>>> So, being that it is not an obvious switch to using jackson, what is the impact
>>> of moving to the latest gson? What were the issue encountered when
>>> people tried it before?
>>>
>>> Darren
>>>
>>>
>>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>>> darren.s.shepherd@gmail.com> wrote:
>>>
>>>> I can do some analysis on this. I'm always up for a terribly painful
>>>> refactor :)
>>>>
>>>> Darren
>>>>
>>>>
>>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
>>>> wrote:
>>>>
>>>>> I actually did a quick try to update cloudstack to use newest gson
>>>>> version about 3 months back. Had to roll it back but I didn't try
>>>>> very hard though. Part of the reason why I decided to rollback is
>>>>> due to gson is used differently by various components in CloudStack
>>>>> and I didn't have time to get into each component to see if I break
>>>>> anything. The other is also I thought using Jackson would be much
>>>>> better and thought our next attempt should be with Jackson.
>>>>>
>>>>> Not that any of these helps you. Just know updating CloudStack to
>>>>> latest gson is not simple.
>>>>>
>>>>> --Alex
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>>> To: dev@whirr.apache.org
>>>>>> Cc: dev@cloudstack.apache.org
>>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>>
>>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
>>>>>> add
>>>>> the
>>>>>> feature url. Bonus: you can also use jclouds shell interface (part
>>>>>> of
>>>>> jclouds cli).
>>>>>>
>>>>>> Another option is to upgrade the CloudStack API to use the new version.
>>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>>
>>>>>>> Dear all,
>>>>>>>
>>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>>> conflicting
>>>>>> issue.
>>>>>>>
>>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>>> gson
>>>>> 1.7.1.
>>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>>
>>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>>> java.lang.**NoClassDefFoundError:
>>>>>>> com/google/gson/TypeAdapter
>>>>>>>
>>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
>>>>>>> not build successfully due to the following test error.
>>>>>>>
>>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>>> ------------------------------**------------------------------**
>>>>>>> ------------
>>>>>>> [INFO]
>>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>>> cloud-core
>>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>>> [INFO]
>>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
>>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>>>>>>> encoding to copy filtered resources.
>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>>> 3
>>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>>> 3
>>>>>>> resources [INFO] [INFO] ---
>>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
>>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
>>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>>> cloud-core
>>>>>>> ---
>>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>>> target/surefire-reports
>>>>>>>
>>>>>>> ------------------------------**-------------------------
>>>>>>> T E S T S
>>>>>>> ------------------------------**-------------------------
>>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>>> log4j:WARN No appenders could be found for logger
>>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>> log4j:WARN See
>>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>>> http://logging.apa
>>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>>> 3.054 sec <<< FAILURE!
>>>>>>>
>>>>>>> Results :
>>>>>>>
>>>>>>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>>
>>>>>>> Tests in error:
>>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>>
>>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>>
>>>>>>>
>>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>>> CloudStack building process went through. But when I issued an
>>>>>>> API --launchCluster to CloudStack, the following error related to
>>>>>>> gson popped up on CloudStack Management Server:
>>>>>>>
>>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>>>>>>> problem with
>>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>>>>>>> :"**
>>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>>> **LibvirtComputingResource","*
>>>>>>> *
>>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>>> 10.244.18
>>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
>>>>>>> *
>>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
>>>>>>> ","**
>>>>>>> contextMap":{},"wait":0}}]
>>>>>>> java.lang.ClassCastException:
>>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>>> cannot be cast to java.lang.Class
>>>>>>> at
>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>> ArrayTypeAdaptor.java:84)
>>>>>>> at
>>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>>> ArrayTypeAdaptor.java:37)
>>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>>> TreeTypeAdapter.java:58)
>>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>>> at
>>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>>> java:235)
>>>>>>> at
>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>>> at
>>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>>> )
>>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>>> ThreadPoolExecutor.java:1110)
>>>>>>> at
>>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>>> ThreadPoolExecutor.java:603)
>>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>>> following exception but pushing on
>>>>>>>
>>>>>>> On the CS Managment console I can see the cluster was started
>>>>>>> then quickly died.
>>>>>>> Running on provider cloudstack using identity
>>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>>> ZRGXVO1VApG
>>>>>>> 6q0a
>>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>>>>>>> Configuring template for
>>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>>> (729061754@qtp-385354117-
>>>>>> 0:)
>>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>>> jobtracker
>>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>>
>>>>>>> I attached the debuging log to this email.
>>>>>>>
>>>>>>> The launchCluster API is simple, it calls the
>>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>>
>>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>>> response.setObjectName("**launchCluster");
>>>>>>> LaunchClusterCommand command = null;
>>>>>>> try {
>>>>>>> command = new LaunchClusterCommand();
>>>>>>> } catch (Exception ex) {
>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>> VER
>>>>>>> E,
>>>>>>> null, ex);
>>>>>>> }
>>>>>>> String[] args = new String[2];
>>>>>>> args[0] = "--config";
>>>>>>> args[1] = config;
>>>>>>>
>>>>>>> try {
>>>>>>> command.run(System.in, System.out, System.err,
>>>>>>> Arrays.asList(args));
>>>>>>> } catch (Exception ex) {
>>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>>> VER
>>>>>>> E,
>>>>>>> null, ex);
>>>>>>> }
>>>>>>> response.setResponseName(**getCommandName());
>>>>>>> output = "successfully launched the cluster.";
>>>>>>> response.setOutPut(output);
>>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>>> this.setResponseObject(**response);
>>>>>>>
>>>>>>> Could someone help me out here? What would be a proper gson
>>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>>
>>>>>>> Thanks loads.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Meng
>>>>
>>>>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
After much searching I found the two lines of code I needed to get it to serialize right. I'll keep looking and see if I find other snags. Maybe we can just move to jackson.... we'll see, haven't tried to deserialize yet.
Darren
> On Sep 19, 2013, at 4:25 PM, Alex Huang <Al...@citrix.com> wrote:
>
> Darren,
>
> When I looked into updating to the latest gson, the problem, IIRC, is things that weren't considered cyclical dependencies are suddenly considered cyclical with the latest. I don't recall where though.
>
> --Alex
>
>> -----Original Message-----
>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>> Sent: Thursday, September 19, 2013 1:33 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>
>> Alright, I looked into this and it will take a bit more work to switch to Jackson.
>> The snag I hit was how we serialize commands for the agents. We use a
>> customer typeadatper in gson to produce a format like { "org.MyClass"
>> : {...} }. In jackson I don't see producing a similar format as being so straight
>> forward. I'm sure there's an elegant solution, but I didn't immediately find it.
>> The problem is if you use a custom serializer in jackson and do
>> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop
>> because it will call your serializer again. So if somebody can figure out how to
>> do this format easily in jackson, the rest looks simple enough.
>>
>> So, being that it is not an obvious switch to using jackson, what is the impact
>> of moving to the latest gson? What were the issue encountered when
>> people tried it before?
>>
>> Darren
>>
>>
>> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
>> darren.s.shepherd@gmail.com> wrote:
>>
>>> I can do some analysis on this. I'm always up for a terribly painful
>>> refactor :)
>>>
>>> Darren
>>>
>>>
>>>> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
>>> wrote:
>>>
>>>> I actually did a quick try to update cloudstack to use newest gson
>>>> version about 3 months back. Had to roll it back but I didn't try
>>>> very hard though. Part of the reason why I decided to rollback is
>>>> due to gson is used differently by various components in CloudStack
>>>> and I didn't have time to get into each component to see if I break
>>>> anything. The other is also I thought using Jackson would be much
>>>> better and thought our next attempt should be with Jackson.
>>>>
>>>> Not that any of these helps you. Just know updating CloudStack to
>>>> latest gson is not simple.
>>>>
>>>> --Alex
>>>>
>>>>> -----Original Message-----
>>>>> From: Andrei Savu [mailto:savu.andrei@gmail.com]
>>>>> Sent: Wednesday, September 18, 2013 10:53 AM
>>>>> To: dev@whirr.apache.org
>>>>> Cc: dev@cloudstack.apache.org
>>>>> Subject: Re: conflicting dependencies between CloudStack and Whirr
>>>>>
>>>>> It's easy to usr jclouds and whirr inside an OSGi container - just
>>>>> add
>>>> the
>>>>> feature url. Bonus: you can also use jclouds shell interface (part
>>>>> of
>>>> jclouds cli).
>>>>>
>>>>> Another option is to upgrade the CloudStack API to use the new version.
>>>>>> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> I am adding an API to CloudStack which utilizes Whirr to launch
>>>>>> various clusters on CloudStack. Now I am facing a dependency
>>>> conflicting
>>>>> issue.
>>>>>>
>>>>>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
>>>>>> gson
>>>> 1.7.1.
>>>>>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>>>>>
>>>>>> com.google.common.util.**concurrent.ExecutionError:
>>>>> java.lang.**NoClassDefFoundError:
>>>>>> com/google/gson/TypeAdapter
>>>>>>
>>>>>> This TypeAdapter class can be found inside gson-2.2.2.jar.
>>>>>> However if I modify CloudStack to use gson 2.2.2 CloudStack will
>>>>>> not build successfully due to the following test error.
>>>>>>
>>>>>> [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>>>>>> ------------------------------**------------------------------**
>>>>>> ------------
>>>>>> [INFO]
>>>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
>>>>>> cloud-core
>>>>>> --- [INFO] Deleting /home/meng/cloudstack/core/**target
>>>>>> [INFO]
>>>>>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
>>>>>> @ cloud-core --- [INFO] [INFO] ---
>>>>>> maven-resources-plugin:2.5:**resources (default-resources) @
>>>>>> cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>>>>>> encoding to copy filtered resources.
>>>>>> [INFO] skip non existing resourceDirectory
>>>>>> /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
>> 3
>>>>>> resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>>>>>> (default-compile) @ cloud-core --- [INFO] Compiling 156 source
>>>>>> files to /home/meng/cloudstack/core/** target/classes [INFO]
>>>>>> [INFO] --- maven-resources-plugin:2.5:**testResources
>>>>>> (default-testResources) @ cloud-core --- [debug] execute
>>>>>> contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>>> [INFO] skip non existing resourceDirectory
>>>>>> /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
>> 3
>>>>>> resources [INFO] [INFO] ---
>>>>>> maven-compiler-plugin:2.5.1:**testCompile
>>>>>> (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
>>>>>> file to /home/meng/cloudstack/core/** target/test-classes [INFO]
>>>>>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
>>>>>> cloud-core
>>>>>> ---
>>>>>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>>>>>> target/surefire-reports
>>>>>>
>>>>>> ------------------------------**-------------------------
>>>>>> T E S T S
>>>>>> ------------------------------**-------------------------
>>>>>> Running com.cloud.agent.transport.**RequestTest
>>>>>> log4j:WARN No appenders could be found for logger
>>>>>> (com.cloud.agent.transport.**RequestTest).
>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>> log4j:WARN See
>>>>> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>>>> http://logging.apa
>>>>> che.org/log4j/1.2/faq.html#noconfig>for more info.
>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
>>>>>> 3.054 sec <<< FAILURE!
>>>>>>
>>>>>> Results :
>>>>>>
>>>>>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>>>>>>
>>>>>> Tests in error:
>>>>>> testLogging(com.cloud.agent.**transport.RequestTest)
>>>>>>
>>>>>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>>>>>
>>>>>>
>>>>>> I ran "mvn -P developer clean install -DskipTests" , the
>>>>>> CloudStack building process went through. But when I issued an
>>>>>> API --launchCluster to CloudStack, the following error related to
>>>>>> gson popped up on CloudStack Management Server:
>>>>>>
>>>>>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>>>>>> problem with
>>>>>> [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>>>>>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>>>>>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>>>>>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>>>>>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>>>>>> :"**
>>>>>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>>>>>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>>>>>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>>>>>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>>>>> **LibvirtComputingResource","*
>>>>>> *
>>>>>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>>>>>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>>>>>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>>>>>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>>>>>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>>>>>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>>>>>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>>>>>> LibvirtComputingResource","**gatewayIpAddress":"!
>>>>>> 10.244.18
>>>>>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>>>>>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>>>>>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>>>>>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>>>>>> images/","poolType":"**Filesystem","capacityBytes":**
>>>>>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>>>>>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>>>>>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
>>>>>> *
>>>>>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>>>>>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
>>>>>> ","**
>>>>>> contextMap":{},"wait":0}}]
>>>>>> java.lang.ClassCastException:
>>>>>> com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>>>>>> cannot be cast to java.lang.Class
>>>>>> at
>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>> ArrayTypeAdaptor.java:84)
>>>>>> at
>>>>>> com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>>>>>> ArrayTypeAdaptor.java:37)
>>>>>> at com.google.gson.**TreeTypeAdapter.read(**
>>>>>> TreeTypeAdapter.java:58)
>>>>>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>>>>>> at
>>>>>> com.cloud.agent.transport.**Request.getCommands(Request.**
>>>>>> java:235)
>>>>>> at
>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>> processRequest(**AgentManagerImpl.java:1221)
>>>>>> at
>>>>>> com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>>>>>> doTask(AgentManagerImpl.java:**1374)
>>>>>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
>>>>> )
>>>>>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>>>>>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>>>>>> ThreadPoolExecutor.java:1110)
>>>>>> at
>>>>>> java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>>>>>> ThreadPoolExecutor.java:603)
>>>>>> at java.lang.Thread.run(Thread.**java:722)
>>>>>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
>>>>>> following exception but pushing on
>>>>>>
>>>>>> On the CS Managment console I can see the cluster was started
>>>>>> then quickly died.
>>>>>> Running on provider cloudstack using identity
>> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>>>>> ZRGXVO1VApG
>>>>>> 6q0a
>>>>>> **K-A-tQRQsZFwnOXQ
>>>>>> INFO [whirr.actions.**BootstrapClusterAction]
>>>>>> (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>>>>>> [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>>>>>> Configuring template for
>>>>>> bootstrap-hadoop-datanode_**hadoop-tasktracker
>>>>>> INFO [whirr.compute.**BootstrapTemplate]
>>>>>> (729061754@qtp-385354117-
>>>>> 0:)
>>>>>> Configuring template for bootstrap-hadoop-namenode_**hadoop-
>>>>> jobtracker
>>>>>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>>>>>> node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>>>>>> [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>>>>>> with roles [hadoop-namenode, hadoop-jobtracker] ...
>>>>>> INFO [whirr.actions.**DestroyClusterAction]
>>>>>> (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>>>>>>
>>>>>> I attached the debuging log to this email.
>>>>>>
>>>>>> The launchCluster API is simple, it calls the
>>>>>> LaunchClusterCommand in Whirr to launch a cluster.
>>>>>>
>>>>>> LaunchClusterResponse response = new LaunchClusterResponse();
>>>>>> response.setObjectName("**launchCluster");
>>>>>> LaunchClusterCommand command = null;
>>>>>> try {
>>>>>> command = new LaunchClusterCommand();
>>>>>> } catch (Exception ex) {
>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>> VER
>>>>>> E,
>>>>>> null, ex);
>>>>>> }
>>>>>> String[] args = new String[2];
>>>>>> args[0] = "--config";
>>>>>> args[1] = config;
>>>>>>
>>>>>> try {
>>>>>> command.run(System.in, System.out, System.err,
>>>>>> Arrays.asList(args));
>>>>>> } catch (Exception ex) {
>> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
>>>>> VER
>>>>>> E,
>>>>>> null, ex);
>>>>>> }
>>>>>> response.setResponseName(**getCommandName());
>>>>>> output = "successfully launched the cluster.";
>>>>>> response.setOutPut(output);
>>>>>> response.setAsync(Boolean.**FALSE);
>>>>>> this.setResponseObject(**response);
>>>>>>
>>>>>> Could someone help me out here? What would be a proper gson
>>>>>> version for CloudStack and Whirr to run at the same time?
>>>>>>
>>>>>> Thanks loads.
>>>>>>
>>>>>> Best Regards,
>>>>>> Meng
>>>
>>>
RE: conflicting dependencies between CloudStack and Whirr
Posted by Alex Huang <Al...@citrix.com>.
Darren,
When I looked into updating to the latest gson, the problem, IIRC, is things that weren't considered cyclical dependencies are suddenly considered cyclical with the latest. I don't recall where though.
--Alex
> -----Original Message-----
> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> Sent: Thursday, September 19, 2013 1:33 PM
> To: dev@cloudstack.apache.org
> Subject: Re: conflicting dependencies between CloudStack and Whirr
>
> Alright, I looked into this and it will take a bit more work to switch to Jackson.
> The snag I hit was how we serialize commands for the agents. We use a
> customer typeadatper in gson to produce a format like { "org.MyClass"
> : {...} }. In jackson I don't see producing a similar format as being so straight
> forward. I'm sure there's an elegant solution, but I didn't immediately find it.
> The problem is if you use a custom serializer in jackson and do
> writeObjectField(x.getClass().getName(), x), you'll get stuck in a loop
> because it will call your serializer again. So if somebody can figure out how to
> do this format easily in jackson, the rest looks simple enough.
>
> So, being that it is not an obvious switch to using jackson, what is the impact
> of moving to the latest gson? What were the issue encountered when
> people tried it before?
>
> Darren
>
>
> On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
> darren.s.shepherd@gmail.com> wrote:
>
> > I can do some analysis on this. I'm always up for a terribly painful
> > refactor :)
> >
> > Darren
> >
> >
> > On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com>
> wrote:
> >
> >> I actually did a quick try to update cloudstack to use newest gson
> >> version about 3 months back. Had to roll it back but I didn't try
> >> very hard though. Part of the reason why I decided to rollback is
> >> due to gson is used differently by various components in CloudStack
> >> and I didn't have time to get into each component to see if I break
> >> anything. The other is also I thought using Jackson would be much
> >> better and thought our next attempt should be with Jackson.
> >>
> >> Not that any of these helps you. Just know updating CloudStack to
> >> latest gson is not simple.
> >>
> >> --Alex
> >>
> >> > -----Original Message-----
> >> > From: Andrei Savu [mailto:savu.andrei@gmail.com]
> >> > Sent: Wednesday, September 18, 2013 10:53 AM
> >> > To: dev@whirr.apache.org
> >> > Cc: dev@cloudstack.apache.org
> >> > Subject: Re: conflicting dependencies between CloudStack and Whirr
> >> >
> >> > It's easy to usr jclouds and whirr inside an OSGi container - just
> >> > add
> >> the
> >> > feature url. Bonus: you can also use jclouds shell interface (part
> >> > of
> >> jclouds cli).
> >> >
> >> > Another option is to upgrade the CloudStack API to use the new version.
> >> > On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
> >> >
> >> > > Dear all,
> >> > >
> >> > > I am adding an API to CloudStack which utilizes Whirr to launch
> >> > > various clusters on CloudStack. Now I am facing a dependency
> >> conflicting
> >> > issue.
> >> > >
> >> > > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires
> >> > > gson
> >> 1.7.1.
> >> > > If I use gson 1.7.1 for Whirr, the following error will happen:
> >> > >
> >> > > com.google.common.util.**concurrent.ExecutionError:
> >> > java.lang.**NoClassDefFoundError:
> >> > > com/google/gson/TypeAdapter
> >> > >
> >> > > This TypeAdapter class can be found inside gson-2.2.2.jar.
> >> > > However if I modify CloudStack to use gson 2.2.2 CloudStack will
> >> > > not build successfully due to the following test error.
> >> > >
> >> > > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> >> > > ------------------------------**------------------------------**
> >> > > ------------
> >> > > [INFO]
> >> > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
> >> > > cloud-core
> >> > > --- [INFO] Deleting /home/meng/cloudstack/core/**target
> >> > > [INFO]
> >> > > [INFO] --- maven-remote-resources-plugin:**1.3:process (default)
> >> > > @ cloud-core --- [INFO] [INFO] ---
> >> > > maven-resources-plugin:2.5:**resources (default-resources) @
> >> > > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> >> > > encoding to copy filtered resources.
> >> > > [INFO] skip non existing resourceDirectory
> >> > > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying
> 3
> >> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> >> > > (default-compile) @ cloud-core --- [INFO] Compiling 156 source
> >> > > files to /home/meng/cloudstack/core/** target/classes [INFO]
> >> > > [INFO] --- maven-resources-plugin:2.5:**testResources
> >> > > (default-testResources) @ cloud-core --- [debug] execute
> >> > > contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources.
> >> > > [INFO] skip non existing resourceDirectory
> >> > > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying
> 3
> >> > > resources [INFO] [INFO] ---
> >> > > maven-compiler-plugin:2.5.1:**testCompile
> >> > > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source
> >> > > file to /home/meng/cloudstack/core/** target/test-classes [INFO]
> >> > > [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @
> >> > > cloud-core
> >> > > ---
> >> > > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> >> > > target/surefire-reports
> >> > >
> >> > > ------------------------------**-------------------------
> >> > > T E S T S
> >> > > ------------------------------**-------------------------
> >> > > Running com.cloud.agent.transport.**RequestTest
> >> > > log4j:WARN No appenders could be found for logger
> >> > > (com.cloud.agent.transport.**RequestTest).
> >> > > log4j:WARN Please initialize the log4j system properly.
> >> > > log4j:WARN See
> >> > http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
> >> http://logging.apa
> >> > che.org/log4j/1.2/faq.html#noconfig>for more info.
> >> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
> >> > > 3.054 sec <<< FAILURE!
> >> > >
> >> > > Results :
> >> > >
> >> > > Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
> >> > >
> >> > > Tests in error:
> >> > > testLogging(com.cloud.agent.**transport.RequestTest)
> >> > >
> >> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> >> > >
> >> > >
> >> > > I ran "mvn -P developer clean install -DskipTests" , the
> >> > > CloudStack building process went through. But when I issued an
> >> > > API --launchCluster to CloudStack, the following error related to
> >> > > gson popped up on CloudStack Management Server:
> >> > >
> >> > > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> >> > > problem with
> >> > > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> >> > > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> >> > > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> >> > > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> >> > > snapshot","pool":"/root","**hypervisorType":"KVM","**
> >> > >
> >> >
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
> >> > > :"**
> >> > > HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> >> > > 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> >> > > Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> >> > > "a7320748-6346-3c9a-975e-**90ac4ae4a986-
> >> > **LibvirtComputingResource","*
> >> > > *
> >> > > name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> >> > > publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> >> > > 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> >> > > privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> >> > > 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> >> > > storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> >> > > 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> >> > > LibvirtComputingResource","**gatewayIpAddress":"!
> >> > > 10.244.18
> >> > > .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> >> > > totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> >> > > 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> >> > > lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> >> > > images/","poolType":"**Filesystem","capacityBytes":**
> >> > > 52844687360,"availableBytes":**41535332352},"resourceType":"**
> >> > > STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> >> > > :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-*
> >> > > *
> >> > > LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> >> > > :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource
> >> > > ","**
> >> > > contextMap":{},"wait":0}}]
> >> > > java.lang.ClassCastException:
> >> > > com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> >> > > cannot be cast to java.lang.Class
> >> > > at
> >> > > com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> >> > > ArrayTypeAdaptor.java:84)
> >> > > at
> >> > > com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> >> > > ArrayTypeAdaptor.java:37)
> >> > > at com.google.gson.**TreeTypeAdapter.read(**
> >> > > TreeTypeAdapter.java:58)
> >> > > at com.google.gson.Gson.fromJson(**Gson.java:795)
> >> > > at
> >> > > com.cloud.agent.transport.**Request.getCommands(Request.**
> >> > > java:235)
> >> > > at
> >> > > com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> >> > > processRequest(**AgentManagerImpl.java:1221)
> >> > > at
> >> > > com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> >> > > doTask(AgentManagerImpl.java:**1374)
> >> > > at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> >> > >
> >> >
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659
> >> > )
> >> > > at com.cloud.utils.nio.Task.run(**Task.java:83)
> >> > > at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> >> > > ThreadPoolExecutor.java:1110)
> >> > > at
> >> > > java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> >> > > ThreadPoolExecutor.java:603)
> >> > > at java.lang.Thread.run(Thread.**java:722)
> >> > > WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the
> >> > > following exception but pushing on
> >> > >
> >> > > On the CS Managment console I can see the cluster was started
> >> > > then quickly died.
> >> > > Running on provider cloudstack using identity
> >> > >
> >> >
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
> >> > ZRGXVO1VApG
> >> > > 6q0a
> >> > > **K-A-tQRQsZFwnOXQ
> >> > > INFO [whirr.actions.**BootstrapClusterAction]
> >> > > (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
> >> > > [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> >> > > Configuring template for
> >> > > bootstrap-hadoop-datanode_**hadoop-tasktracker
> >> > > INFO [whirr.compute.**BootstrapTemplate]
> >> > > (729061754@qtp-385354117-
> >> > 0:)
> >> > > Configuring template for bootstrap-hadoop-namenode_**hadoop-
> >> > jobtracker
> >> > > INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
> >> > > node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
> >> > > [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> >> > > with roles [hadoop-namenode, hadoop-jobtracker] ...
> >> > > INFO [whirr.actions.**DestroyClusterAction]
> >> > > (729061754@qtp-385354117-0:) Cluster hadoop destroyed
> >> > >
> >> > > I attached the debuging log to this email.
> >> > >
> >> > > The launchCluster API is simple, it calls the
> >> > > LaunchClusterCommand in Whirr to launch a cluster.
> >> > >
> >> > > LaunchClusterResponse response = new LaunchClusterResponse();
> >> > > response.setObjectName("**launchCluster");
> >> > > LaunchClusterCommand command = null;
> >> > > try {
> >> > > command = new LaunchClusterCommand();
> >> > > } catch (Exception ex) {
> >> > >
> >> > >
> >> >
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
> >> > VER
> >> > > E,
> >> > > null, ex);
> >> > > }
> >> > > String[] args = new String[2];
> >> > > args[0] = "--config";
> >> > > args[1] = config;
> >> > >
> >> > > try {
> >> > > command.run(System.in, System.out, System.err,
> >> > > Arrays.asList(args));
> >> > > } catch (Exception ex) {
> >> > >
> >> > >
> >> >
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SE
> >> > VER
> >> > > E,
> >> > > null, ex);
> >> > > }
> >> > > response.setResponseName(**getCommandName());
> >> > > output = "successfully launched the cluster.";
> >> > > response.setOutPut(output);
> >> > > response.setAsync(Boolean.**FALSE);
> >> > > this.setResponseObject(**response);
> >> > >
> >> > > Could someone help me out here? What would be a proper gson
> >> > > version for CloudStack and Whirr to run at the same time?
> >> > >
> >> > > Thanks loads.
> >> > >
> >> > > Best Regards,
> >> > > Meng
> >> > >
> >> > >
> >> > >
> >>
> >
> >
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
Alright, I looked into this and it will take a bit more work to switch to
Jackson. The snag I hit was how we serialize commands for the agents. We
use a customer typeadatper in gson to produce a format like { "org.MyClass"
: {...} }. In jackson I don't see producing a similar format as being so
straight forward. I'm sure there's an elegant solution, but I didn't
immediately find it. The problem is if you use a custom serializer in
jackson and do writeObjectField(x.getClass().getName(), x), you'll get
stuck in a loop because it will call your serializer again. So if somebody
can figure out how to do this format easily in jackson, the rest looks
simple enough.
So, being that it is not an obvious switch to using jackson, what is the
impact of moving to the latest gson? What were the issue encountered when
people tried it before?
Darren
On Wed, Sep 18, 2013 at 4:42 PM, Darren Shepherd <
darren.s.shepherd@gmail.com> wrote:
> I can do some analysis on this. I'm always up for a terribly painful
> refactor :)
>
> Darren
>
>
> On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com> wrote:
>
>> I actually did a quick try to update cloudstack to use newest gson
>> version about 3 months back. Had to roll it back but I didn't try very
>> hard though. Part of the reason why I decided to rollback is due to gson
>> is used differently by various components in CloudStack and I didn't have
>> time to get into each component to see if I break anything. The other is
>> also I thought using Jackson would be much better and thought our next
>> attempt should be with Jackson.
>>
>> Not that any of these helps you. Just know updating CloudStack to latest
>> gson is not simple.
>>
>> --Alex
>>
>> > -----Original Message-----
>> > From: Andrei Savu [mailto:savu.andrei@gmail.com]
>> > Sent: Wednesday, September 18, 2013 10:53 AM
>> > To: dev@whirr.apache.org
>> > Cc: dev@cloudstack.apache.org
>> > Subject: Re: conflicting dependencies between CloudStack and Whirr
>> >
>> > It's easy to usr jclouds and whirr inside an OSGi container - just add
>> the
>> > feature url. Bonus: you can also use jclouds shell interface (part of
>> jclouds cli).
>> >
>> > Another option is to upgrade the CloudStack API to use the new version.
>> > On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>> >
>> > > Dear all,
>> > >
>> > > I am adding an API to CloudStack which utilizes Whirr to launch
>> > > various clusters on CloudStack. Now I am facing a dependency
>> conflicting
>> > issue.
>> > >
>> > > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
>> 1.7.1.
>> > > If I use gson 1.7.1 for Whirr, the following error will happen:
>> > >
>> > > com.google.common.util.**concurrent.ExecutionError:
>> > java.lang.**NoClassDefFoundError:
>> > > com/google/gson/TypeAdapter
>> > >
>> > > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
>> > > I modify CloudStack to use gson 2.2.2 CloudStack will not build
>> > > successfully due to the following test error.
>> > >
>> > > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
>> > > ------------------------------**------------------------------**
>> > > ------------
>> > > [INFO]
>> > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
>> > > --- [INFO] Deleting /home/meng/cloudstack/core/**target
>> > > [INFO]
>> > > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
>> > > cloud-core --- [INFO] [INFO] ---
>> > > maven-resources-plugin:2.5:**resources (default-resources) @
>> > > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
>> > > encoding to copy filtered resources.
>> > > [INFO] skip non existing resourceDirectory
>> > > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
>> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
>> > > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
>> > > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
>> > > maven-resources-plugin:2.5:**testResources
>> > > (default-testResources) @ cloud-core --- [debug] execute contextualize
>> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > > [INFO] skip non existing resourceDirectory
>> > > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
>> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
>> > > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
>> > > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
>> > > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
>> > > ---
>> > > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>> > > target/surefire-reports
>> > >
>> > > ------------------------------**-------------------------
>> > > T E S T S
>> > > ------------------------------**-------------------------
>> > > Running com.cloud.agent.transport.**RequestTest
>> > > log4j:WARN No appenders could be found for logger
>> > > (com.cloud.agent.transport.**RequestTest).
>> > > log4j:WARN Please initialize the log4j system properly.
>> > > log4j:WARN See
>> > http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
>> http://logging.apa
>> > che.org/log4j/1.2/faq.html#noconfig>for more info.
>> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
>> > > sec <<< FAILURE!
>> > >
>> > > Results :
>> > >
>> > > Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>> > >
>> > > Tests in error:
>> > > testLogging(com.cloud.agent.**transport.RequestTest)
>> > >
>> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>> > >
>> > >
>> > > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
>> > > building process went through. But when I issued an API
>> > > --launchCluster to CloudStack, the following error related to gson
>> > > popped up on CloudStack Management Server:
>> > >
>> > > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
>> > > problem with
>> > > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>> > > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>> > > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>> > > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>> > > snapshot","pool":"/root","**hypervisorType":"KVM","**
>> > >
>> > hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
>> > > :"**
>> > > HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>> > > 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>> > > Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>> > > "a7320748-6346-3c9a-975e-**90ac4ae4a986-
>> > **LibvirtComputingResource","*
>> > > *
>> > > name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>> > > publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>> > > 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>> > > privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>> > > 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>> > > storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>> > > 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>> > > LibvirtComputingResource","**gatewayIpAddress":"!
>> > > 10.244.18
>> > > .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>> > > totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>> > > 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>> > > lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>> > > images/","poolType":"**Filesystem","capacityBytes":**
>> > > 52844687360,"availableBytes":**41535332352},"resourceType":"**
>> > > STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>> > > :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
>> > > LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>> > > :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
>> > > contextMap":{},"wait":0}}]
>> > > java.lang.ClassCastException:
>> > > com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>> > > cannot be cast to java.lang.Class
>> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> > > ArrayTypeAdaptor.java:84)
>> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> > > ArrayTypeAdaptor.java:37)
>> > > at com.google.gson.**TreeTypeAdapter.read(**
>> > > TreeTypeAdapter.java:58)
>> > > at com.google.gson.Gson.fromJson(**Gson.java:795)
>> > > at com.cloud.agent.transport.**Request.getCommands(Request.**
>> > > java:235)
>> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> > > processRequest(**AgentManagerImpl.java:1221)
>> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> > > doTask(AgentManagerImpl.java:**1374)
>> > > at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>> > >
>> > ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
>> > > at com.cloud.utils.nio.Task.run(**Task.java:83)
>> > > at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>> > > ThreadPoolExecutor.java:1110)
>> > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> > > ThreadPoolExecutor.java:603)
>> > > at java.lang.Thread.run(Thread.**java:722)
>> > > WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
>> > > exception but pushing on
>> > >
>> > > On the CS Managment console I can see the cluster was started then
>> > > quickly died.
>> > > Running on provider cloudstack using identity
>> > >
>> > h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
>> > ZRGXVO1VApG
>> > > 6q0a
>> > > **K-A-tQRQsZFwnOXQ
>> > > INFO [whirr.actions.**BootstrapClusterAction]
>> > > (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
>> > > [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>> > > Configuring template for
>> > > bootstrap-hadoop-datanode_**hadoop-tasktracker
>> > > INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-
>> > 0:)
>> > > Configuring template for bootstrap-hadoop-namenode_**hadoop-
>> > jobtracker
>> > > INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
>> > > node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
>> > > [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) with
>> > > roles [hadoop-namenode, hadoop-jobtracker] ...
>> > > INFO [whirr.actions.**DestroyClusterAction]
>> > > (729061754@qtp-385354117-0:) Cluster hadoop destroyed
>> > >
>> > > I attached the debuging log to this email.
>> > >
>> > > The launchCluster API is simple, it calls the LaunchClusterCommand in
>> > > Whirr to launch a cluster.
>> > >
>> > > LaunchClusterResponse response = new LaunchClusterResponse();
>> > > response.setObjectName("**launchCluster");
>> > > LaunchClusterCommand command = null;
>> > > try {
>> > > command = new LaunchClusterCommand();
>> > > } catch (Exception ex) {
>> > >
>> > >
>> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
>> > > E,
>> > > null, ex);
>> > > }
>> > > String[] args = new String[2];
>> > > args[0] = "--config";
>> > > args[1] = config;
>> > >
>> > > try {
>> > > command.run(System.in, System.out, System.err,
>> > > Arrays.asList(args));
>> > > } catch (Exception ex) {
>> > >
>> > >
>> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
>> > > E,
>> > > null, ex);
>> > > }
>> > > response.setResponseName(**getCommandName());
>> > > output = "successfully launched the cluster.";
>> > > response.setOutPut(output);
>> > > response.setAsync(Boolean.**FALSE);
>> > > this.setResponseObject(**response);
>> > >
>> > > Could someone help me out here? What would be a proper gson version
>> > > for CloudStack and Whirr to run at the same time?
>> > >
>> > > Thanks loads.
>> > >
>> > > Best Regards,
>> > > Meng
>> > >
>> > >
>> > >
>>
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
I can do some analysis on this. I'm always up for a terribly painful
refactor :)
Darren
On Wed, Sep 18, 2013 at 4:06 PM, Alex Huang <Al...@citrix.com> wrote:
> I actually did a quick try to update cloudstack to use newest gson version
> about 3 months back. Had to roll it back but I didn't try very hard
> though. Part of the reason why I decided to rollback is due to gson is
> used differently by various components in CloudStack and I didn't have time
> to get into each component to see if I break anything. The other is also I
> thought using Jackson would be much better and thought our next attempt
> should be with Jackson.
>
> Not that any of these helps you. Just know updating CloudStack to latest
> gson is not simple.
>
> --Alex
>
> > -----Original Message-----
> > From: Andrei Savu [mailto:savu.andrei@gmail.com]
> > Sent: Wednesday, September 18, 2013 10:53 AM
> > To: dev@whirr.apache.org
> > Cc: dev@cloudstack.apache.org
> > Subject: Re: conflicting dependencies between CloudStack and Whirr
> >
> > It's easy to usr jclouds and whirr inside an OSGi container - just add
> the
> > feature url. Bonus: you can also use jclouds shell interface (part of
> jclouds cli).
> >
> > Another option is to upgrade the CloudStack API to use the new version.
> > On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
> >
> > > Dear all,
> > >
> > > I am adding an API to CloudStack which utilizes Whirr to launch
> > > various clusters on CloudStack. Now I am facing a dependency
> conflicting
> > issue.
> > >
> > > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
> 1.7.1.
> > > If I use gson 1.7.1 for Whirr, the following error will happen:
> > >
> > > com.google.common.util.**concurrent.ExecutionError:
> > java.lang.**NoClassDefFoundError:
> > > com/google/gson/TypeAdapter
> > >
> > > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
> > > I modify CloudStack to use gson 2.2.2 CloudStack will not build
> > > successfully due to the following test error.
> > >
> > > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> > > ------------------------------**------------------------------**
> > > ------------
> > > [INFO]
> > > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
> > > --- [INFO] Deleting /home/meng/cloudstack/core/**target
> > > [INFO]
> > > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> > > cloud-core --- [INFO] [INFO] ---
> > > maven-resources-plugin:2.5:**resources (default-resources) @
> > > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> > > encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory
> > > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> > > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
> > > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
> > > maven-resources-plugin:2.5:**testResources
> > > (default-testResources) @ cloud-core --- [debug] execute contextualize
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory
> > > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
> > > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> > > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
> > > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
> > > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> > > ---
> > > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> > > target/surefire-reports
> > >
> > > ------------------------------**-------------------------
> > > T E S T S
> > > ------------------------------**-------------------------
> > > Running com.cloud.agent.transport.**RequestTest
> > > log4j:WARN No appenders could be found for logger
> > > (com.cloud.agent.transport.**RequestTest).
> > > log4j:WARN Please initialize the log4j system properly.
> > > log4j:WARN See
> > http://logging.apache.org/**log4j/1.2/faq.html#noconfig<
> http://logging.apa
> > che.org/log4j/1.2/faq.html#noconfig>for more info.
> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
> > > sec <<< FAILURE!
> > >
> > > Results :
> > >
> > > Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
> > >
> > > Tests in error:
> > > testLogging(com.cloud.agent.**transport.RequestTest)
> > >
> > > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> > >
> > >
> > > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> > > building process went through. But when I issued an API
> > > --launchCluster to CloudStack, the following error related to gson
> > > popped up on CloudStack Management Server:
> > >
> > > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> > > problem with
> > > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> > > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> > > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> > > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> > > snapshot","pool":"/root","**hypervisorType":"KVM","**
> > >
> > hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
> > > :"**
> > > HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> > > 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> > > Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> > > "a7320748-6346-3c9a-975e-**90ac4ae4a986-
> > **LibvirtComputingResource","*
> > > *
> > > name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> > > publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> > > 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> > > privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> > > 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> > > storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> > > 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> > > LibvirtComputingResource","**gatewayIpAddress":"!
> > > 10.244.18
> > > .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> > > totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> > > 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> > > lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> > > images/","poolType":"**Filesystem","capacityBytes":**
> > > 52844687360,"availableBytes":**41535332352},"resourceType":"**
> > > STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> > > :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> > > LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> > > :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> > > contextMap":{},"wait":0}}]
> > > java.lang.ClassCastException:
> > > com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> > > cannot be cast to java.lang.Class
> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> > > ArrayTypeAdaptor.java:84)
> > > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> > > ArrayTypeAdaptor.java:37)
> > > at com.google.gson.**TreeTypeAdapter.read(**
> > > TreeTypeAdapter.java:58)
> > > at com.google.gson.Gson.fromJson(**Gson.java:795)
> > > at com.cloud.agent.transport.**Request.getCommands(Request.**
> > > java:235)
> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> > > processRequest(**AgentManagerImpl.java:1221)
> > > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> > > doTask(AgentManagerImpl.java:**1374)
> > > at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> > >
> > ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> > > at com.cloud.utils.nio.Task.run(**Task.java:83)
> > > at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> > > ThreadPoolExecutor.java:1110)
> > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> > > ThreadPoolExecutor.java:603)
> > > at java.lang.Thread.run(Thread.**java:722)
> > > WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> > > exception but pushing on
> > >
> > > On the CS Managment console I can see the cluster was started then
> > > quickly died.
> > > Running on provider cloudstack using identity
> > >
> > h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
> > ZRGXVO1VApG
> > > 6q0a
> > > **K-A-tQRQsZFwnOXQ
> > > INFO [whirr.actions.**BootstrapClusterAction]
> > > (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
> > > [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> > > Configuring template for
> > > bootstrap-hadoop-datanode_**hadoop-tasktracker
> > > INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-
> > 0:)
> > > Configuring template for bootstrap-hadoop-namenode_**hadoop-
> > jobtracker
> > > INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
> > > node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
> > > [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) with
> > > roles [hadoop-namenode, hadoop-jobtracker] ...
> > > INFO [whirr.actions.**DestroyClusterAction]
> > > (729061754@qtp-385354117-0:) Cluster hadoop destroyed
> > >
> > > I attached the debuging log to this email.
> > >
> > > The launchCluster API is simple, it calls the LaunchClusterCommand in
> > > Whirr to launch a cluster.
> > >
> > > LaunchClusterResponse response = new LaunchClusterResponse();
> > > response.setObjectName("**launchCluster");
> > > LaunchClusterCommand command = null;
> > > try {
> > > command = new LaunchClusterCommand();
> > > } catch (Exception ex) {
> > >
> > >
> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
> > > E,
> > > null, ex);
> > > }
> > > String[] args = new String[2];
> > > args[0] = "--config";
> > > args[1] = config;
> > >
> > > try {
> > > command.run(System.in, System.out, System.err,
> > > Arrays.asList(args));
> > > } catch (Exception ex) {
> > >
> > >
> > Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
> > > E,
> > > null, ex);
> > > }
> > > response.setResponseName(**getCommandName());
> > > output = "successfully launched the cluster.";
> > > response.setOutPut(output);
> > > response.setAsync(Boolean.**FALSE);
> > > this.setResponseObject(**response);
> > >
> > > Could someone help me out here? What would be a proper gson version
> > > for CloudStack and Whirr to run at the same time?
> > >
> > > Thanks loads.
> > >
> > > Best Regards,
> > > Meng
> > >
> > >
> > >
>
RE: conflicting dependencies between CloudStack and Whirr
Posted by Alex Huang <Al...@citrix.com>.
I actually did a quick try to update cloudstack to use newest gson version about 3 months back. Had to roll it back but I didn't try very hard though. Part of the reason why I decided to rollback is due to gson is used differently by various components in CloudStack and I didn't have time to get into each component to see if I break anything. The other is also I thought using Jackson would be much better and thought our next attempt should be with Jackson.
Not that any of these helps you. Just know updating CloudStack to latest gson is not simple.
--Alex
> -----Original Message-----
> From: Andrei Savu [mailto:savu.andrei@gmail.com]
> Sent: Wednesday, September 18, 2013 10:53 AM
> To: dev@whirr.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: conflicting dependencies between CloudStack and Whirr
>
> It's easy to usr jclouds and whirr inside an OSGi container - just add the
> feature url. Bonus: you can also use jclouds shell interface (part of jclouds cli).
>
> Another option is to upgrade the CloudStack API to use the new version.
> On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
>
> > Dear all,
> >
> > I am adding an API to CloudStack which utilizes Whirr to launch
> > various clusters on CloudStack. Now I am facing a dependency conflicting
> issue.
> >
> > Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> > If I use gson 1.7.1 for Whirr, the following error will happen:
> >
> > com.google.common.util.**concurrent.ExecutionError:
> java.lang.**NoClassDefFoundError:
> > com/google/gson/TypeAdapter
> >
> > This TypeAdapter class can be found inside gson-2.2.2.jar. However if
> > I modify CloudStack to use gson 2.2.2 CloudStack will not build
> > successfully due to the following test error.
> >
> > [INFO] Building Apache CloudStack Core 4.1.1 [INFO]
> > ------------------------------**------------------------------**
> > ------------
> > [INFO]
> > [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core
> > --- [INFO] Deleting /home/meng/cloudstack/core/**target
> > [INFO]
> > [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> > cloud-core --- [INFO] [INFO] ---
> > maven-resources-plugin:2.5:**resources (default-resources) @
> > cloud-core --- [debug] execute contextualize [INFO] Using 'UTF-8'
> > encoding to copy filtered resources.
> > [INFO] skip non existing resourceDirectory
> > /home/meng/cloudstack/core/** src/main/resources [INFO] Copying 3
> > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**compile
> > (default-compile) @ cloud-core --- [INFO] Compiling 156 source files
> > to /home/meng/cloudstack/core/** target/classes [INFO] [INFO] ---
> > maven-resources-plugin:2.5:**testResources
> > (default-testResources) @ cloud-core --- [debug] execute contextualize
> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > [INFO] skip non existing resourceDirectory
> > /home/meng/cloudstack/core/** src/test/resources [INFO] Copying 3
> > resources [INFO] [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> > (default-testCompile) @ cloud-core --- [INFO] Compiling 1 source file
> > to /home/meng/cloudstack/core/** target/test-classes [INFO] [INFO] ---
> > maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> > ---
> > [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> > target/surefire-reports
> >
> > ------------------------------**-------------------------
> > T E S T S
> > ------------------------------**-------------------------
> > Running com.cloud.agent.transport.**RequestTest
> > log4j:WARN No appenders could be found for logger
> > (com.cloud.agent.transport.**RequestTest).
> > log4j:WARN Please initialize the log4j system properly.
> > log4j:WARN See
> http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apa
> che.org/log4j/1.2/faq.html#noconfig>for more info.
> > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
> > sec <<< FAILURE!
> >
> > Results :
> >
> > Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
> >
> > Tests in error:
> > testLogging(com.cloud.agent.**transport.RequestTest)
> >
> > Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
> >
> >
> > I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> > building process went through. But when I issued an API
> > --launchCluster to CloudStack, the following error related to gson
> > popped up on CloudStack Management Server:
> >
> > ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught
> > problem with
> > [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> > 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> > vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> > Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> > snapshot","pool":"/root","**hypervisorType":"KVM","**
> >
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy"
> > :"**
> > HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> > 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> > Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> > "a7320748-6346-3c9a-975e-**90ac4ae4a986-
> **LibvirtComputingResource","*
> > *
> > name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> > publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> > 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> > privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> > 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> > storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> > 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> > LibvirtComputingResource","**gatewayIpAddress":"!
> > 10.244.18
> > .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> > totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> > 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> > lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> > images/","poolType":"**Filesystem","capacityBytes":**
> > 52844687360,"availableBytes":**41535332352},"resourceType":"**
> > STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> > :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> > LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> > :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> > contextMap":{},"wait":0}}]
> > java.lang.ClassCastException:
> > com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> > cannot be cast to java.lang.Class
> > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> > ArrayTypeAdaptor.java:84)
> > at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> > ArrayTypeAdaptor.java:37)
> > at com.google.gson.**TreeTypeAdapter.read(**
> > TreeTypeAdapter.java:58)
> > at com.google.gson.Gson.fromJson(**Gson.java:795)
> > at com.cloud.agent.transport.**Request.getCommands(Request.**
> > java:235)
> > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> > processRequest(**AgentManagerImpl.java:1221)
> > at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> > doTask(AgentManagerImpl.java:**1374)
> > at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> >
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> > at com.cloud.utils.nio.Task.run(**Task.java:83)
> > at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> > ThreadPoolExecutor.java:1110)
> > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> > ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.**java:722)
> > WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> > exception but pushing on
> >
> > On the CS Managment console I can see the cluster was started then
> > quickly died.
> > Running on provider cloudstack using identity
> >
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7a
> ZRGXVO1VApG
> > 6q0a
> > **K-A-tQRQsZFwnOXQ
> > INFO [whirr.actions.**BootstrapClusterAction]
> > (729061754@qtp-385354117-0:) Bootstrapping cluster INFO
> > [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> > Configuring template for
> > bootstrap-hadoop-datanode_**hadoop-tasktracker
> > INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-
> 0:)
> > Configuring template for bootstrap-hadoop-namenode_**hadoop-
> jobtracker
> > INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1
> > node(s) with roles [hadoop-datanode, hadoop-tasktracker] INFO
> > [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s) with
> > roles [hadoop-namenode, hadoop-jobtracker] ...
> > INFO [whirr.actions.**DestroyClusterAction]
> > (729061754@qtp-385354117-0:) Cluster hadoop destroyed
> >
> > I attached the debuging log to this email.
> >
> > The launchCluster API is simple, it calls the LaunchClusterCommand in
> > Whirr to launch a cluster.
> >
> > LaunchClusterResponse response = new LaunchClusterResponse();
> > response.setObjectName("**launchCluster");
> > LaunchClusterCommand command = null;
> > try {
> > command = new LaunchClusterCommand();
> > } catch (Exception ex) {
> >
> >
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
> > E,
> > null, ex);
> > }
> > String[] args = new String[2];
> > args[0] = "--config";
> > args[1] = config;
> >
> > try {
> > command.run(System.in, System.out, System.err,
> > Arrays.asList(args));
> > } catch (Exception ex) {
> >
> >
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVER
> > E,
> > null, ex);
> > }
> > response.setResponseName(**getCommandName());
> > output = "successfully launched the cluster.";
> > response.setOutPut(output);
> > response.setAsync(Boolean.**FALSE);
> > this.setResponseObject(**response);
> >
> > Could someone help me out here? What would be a proper gson version
> > for CloudStack and Whirr to run at the same time?
> >
> > Thanks loads.
> >
> > Best Regards,
> > Meng
> >
> >
> >
Re: conflicting dependencies between CloudStack and Whirr
Posted by Andrei Savu <sa...@gmail.com>.
It's easy to usr jclouds and whirr inside an OSGi container - just add the
feature url. Bonus: you can also use jclouds shell interface (part of
jclouds cli).
Another option is to upgrade the CloudStack API to use the new version.
On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] ------------------------------**------------------------------**
> ------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> ------------------------------**-------------------------
> T E S T S
> ------------------------------**-------------------------
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apache.org/log4j/1.2/faq.html#noconfig>for more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
> testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> contextMap":{},"wait":0}}]
> java.lang.ClassCastException: com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> cannot be cast to java.lang.Class
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:84)
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:37)
> at com.google.gson.**TreeTypeAdapter.read(**
> TreeTypeAdapter.java:58)
> at com.google.gson.Gson.fromJson(**Gson.java:795)
> at com.cloud.agent.transport.**Request.getCommands(Request.**
> java:235)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> processRequest(**AgentManagerImpl.java:1221)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> doTask(AgentManagerImpl.java:**1374)
> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> at com.cloud.utils.nio.Task.run(**Task.java:83)
> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.**java:722)
> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> exception but pushing on
>
> On the CS Managment console I can see the cluster was started then quickly
> died.
> Running on provider cloudstack using identity
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0a
> **K-A-tQRQsZFwnOXQ
> INFO [whirr.actions.**BootstrapClusterAction] (729061754@qtp-385354117-0:)
> Bootstrapping cluster
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
> with roles [hadoop-datanode, hadoop-tasktracker]
> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> with roles [hadoop-namenode, hadoop-jobtracker]
> ...
> INFO [whirr.actions.**DestroyClusterAction] (729061754@qtp-385354117-0:)
> Cluster hadoop destroyed
>
> I attached the debuging log to this email.
>
> The launchCluster API is simple, it calls the LaunchClusterCommand in
> Whirr to launch a cluster.
>
> LaunchClusterResponse response = new LaunchClusterResponse();
> response.setObjectName("**launchCluster");
> LaunchClusterCommand command = null;
> try {
> command = new LaunchClusterCommand();
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> String[] args = new String[2];
> args[0] = "--config";
> args[1] = config;
>
> try {
> command.run(System.in, System.out, System.err,
> Arrays.asList(args));
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> response.setResponseName(**getCommandName());
> output = "successfully launched the cluster.";
> response.setOutPut(output);
> response.setAsync(Boolean.**FALSE);
> this.setResponseObject(**response);
>
> Could someone help me out here? What would be a proper gson version for
> CloudStack and Whirr to run at the same time?
>
> Thanks loads.
>
> Best Regards,
> Meng
>
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Andrew Bayer <an...@gmail.com>.
Good question. The gson dependency in Whirr is coming from jclouds, and
jclouds is very particular about its gson dependencies. Any possibility of
using isolated classloaders in some way?
A.
On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng <me...@ufl.edu> wrote:
> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] ------------------------------**------------------------------**
> ------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> ------------------------------**-------------------------
> T E S T S
> ------------------------------**-------------------------
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apache.org/log4j/1.2/faq.html#noconfig>for more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
> testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> contextMap":{},"wait":0}}]
> java.lang.ClassCastException: com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> cannot be cast to java.lang.Class
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:84)
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:37)
> at com.google.gson.**TreeTypeAdapter.read(**
> TreeTypeAdapter.java:58)
> at com.google.gson.Gson.fromJson(**Gson.java:795)
> at com.cloud.agent.transport.**Request.getCommands(Request.**
> java:235)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> processRequest(**AgentManagerImpl.java:1221)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> doTask(AgentManagerImpl.java:**1374)
> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> at com.cloud.utils.nio.Task.run(**Task.java:83)
> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.**java:722)
> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> exception but pushing on
>
> On the CS Managment console I can see the cluster was started then quickly
> died.
> Running on provider cloudstack using identity
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0a
> **K-A-tQRQsZFwnOXQ
> INFO [whirr.actions.**BootstrapClusterAction] (729061754@qtp-385354117-0:)
> Bootstrapping cluster
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
> with roles [hadoop-datanode, hadoop-tasktracker]
> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> with roles [hadoop-namenode, hadoop-jobtracker]
> ...
> INFO [whirr.actions.**DestroyClusterAction] (729061754@qtp-385354117-0:)
> Cluster hadoop destroyed
>
> I attached the debuging log to this email.
>
> The launchCluster API is simple, it calls the LaunchClusterCommand in
> Whirr to launch a cluster.
>
> LaunchClusterResponse response = new LaunchClusterResponse();
> response.setObjectName("**launchCluster");
> LaunchClusterCommand command = null;
> try {
> command = new LaunchClusterCommand();
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> String[] args = new String[2];
> args[0] = "--config";
> args[1] = config;
>
> try {
> command.run(System.in, System.out, System.err,
> Arrays.asList(args));
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> response.setResponseName(**getCommandName());
> output = "successfully launched the cluster.";
> response.setOutPut(output);
> response.setAsync(Boolean.**FALSE);
> this.setResponseObject(**response);
>
> Could someone help me out here? What would be a proper gson version for
> CloudStack and Whirr to run at the same time?
>
> Thanks loads.
>
> Best Regards,
> Meng
>
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Andrei Savu <sa...@gmail.com>.
It's easy to usr jclouds and whirr inside an OSGi container - just add the
feature url. Bonus: you can also use jclouds shell interface (part of
jclouds cli).
Another option is to upgrade the CloudStack API to use the new version.
On Sep 18, 2013 5:14 AM, "Han,Meng" <me...@ufl.edu> wrote:
> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] ------------------------------**------------------------------**
> ------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> ------------------------------**-------------------------
> T E S T S
> ------------------------------**-------------------------
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apache.org/log4j/1.2/faq.html#noconfig>for more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
> testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> contextMap":{},"wait":0}}]
> java.lang.ClassCastException: com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> cannot be cast to java.lang.Class
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:84)
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:37)
> at com.google.gson.**TreeTypeAdapter.read(**
> TreeTypeAdapter.java:58)
> at com.google.gson.Gson.fromJson(**Gson.java:795)
> at com.cloud.agent.transport.**Request.getCommands(Request.**
> java:235)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> processRequest(**AgentManagerImpl.java:1221)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> doTask(AgentManagerImpl.java:**1374)
> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> at com.cloud.utils.nio.Task.run(**Task.java:83)
> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.**java:722)
> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> exception but pushing on
>
> On the CS Managment console I can see the cluster was started then quickly
> died.
> Running on provider cloudstack using identity
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0a
> **K-A-tQRQsZFwnOXQ
> INFO [whirr.actions.**BootstrapClusterAction] (729061754@qtp-385354117-0:)
> Bootstrapping cluster
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
> with roles [hadoop-datanode, hadoop-tasktracker]
> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> with roles [hadoop-namenode, hadoop-jobtracker]
> ...
> INFO [whirr.actions.**DestroyClusterAction] (729061754@qtp-385354117-0:)
> Cluster hadoop destroyed
>
> I attached the debuging log to this email.
>
> The launchCluster API is simple, it calls the LaunchClusterCommand in
> Whirr to launch a cluster.
>
> LaunchClusterResponse response = new LaunchClusterResponse();
> response.setObjectName("**launchCluster");
> LaunchClusterCommand command = null;
> try {
> command = new LaunchClusterCommand();
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> String[] args = new String[2];
> args[0] = "--config";
> args[1] = config;
>
> try {
> command.run(System.in, System.out, System.err,
> Arrays.asList(args));
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> response.setResponseName(**getCommandName());
> output = "successfully launched the cluster.";
> response.setOutPut(output);
> response.setAsync(Boolean.**FALSE);
> this.setResponseObject(**response);
>
> Could someone help me out here? What would be a proper gson version for
> CloudStack and Whirr to run at the same time?
>
> Thanks loads.
>
> Best Regards,
> Meng
>
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Kelven Yang <ke...@citrix.com>.
I almost faced that there is no choice but to go for it for VMware
recently, we found that VMware vSphere 5.1 SDK we are currently using has
some backwards compatibility issue with VMware vSphere 4.x systems.
Therefore it makes a valid business case that CloudStack may have to
support different versions of VMware SDK side-by-side at runtime.
Kelven
On 9/18/13 10:42 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>You know what would be really swell is to just switch to jackson. The
>gson
>we use is antiquated. I have no idea what the impact of moving to a
>modern
>version would be. Jackson, IMO, is a far better framework that has a lot
>of momentum. Additionally it allows you to use JAXB annotations so that
>you don't have to use framework dependent annotation in your code. Plus
>it
>does tons and tons of more stuff.
>
>I really don't expect isolated classloaders will come anytime soon. That
>actually relates to things I'm working on right now, but I have a hard
>time
>justifying adding the overhead and complexity of isolated classloaders at
>the moment.
>
>If we really, really, really, really, really think isolating classloaders
>is a good thing, we can talk about it, but it will be a can of worms.
>
>Darren
>
>
>On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng <me...@ufl.edu> wrote:
>
>> Dear all,
>>
>> I am adding an API to CloudStack which utilizes Whirr to launch various
>> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>>
>> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson
>>1.7.1.
>> If I use gson 1.7.1 for Whirr, the following error will happen:
>>
>> com.google.common.util.**concurrent.ExecutionError:
>>java.lang.**NoClassDefFoundError:
>> com/google/gson/TypeAdapter
>>
>> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
>> modify CloudStack to use gson 2.2.2 CloudStack will not build
>>successfully
>> due to the following test error.
>>
>> [INFO] Building Apache CloudStack Core 4.1.1
>> [INFO] ------------------------------**------------------------------**
>> ------------
>> [INFO]
>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
>> [INFO] Deleting /home/meng/cloudstack/core/**target
>> [INFO]
>> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
>> cloud-core ---
>> [INFO]
>> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
>> cloud-core ---
>> [debug] execute contextualize
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
>> src/main/resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
>> cloud-core ---
>> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
>> target/classes
>> [INFO]
>> [INFO] --- maven-resources-plugin:2.5:**testResources
>> (default-testResources) @ cloud-core ---
>> [debug] execute contextualize
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
>> src/test/resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
>> (default-testCompile) @ cloud-core ---
>> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
>> target/test-classes
>> [INFO]
>> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
>> ---
>> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
>> target/surefire-reports
>>
>> ------------------------------**-------------------------
>> T E S T S
>> ------------------------------**-------------------------
>> Running com.cloud.agent.transport.**RequestTest
>> log4j:WARN No appenders could be found for logger
>> (com.cloud.agent.transport.**RequestTest).
>> log4j:WARN Please initialize the log4j system properly.
>> log4j:WARN See
>>http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.ap
>>ache.org/log4j/1.2/faq.html#noconfig>for more info.
>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054
>>sec
>> <<< FAILURE!
>>
>> Results :
>>
>> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>>
>> Tests in error:
>> testLogging(com.cloud.agent.**transport.RequestTest)
>>
>> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>>
>>
>> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
>> building process went through. But when I issued an API --launchCluster
>>to
>> CloudStack, the following error related to gson popped up on CloudStack
>> Management Server:
>>
>> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
>> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
>> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
>> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
>> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
>> snapshot","pool":"/root","**hypervisorType":"KVM","**
>>
>>hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"*
>>*
>> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
>> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
>> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
>> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
>> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
>> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
>> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
>> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
>> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
>> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
>> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
>> LibvirtComputingResource","**gatewayIpAddress":"!
>> 10.244.18
>> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
>> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
>> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
>> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
>> images/","poolType":"**Filesystem","capacityBytes":**
>> 52844687360,"availableBytes":**41535332352},"resourceType":"**
>> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
>> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
>> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
>> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
>> contextMap":{},"wait":0}}]
>> java.lang.ClassCastException:
>>com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
>> cannot be cast to java.lang.Class
>> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> ArrayTypeAdaptor.java:84)
>> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
>> ArrayTypeAdaptor.java:37)
>> at com.google.gson.**TreeTypeAdapter.read(**
>> TreeTypeAdapter.java:58)
>> at com.google.gson.Gson.fromJson(**Gson.java:795)
>> at com.cloud.agent.transport.**Request.getCommands(Request.**
>> java:235)
>> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> processRequest(**AgentManagerImpl.java:1221)
>> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
>> doTask(AgentManagerImpl.java:**1374)
>> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
>> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
>> at com.cloud.utils.nio.Task.run(**Task.java:83)
>> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
>> ThreadPoolExecutor.java:1110)
>> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.**java:722)
>> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
>> exception but pushing on
>>
>> On the CS Managment console I can see the cluster was started then
>>quickly
>> died.
>> Running on provider cloudstack using identity
>>
>>h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0
>>a
>> **K-A-tQRQsZFwnOXQ
>> INFO [whirr.actions.**BootstrapClusterAction]
>>(729061754@qtp-385354117-0:)
>> Bootstrapping cluster
>> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
>> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
>> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
>> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
>> with roles [hadoop-datanode, hadoop-tasktracker]
>> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
>> with roles [hadoop-namenode, hadoop-jobtracker]
>> ...
>> INFO [whirr.actions.**DestroyClusterAction]
>>(729061754@qtp-385354117-0:)
>> Cluster hadoop destroyed
>>
>> I attached the debuging log to this email.
>>
>> The launchCluster API is simple, it calls the LaunchClusterCommand in
>> Whirr to launch a cluster.
>>
>> LaunchClusterResponse response = new LaunchClusterResponse();
>> response.setObjectName("**launchCluster");
>> LaunchClusterCommand command = null;
>> try {
>> command = new LaunchClusterCommand();
>> } catch (Exception ex) {
>>
>>Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
>> null, ex);
>> }
>> String[] args = new String[2];
>> args[0] = "--config";
>> args[1] = config;
>>
>> try {
>> command.run(System.in, System.out, System.err,
>> Arrays.asList(args));
>> } catch (Exception ex) {
>>
>>Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
>> null, ex);
>> }
>> response.setResponseName(**getCommandName());
>> output = "successfully launched the cluster.";
>> response.setOutPut(output);
>> response.setAsync(Boolean.**FALSE);
>> this.setResponseObject(**response);
>>
>> Could someone help me out here? What would be a proper gson version for
>> CloudStack and Whirr to run at the same time?
>>
>> Thanks loads.
>>
>> Best Regards,
>> Meng
>>
>>
>>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Darren Shepherd <da...@gmail.com>.
You know what would be really swell is to just switch to jackson. The gson
we use is antiquated. I have no idea what the impact of moving to a modern
version would be. Jackson, IMO, is a far better framework that has a lot
of momentum. Additionally it allows you to use JAXB annotations so that
you don't have to use framework dependent annotation in your code. Plus it
does tons and tons of more stuff.
I really don't expect isolated classloaders will come anytime soon. That
actually relates to things I'm working on right now, but I have a hard time
justifying adding the overhead and complexity of isolated classloaders at
the moment.
If we really, really, really, really, really think isolating classloaders
is a good thing, we can talk about it, but it will be a can of worms.
Darren
On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng <me...@ufl.edu> wrote:
> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] ------------------------------**------------------------------**
> ------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> ------------------------------**-------------------------
> T E S T S
> ------------------------------**-------------------------
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apache.org/log4j/1.2/faq.html#noconfig>for more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
> testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> contextMap":{},"wait":0}}]
> java.lang.ClassCastException: com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> cannot be cast to java.lang.Class
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:84)
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:37)
> at com.google.gson.**TreeTypeAdapter.read(**
> TreeTypeAdapter.java:58)
> at com.google.gson.Gson.fromJson(**Gson.java:795)
> at com.cloud.agent.transport.**Request.getCommands(Request.**
> java:235)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> processRequest(**AgentManagerImpl.java:1221)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> doTask(AgentManagerImpl.java:**1374)
> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> at com.cloud.utils.nio.Task.run(**Task.java:83)
> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.**java:722)
> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> exception but pushing on
>
> On the CS Managment console I can see the cluster was started then quickly
> died.
> Running on provider cloudstack using identity
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0a
> **K-A-tQRQsZFwnOXQ
> INFO [whirr.actions.**BootstrapClusterAction] (729061754@qtp-385354117-0:)
> Bootstrapping cluster
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
> with roles [hadoop-datanode, hadoop-tasktracker]
> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> with roles [hadoop-namenode, hadoop-jobtracker]
> ...
> INFO [whirr.actions.**DestroyClusterAction] (729061754@qtp-385354117-0:)
> Cluster hadoop destroyed
>
> I attached the debuging log to this email.
>
> The launchCluster API is simple, it calls the LaunchClusterCommand in
> Whirr to launch a cluster.
>
> LaunchClusterResponse response = new LaunchClusterResponse();
> response.setObjectName("**launchCluster");
> LaunchClusterCommand command = null;
> try {
> command = new LaunchClusterCommand();
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> String[] args = new String[2];
> args[0] = "--config";
> args[1] = config;
>
> try {
> command.run(System.in, System.out, System.err,
> Arrays.asList(args));
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> response.setResponseName(**getCommandName());
> output = "successfully launched the cluster.";
> response.setOutPut(output);
> response.setAsync(Boolean.**FALSE);
> this.setResponseObject(**response);
>
> Could someone help me out here? What would be a proper gson version for
> CloudStack and Whirr to run at the same time?
>
> Thanks loads.
>
> Best Regards,
> Meng
>
>
>
Re: conflicting dependencies between CloudStack and Whirr
Posted by Andrew Bayer <an...@gmail.com>.
Good question. The gson dependency in Whirr is coming from jclouds, and
jclouds is very particular about its gson dependencies. Any possibility of
using isolated classloaders in some way?
A.
On Tue, Sep 17, 2013 at 7:14 PM, Han,Meng <me...@ufl.edu> wrote:
> Dear all,
>
> I am adding an API to CloudStack which utilizes Whirr to launch various
> clusters on CloudStack. Now I am facing a dependency conflicting issue.
>
> Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 1.7.1.
> If I use gson 1.7.1 for Whirr, the following error will happen:
>
> com.google.common.util.**concurrent.ExecutionError: java.lang.**NoClassDefFoundError:
> com/google/gson/TypeAdapter
>
> This TypeAdapter class can be found inside gson-2.2.2.jar. However if I
> modify CloudStack to use gson 2.2.2 CloudStack will not build successfully
> due to the following test error.
>
> [INFO] Building Apache CloudStack Core 4.1.1
> [INFO] ------------------------------**------------------------------**
> ------------
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core ---
> [INFO] Deleting /home/meng/cloudstack/core/**target
> [INFO]
> [INFO] --- maven-remote-resources-plugin:**1.3:process (default) @
> cloud-core ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**resources (default-resources) @
> cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**compile (default-compile) @
> cloud-core ---
> [INFO] Compiling 156 source files to /home/meng/cloudstack/core/**
> target/classes
> [INFO]
> [INFO] --- maven-resources-plugin:2.5:**testResources
> (default-testResources) @ cloud-core ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/meng/cloudstack/core/**
> src/test/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.5.1:**testCompile
> (default-testCompile) @ cloud-core ---
> [INFO] Compiling 1 source file to /home/meng/cloudstack/core/**
> target/test-classes
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12:**test (default-test) @ cloud-core
> ---
> [INFO] Surefire report directory: /home/meng/cloudstack/core/**
> target/surefire-reports
>
> ------------------------------**-------------------------
> T E S T S
> ------------------------------**-------------------------
> Running com.cloud.agent.transport.**RequestTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.agent.transport.**RequestTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/**log4j/1.2/faq.html#noconfig<http://logging.apache.org/log4j/1.2/faq.html#noconfig>for more info.
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 sec
> <<< FAILURE!
>
> Results :
>
> Failed tests: testSerDeser(com.cloud.agent.**transport.RequestTest)
>
> Tests in error:
> testLogging(com.cloud.agent.**transport.RequestTest)
>
> Tests run: 4, Failures: 1, Errors: 1, Skipped: 0
>
>
> I ran "mvn -P developer clean install -DskipTests" , the CloudStack
> building process went through. But when I issued an API --launchCluster to
> CloudStack, the following error related to gson popped up on CloudStack
> Management Server:
>
> ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught problem
> with [{"StartupRoutingCommand":{"**cpus":2,"speed":3000,"memory":**
> 3844370432,"dom0MinMemory":**384437043,"poolSync":false,"**
> vms":{"v-2-VM":{"state":"**Running"},"s-1-VM":{"state":"**
> Running"},"r-4-VM":{"state":"**Running"}},"caps":"hvm,**
> snapshot","pool":"/root","**hypervisorType":"KVM","**
> hostDetails":{"com.cloud.**network.Networks.**RouterPrivateIpStrategy":"**
> HostLocal","Host.OS":"CentOS",**"Host.OS.Kernel.Version":"2.6.**
> 32-358.el6.x86_64","Host.OS.**Version":"6.4"},"type":"**
> Routing","dataCenter":"1","**pod":"1","cluster":"1","guid":**
> "a7320748-6346-3c9a-975e-**90ac4ae4a986-**LibvirtComputingResource","**
> name":"meng.acis.ufl.edu","id"**:2,"version":"4.1.1","**
> publicIpAddress":"10.244.18.**55","publicNetmask":"255.0.0.**
> 0","publicMacAddress":"00:23:**ae:94:f7:22","**
> privateIpAddress":"10.244.18.**55","privateMacAddress":"00:**
> 23:ae:94:f7:22","**privateNetmask":"255.0.0.0","**
> storageIpAddress":"10.244.18.**55","storageNetmask":"255.0.0.**
> 0","storageMacAddress":"00:23:**ae:94:f7:22","resourceName":"**
> LibvirtComputingResource","**gatewayIpAddress":"!
> 10.244.18
> .1","contextMap":{},"wait":0}}**,{"StartupStorageCommand":{"**
> totalSize":0,"poolInfo":{"**uuid":"9447c0b1-cc3f-439f-**
> 85f6-13d35539a9ed","host":"10.**244.18.55","localPath":"/var/**
> lib/libvirt/images/","**hostPath":"/var/lib/libvirt/**
> images/","poolType":"**Filesystem","capacityBytes":**
> 52844687360,"availableBytes":**41535332352},"resourceType":"**
> STORAGE_POOL","hostDetails":{}**,"type":"Storage","dataCenter"**
> :"1","pod":"1","guid":"**a7320748-6346-3c9a-975e-**90ac4ae4a986-**
> LibvirtComputingResource","**name":"meng.acis.ufl.edu","id"**
> :2,"version":"4.1.1","**resourceName":"**LibvirtComputingResource","**
> contextMap":{},"wait":0}}]
> java.lang.ClassCastException: com.google.gson.internal.$**Gson$Types$**GenericArrayTypeImpl
> cannot be cast to java.lang.Class
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:84)
> at com.cloud.agent.transport.**ArrayTypeAdaptor.deserialize(**
> ArrayTypeAdaptor.java:37)
> at com.google.gson.**TreeTypeAdapter.read(**
> TreeTypeAdapter.java:58)
> at com.google.gson.Gson.fromJson(**Gson.java:795)
> at com.cloud.agent.transport.**Request.getCommands(Request.**
> java:235)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> processRequest(**AgentManagerImpl.java:1221)
> at com.cloud.agent.manager.**AgentManagerImpl$AgentHandler.**
> doTask(AgentManagerImpl.java:**1374)
> at com.cloud.agent.manager.**ClusteredAgentManagerImpl$**
> ClusteredAgentHandler.doTask(**ClusteredAgentManagerImpl.**java:659)
> at com.cloud.utils.nio.Task.run(**Task.java:83)
> at java.util.concurrent.**ThreadPoolExecutor.runWorker(**
> ThreadPoolExecutor.java:1110)
> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.**java:722)
> WARN [utils.nio.Task] (AgentManager-Handler-2:) Caught the following
> exception but pushing on
>
> On the CS Managment console I can see the cluster was started then quickly
> died.
> Running on provider cloudstack using identity
> h3DKHC9AVlhKnUhpyThMuLhC119QfN**QQ8xhyjbf_**rnu5ZL1QeOWdw7aZRGXVO1VApG6q0a
> **K-A-tQRQsZFwnOXQ
> INFO [whirr.actions.**BootstrapClusterAction] (729061754@qtp-385354117-0:)
> Bootstrapping cluster
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-datanode_**hadoop-tasktracker
> INFO [whirr.compute.**BootstrapTemplate] (729061754@qtp-385354117-0:)
> Configuring template for bootstrap-hadoop-namenode_**hadoop-jobtracker
> INFO [whirr.compute.NodeStarter] (pool-4-thread-2:) Starting 1 node(s)
> with roles [hadoop-datanode, hadoop-tasktracker]
> INFO [whirr.compute.NodeStarter] (pool-4-thread-4:) Starting 1 node(s)
> with roles [hadoop-namenode, hadoop-jobtracker]
> ...
> INFO [whirr.actions.**DestroyClusterAction] (729061754@qtp-385354117-0:)
> Cluster hadoop destroyed
>
> I attached the debuging log to this email.
>
> The launchCluster API is simple, it calls the LaunchClusterCommand in
> Whirr to launch a cluster.
>
> LaunchClusterResponse response = new LaunchClusterResponse();
> response.setObjectName("**launchCluster");
> LaunchClusterCommand command = null;
> try {
> command = new LaunchClusterCommand();
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> String[] args = new String[2];
> args[0] = "--config";
> args[1] = config;
>
> try {
> command.run(System.in, System.out, System.err,
> Arrays.asList(args));
> } catch (Exception ex) {
> Logger.getLogger(**LaunchClusterCmd.class.**getName()).log(Level.SEVERE,
> null, ex);
> }
> response.setResponseName(**getCommandName());
> output = "successfully launched the cluster.";
> response.setOutPut(output);
> response.setAsync(Boolean.**FALSE);
> this.setResponseObject(**response);
>
> Could someone help me out here? What would be a proper gson version for
> CloudStack and Whirr to run at the same time?
>
> Thanks loads.
>
> Best Regards,
> Meng
>
>
>