You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Han,Meng" <me...@ufl.edu> on 2013/09/10 20:23:53 UTC
java.lang.NullPointerException
Hi all,
I am using whirr to start a hadoop cluster on Cloudstack, I encounter
the following exception
2013-09-10 14:06:49,096 ERROR [org.apache.whirr.compute.StartupProcess]
(pool-3-thread-2) Unexpected error while starting 1 nodes, minimum 1
nodes for [hadoop-namenode, hadoop-jobtracker] of cluster hadoop
java.util.concurrent.ExecutionException:
java.lang.NullPointerException: encoded
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at
org.apache.whirr.compute.StartupProcess.waitForOutcomes(StartupProcess.java:130)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:83)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:41)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NullPointerException: encoded
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:208)
at
org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.firstGroupInPatternOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:140)
at
org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.groupInUniqueNameOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:136)
at
org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:107)
at
org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:61)
at
com.google.common.base.Functions$FunctionComposition.apply(Functions.java:210)
at com.google.common.collect.Iterators$8.transform(Iterators.java:860)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:701)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators.any(Iterators.java:736)
at com.google.common.collect.Iterables.any(Iterables.java:623)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.getNextNames(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:188)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:122)
at
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:212)
at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:55)
at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:30)
... 5 more
2013-09-10 14:06:49,106 INFO
[org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished
running destroy phase scripts on all cluster instances
2013-09-10 14:06:49,106 INFO
[org.apache.whirr.actions.DestroyClusterAction] (main) Destroying hadoop
cluster
2013-09-10 14:06:49,108 DEBUG [jclouds.compute] (main) >> destroying
nodes matching(inGroup(hadoop))
2013-09-10 14:06:49,110 DEBUG
[org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod] (main) >>
invoking listVirtualMachines
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true
HTTP/1.1
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> Accept:
application/json
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >>
"apikey=h3dkhc9avlhknuhpythmulhc119qfnqq8xhyjbf_rnu5zl1qeowdw7azrgxvo1vapg6q0ak-a-tqrqszfwnoxq&command=listvirtualmachines&listall=true&response=json"
2013-09-10 14:06:49,112 DEBUG [jclouds.signature] (main) <<
"8CdMYkm15Udov76n0lspNNZG0fQ="
2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << Accept:
application/json
2013-09-10 14:06:49,114 DEBUG
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
Sending request 1519465049: GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,114 DEBUG [jclouds.headers] (main) >> GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,115 DEBUG [jclouds.headers] (main) >> Accept:
application/json
2013-09-10 14:06:49,131 DEBUG
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
Receiving response 1519465049: HTTP/1.1 200 OK
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << HTTP/1.1 200
OK
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Date: Tue, 10
Sep 2013 18:06:48 GMT
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Server:
Apache-Coyote/1.1
2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) << Content-Type:
text/javascript;charset=UTF-8
2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) <<
Content-Length: 1550
2013-09-10 14:06:49,132 DEBUG [jclouds.wire] (main) << "{
"listvirtualmachinesresponse" : { "count":1 ,"virtualmachine" : [
{"id":"a978e52f-9ee8-433e-abf5-10986fd0c16e","name":"a978e52f-9ee8-433e-abf5-10986fd0c16e","account":"admin","domainid":"611281e6-1a1b-11e3-bce5-0023ae94f722","domain":"ROOT","created":"2013-09-10T09:30:29-0400","state":"Running","haenable":false,"zoneid":"62857481-bb61-4607-ac7d-a450135c83a6","zonename":"zone1","hostid":"08b35a58-6a82-4150-be18-e95f110d44de","hostname":"meng.acis.ufl.edu","templateid":"616bd368-1a1b-11e3-bce5-0023ae94f722","templatename":"CentOS
5.5(64-bit) no GUI (KVM)","templatedisplaytext":"CentOS 5.5(64-bit) no
GUI
(KVM)","passwordenabled":false,"serviceofferingid":"cfd100e5-5f59-4e8a-a8e1-9706c7a11488","serviceofferingname":"Small
Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"5.14%","networkkbsread":4479,"networkkbswrite":142,"guestosid":"6716e852-1a1b-11e3-bce5-0023ae94f722","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"b414da10-1a1b-11e3-bce5-0023ae94f722","name":"default","description":"Default
Security
Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[]}],"nic":[{"id":"d66a5aac-d15f-4e08-90c5-73a476db1240","networkid":"7da7d83c-ec6b-4e1c-9791-cdc40191eb96","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"10.244.18.1","ipaddress":"10.244.18.61","isolationuri":"ec2://untagged","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"06:23:bc:00:00:10"}],"hypervisor":"KVM","instancename":"i-2-3-VM","tags":[]}
] } }"
On the cloudstack management server side, I didn't say any instance
even starting. I couldn't find any DEBUG information on the
management-server.log.
Below is the cluster properties file I use:
# Set the name of your hadoop cluster
whirr.cluster-name=hadoop
whirr.provider=cloudstack
# Change the name of cluster admin user
whirr.cluster-user=${sys:user.name}
# Change the number of machines in the cluster here
# Below we define one hadoop namenode and 3 hadoop datanode
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1
hadoop-datanode+hadoop-tasktracker
# Specify which distribution of hadoop you want to use
# Here we choose to use the Cloudera distribution
whirr.env.repo=cdh4
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
# Use a specific instance type.
# Specify the uuid of the CloudStack service offering to use for the
instances of your hadoop cluster
whirr.hardware-id=cfd100e5-5f59-4e8a-a8e1-9706c7a11488
# If you use ssh key pairs to access instances in the cloud
# Specify them like so
whirr.private-key-file=/home/meng/.ssh/id_rsa
whirr.public-key-file=/home/meng/.ssh/id_rsa.pub
# Specify the template to use for the instances
# This is the uuid of the CloudStack template
whirr.image-id=616bd368-1a1b-11e3-bce5-0023ae94f722
whirr.bootstrap-user=root:password
whirr.use-cloudstack-security-group=true
Could someone give me a pointer what might be the possible reason for
this error?
Thank you.
Best Regards,
Meng
Re: java.lang.NullPointerException
Posted by "Han,Meng" <me...@ufl.edu>.
On Tue, 10 Sep 2013 14:23:53 -0400, Han,Meng wrote:
> Hi all,
>
> I am using whirr to start a hadoop cluster on Cloudstack, I encounter
> the following exception
>
> 2013-09-10 14:06:49,096 ERROR
> [org.apache.whirr.compute.StartupProcess] (pool-3-thread-2)
> Unexpected
> error while starting 1 nodes, minimum 1 nodes for [hadoop-namenode,
> hadoop-jobtracker] of cluster hadoop
> java.util.concurrent.ExecutionException:
> java.lang.NullPointerException: encoded
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
>
> org.apache.whirr.compute.StartupProcess.waitForOutcomes(StartupProcess.java:130)
> at
> org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:83)
> at
> org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:41)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.NullPointerException: encoded
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:208)
> at
>
> org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.firstGroupInPatternOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:140)
> at
>
> org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.groupInUniqueNameOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:136)
> at
>
> org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:107)
> at
>
> org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:61)
> at
>
> com.google.common.base.Functions$FunctionComposition.apply(Functions.java:210)
> at
> com.google.common.collect.Iterators$8.transform(Iterators.java:860)
> at
>
> com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:701)
> at
>
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
>
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at com.google.common.collect.Iterators.any(Iterators.java:736)
> at com.google.common.collect.Iterables.any(Iterables.java:623)
> at
>
> org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.getNextNames(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:188)
> at
>
> org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:122)
> at
>
> org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:212)
> at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:55)
> at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:30)
> ... 5 more
> 2013-09-10 14:06:49,106 INFO
> [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished
> running destroy phase scripts on all cluster instances
> 2013-09-10 14:06:49,106 INFO
> [org.apache.whirr.actions.DestroyClusterAction] (main) Destroying
> hadoop cluster
> 2013-09-10 14:06:49,108 DEBUG [jclouds.compute] (main) >> destroying
> nodes matching(inGroup(hadoop))
> 2013-09-10 14:06:49,110 DEBUG
> [org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod] (main) >>
> invoking listVirtualMachines
> 2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> GET
>
> http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true
> HTTP/1.1
> 2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> Accept:
> application/json
> 2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >>
>
> "apikey=h3dkhc9avlhknuhpythmulhc119qfnqq8xhyjbf_rnu5zl1qeowdw7azrgxvo1vapg6q0ak-a-tqrqszfwnoxq&command=listvirtualmachines&listall=true&response=json"
> 2013-09-10 14:06:49,112 DEBUG [jclouds.signature] (main) <<
> "8CdMYkm15Udov76n0lspNNZG0fQ="
> 2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << GET
>
> http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
> HTTP/1.1
> 2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << Accept:
> application/json
> 2013-09-10 14:06:49,114 DEBUG
> [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
> Sending request 1519465049: GET
>
> http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
> HTTP/1.1
> 2013-09-10 14:06:49,114 DEBUG [jclouds.headers] (main) >> GET
>
> http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
> HTTP/1.1
> 2013-09-10 14:06:49,115 DEBUG [jclouds.headers] (main) >> Accept:
> application/json
> 2013-09-10 14:06:49,131 DEBUG
> [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
> Receiving response 1519465049: HTTP/1.1 200 OK
> 2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << HTTP/1.1
> 200 OK
> 2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Date: Tue,
> 10 Sep 2013 18:06:48 GMT
> 2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Server:
> Apache-Coyote/1.1
> 2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) <<
> Content-Type: text/javascript;charset=UTF-8
> 2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) <<
> Content-Length: 1550
> 2013-09-10 14:06:49,132 DEBUG [jclouds.wire] (main) << "{
> "listvirtualmachinesresponse" : { "count":1 ,"virtualmachine" : [
>
> {"id":"a978e52f-9ee8-433e-abf5-10986fd0c16e","name":"a978e52f-9ee8-433e-abf5-10986fd0c16e","account":"admin","domainid":"611281e6-1a1b-11e3-bce5-0023ae94f722","domain":"ROOT","created":"2013-09-10T09:30:29-0400","state":"Running","haenable":false,"zoneid":"62857481-bb61-4607-ac7d-a450135c83a6","zonename":"zone1","hostid":"08b35a58-6a82-4150-be18-e95f110d44de","hostname":"meng.acis.ufl.edu","templateid":"616bd368-1a1b-11e3-bce5-0023ae94f722","templatename":"CentOS
> 5.5(64-bit) no GUI (KVM)","templatedisplaytext":"CentOS 5.5(64-bit)
> no GUI
>
> (KVM)","passwordenabled":false,"serviceofferingid":"cfd100e5-5f59-4e8a-a8e1-9706c7a11488","serviceofferingname":"Small
>
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"5.14%","networkkbsread":4479,"networkkbswrite":142,"guestosid":"6716e852-1a1b-11e3-bce5-0023ae94f722","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"b414da10-1a1b-11e3-bce5-0023ae94f722","name":"default","description":"Default
> Security
>
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[]}],"nic":[{"id":"d66a5aac-d15f-4e08-90c5-73a476db1240","networkid":"7da7d83c-ec6b-4e1c-9791-cdc40191eb96","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"10.244.18.1","ipaddress":"10.244.18.61","isolationuri":"ec2://untagged","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"06:23:bc:00:00:10"}],"hypervisor":"KVM","instancename":"i-2-3-VM","tags":[]}
> ] } }"
>
>
> On the cloudstack management server side, I didn't say any instance
> even starting. I couldn't find any DEBUG information on the
> management-server.log.
>
> Below is the cluster properties file I use:
>
> # Set the name of your hadoop cluster
> whirr.cluster-name=hadoop
>
> whirr.provider=cloudstack
> # Change the name of cluster admin user
> whirr.cluster-user=${sys:user.name}
> # Change the number of machines in the cluster here
> # Below we define one hadoop namenode and 3 hadoop datanode
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1
> hadoop-datanode+hadoop-tasktracker
> # Specify which distribution of hadoop you want to use
> # Here we choose to use the Cloudera distribution
> whirr.env.repo=cdh4
> whirr.hadoop.install-function=install_cdh_hadoop
> whirr.hadoop.configure-function=configure_cdh_hadoop
> # Use a specific instance type.
> # Specify the uuid of the CloudStack service offering to use for the
> instances of your hadoop cluster
> whirr.hardware-id=cfd100e5-5f59-4e8a-a8e1-9706c7a11488
> # If you use ssh key pairs to access instances in the cloud
> # Specify them like so
> whirr.private-key-file=/home/meng/.ssh/id_rsa
> whirr.public-key-file=/home/meng/.ssh/id_rsa.pub
> # Specify the template to use for the instances
> # This is the uuid of the CloudStack template
> whirr.image-id=616bd368-1a1b-11e3-bce5-0023ae94f722
> whirr.bootstrap-user=root:password
> whirr.use-cloudstack-security-group=true
>
> Could someone give me a pointer what might be the possible reason for
> this error?
>
>
> Thank you.
>
> Best Regards,
> Meng
I restarted the cloudstack-management server, now the issue is gone.