You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Derek Cole <de...@gmail.com> on 2013/12/04 17:54:46 UTC
Cloudstack user-data empty for VMs
Hello,
I am attempting to use the cloudstack API to get user-data for my VMs.
I am creating the VMs with the puppet cloudstack-resources module, and it
seems to populate it (From the cloudstack GUI, under "Group" I see my data)
However, when I do a
wget http://router/latest/user-data
I get back an empty file. On the VR itself I have checked and there are a
bunch of /var/www/html/userdata/IP/user-data files, but every one of them
is empty. This same applies to VMs that I added the "Group" user data
manually.
The VM's are all FreeBSD10, and I am using xenserver 6.2, and cloudstack 4.2
Is this a bug? Is there a workaround or a fix?
Thanks
Re: Cloudstack user-data empty for VMs
Posted by David Nalley <da...@gnsa.us>.
On Wed, Dec 4, 2013 at 11:38 PM, Derek Cole <de...@gmail.com> wrote:
> I will see what I can do. I dont have a github presently, but since I am
> mostly working with FreeBSD I had to fix puppet's service resource, jason
> hancocks section that look for dhcp leases and a couple of other things.
> The userdata fix I put in seems to be working, though not when the puppet
> agent is ubuntu presently, so I am trying to track that down.
>
> I only saw one other branch in dans and it was showing like 9 months ago
> for the latest work - is that the correct place I am looking?
>
Don't worry about github if it's too much hassle - a simple patch is
fine as well.
The problem with Ubuntu is probably related to the DHCP client, but not sure.
Yeah 9 months sounds about right - it's languished a bit since Dan
changed employers.
--David
Re: Cloudstack user-data empty for VMs
Posted by Derek Cole <de...@gmail.com>.
I will see what I can do. I dont have a github presently, but since I am
mostly working with FreeBSD I had to fix puppet's service resource, jason
hancocks section that look for dhcp leases and a couple of other things.
The userdata fix I put in seems to be working, though not when the puppet
agent is ubuntu presently, so I am trying to track that down.
I only saw one other branch in dans and it was showing like 9 months ago
for the latest work - is that the correct place I am looking?
-Derek
On Dec 4, 2013 8:57 PM, "David Nalley" <da...@gnsa.us> wrote:
> Yeah - it's definitely a work in progress - if you look at Dan Bode's
> cloudstack_resources, he actually has a ton of additional work in a
> different branch that hasn't yet been validated. It's on my list to
> get around to that in the next few weeks. What you likely saw (esp if
> it was my presentation) was a combination of Jason Hancock's work
> around userdata and his fact - and then separately the resources and
> types for CloudStack. There are some folks who use CloudStack's
> records of hosts as a ENC - and that's where the 'Group=' stuff leaks
> into the presentation - essentially they query for the host - and
> parse the Group records and make catalog decisions off of that rather
> than a fact. (Keep in mind the 'down side' of going the fact route, is
> that essentially an attacker could pass any information back to puppet
> and get anything in the catalog theoretically)
>
> Please do submit a pull request for the userdata feature (feel free to
> submit it against my repo or Dan's)
>
> --David
>
> On Wed, Dec 4, 2013 at 6:44 PM, Derek Cole <de...@gmail.com> wrote:
> > Well I started to post about getting something broken. I modified
> > cloudstack_resources to take an argument for userdata. I was getting all
> > kinds of issues with that, until I realized that the argument in userdata
> > has to be base64 encoded. I fixed that, and now my VMs can get their
> > metadata! I had seen a demo with this software at a puppet conference but
> > it's become clear to me it's not exactly complete, so I have been
> fighting
> > to learn ruby ever since to make things work.
> >
> > Thanks for the help.
>
Re: Cloudstack user-data empty for VMs
Posted by David Nalley <da...@gnsa.us>.
Yeah - it's definitely a work in progress - if you look at Dan Bode's
cloudstack_resources, he actually has a ton of additional work in a
different branch that hasn't yet been validated. It's on my list to
get around to that in the next few weeks. What you likely saw (esp if
it was my presentation) was a combination of Jason Hancock's work
around userdata and his fact - and then separately the resources and
types for CloudStack. There are some folks who use CloudStack's
records of hosts as a ENC - and that's where the 'Group=' stuff leaks
into the presentation - essentially they query for the host - and
parse the Group records and make catalog decisions off of that rather
than a fact. (Keep in mind the 'down side' of going the fact route, is
that essentially an attacker could pass any information back to puppet
and get anything in the catalog theoretically)
Please do submit a pull request for the userdata feature (feel free to
submit it against my repo or Dan's)
--David
On Wed, Dec 4, 2013 at 6:44 PM, Derek Cole <de...@gmail.com> wrote:
> Well I started to post about getting something broken. I modified
> cloudstack_resources to take an argument for userdata. I was getting all
> kinds of issues with that, until I realized that the argument in userdata
> has to be base64 encoded. I fixed that, and now my VMs can get their
> metadata! I had seen a demo with this software at a puppet conference but
> it's become clear to me it's not exactly complete, so I have been fighting
> to learn ruby ever since to make things work.
>
> Thanks for the help.
Re: Cloudstack user-data empty for VMs
Posted by Derek Cole <de...@gmail.com>.
Well I started to post about getting something broken. I modified
cloudstack_resources to take an argument for userdata. I was getting all
kinds of issues with that, until I realized that the argument in userdata
has to be base64 encoded. I fixed that, and now my VMs can get their
metadata! I had seen a demo with this software at a puppet conference but
it's become clear to me it's not exactly complete, so I have been fighting
to learn ruby ever since to make things work.
Thanks for the help.
Re: Cloudstack user-data empty for VMs
Posted by Carlos Reátegui <cr...@gmail.com>.
What you want to do should work.
Not familiar with cloudstack_resources. Does it have a means to set user data when creating an instance? If so, have you checked the user data manually (ie querying the vr/dhcp server) to see that it was set?
> On Dec 4, 2013, at 3:19 PM, Derek Cole <de...@gmail.com> wrote:
>
> Hrmm - Maybe I am not being exactly clear.
>
> I originally started out trying to use puppet to not only configure my VMs,
> but provision them in the cloud as well. I started out with a template in
> cloudstack, and have been using cloudstack_resources to create new VMs
> automatically when my puppet manifest is applied.
>
> Creating the VMs and all seems to work, however, I need a way to push
> specific configurations to specific boxes based on their role, so I was
> hoping to use the "role=whatever" userdata to be able to distinguish the
> different types of VMs that I have provisioned without having to have a
> bunch of node blocks to distinguish them by name.
>
> Maybe using cloudmonkey or some other tool is more effective, however I
> guess I was just trying to figure out how to use the supposedly already
> written puppet types to give me something out of the box that is "one
> command" provisioning and configuring.
>
> -Derek
>
>
> On Wed, Dec 4, 2013 at 2:56 PM, Hiroki Ohashi <si...@gmail.com>wrote:
>
>> Hi Derek
>>
>> When you use user-data, you have to provide user-data and deliver it
>> to virtual router. Unfortunately, CloudStack only provides REST API to
>> deliver user-data to virtual router now. This API is
>> deployVirtualMachine that creates instance and set instance specific
>> data.
>>
>> For example, I assume you want to deliver a shell script as user-data
>> below.
>>
>> ----------------------------------------------------------------------
>> #! /bin/sh
>>
>> echo "Hello user-data!"
>> ----------------------------------------------------------------------
>>
>> In this case, you can call deployVirtualMachine by CloudMonkey like
>> this.
>>
>> $ cloudmonkey deploy virtualmachine serviceofferingid=XXX
>> templateid=XXX zoneid=XXX networkids=XXX
>> userdata='IyEgL2Jpbi9zaAoKZWNobyAiSGVsbG8gdXNlci1kYXRhISIK'
>>
>> As a result of the command, a new instance is created with user-data
>> delivered to virtual router, so you can access user-data from the
>> instance.
>>
>> CloudMonkey is CLI tool to call CloudStack REST API easily and you can
>> give parameters as key value pair. In the example, user-data is base64
>> encoded shell script and provied as a parameter.
>>
>> Best regards
>> Hiroki Ohashi
>>
>> 2013/12/5 Derek Cole <de...@gmail.com>:
>>> I was kind of wondering about that, when I saw it earlier. It appears
>> that
>>> maybe the cloudstack_resources module has become out of date? I think you
>>> are the preparer of a slide-show I was following here:
>>> http://www.slideshare.net/PuppetLabs/cloudstack-18489665
>>>
>>> It seems like in those slides, Group was being used to hold the roles?
>> This
>>> is why I assumed that was OK. I was doing something like
>>>
>>> case $::role {
>>> 'db': {
>>> notify {"detected a fact named role with value
>> db"
>>> :}
>>> }
>>> }
>>> because I thought that the custom fact here:
>>>
>> https://github.com/jasonhancock/puppet-facts/blob/master/cloudstack_userdata.rb
>>>
>>> Turned the cloudstack userdata into a fact where if you had
>>> "role=db,foo=bar" you would get two custom facts, role and foo, with
>> their
>>> respective values.
>>>
>>> May I ask for some clarity if that is not the intended usage?
>>>
>>> In my case, obviously cloudstack_userdata is not generating the correct
>>> facts, because the wget that is in that code returns nothing because
>>> userdata is empty.
>>>
>>> Thanks,
>>>
>>> -Derek
>>>
>>>
>>>
>>>> On Wed, Dec 4, 2013 at 12:57 PM, David Nalley <da...@gnsa.us> wrote:
>>>>
>>>> Hi Derek:
>>>>
>>>> Glad that the puppet resources are useful.
>>>>
>>>> Setting user-data is different than setting group:
>>>>
>>>> Look at the options here:
>>>>
>>>>
>> http://cloudstack.apache.org/docs/api/apidocs-4.2/user/deployVirtualMachine.html
>>>>
>>>> --David
>>>>
>>>> On Wed, Dec 4, 2013 at 11:54 AM, Derek Cole <de...@gmail.com>
>> wrote:
>>>>> Hello,
>>>>>
>>>>> I am attempting to use the cloudstack API to get user-data for my VMs.
>>>>>
>>>>> I am creating the VMs with the puppet cloudstack-resources module,
>> and it
>>>>> seems to populate it (From the cloudstack GUI, under "Group" I see my
>>>> data)
>>>>>
>>>>> However, when I do a
>>>>>
>>>>> wget http://router/latest/user-data
>>>>>
>>>>> I get back an empty file. On the VR itself I have checked and there
>> are a
>>>>> bunch of /var/www/html/userdata/IP/user-data files, but every one of
>> them
>>>>> is empty. This same applies to VMs that I added the "Group" user data
>>>>> manually.
>>>>>
>>>>> The VM's are all FreeBSD10, and I am using xenserver 6.2, and
>> cloudstack
>>>> 4.2
>>>>>
>>>>> Is this a bug? Is there a workaround or a fix?
>>>>>
>>>>> Thanks
>>>>
>>
>>
>>
>> --
>> 大橋 宏樹
>> mail: silvernshell@gmail.com
>>
Re: Cloudstack user-data empty for VMs
Posted by Derek Cole <de...@gmail.com>.
Hrmm - Maybe I am not being exactly clear.
I originally started out trying to use puppet to not only configure my VMs,
but provision them in the cloud as well. I started out with a template in
cloudstack, and have been using cloudstack_resources to create new VMs
automatically when my puppet manifest is applied.
Creating the VMs and all seems to work, however, I need a way to push
specific configurations to specific boxes based on their role, so I was
hoping to use the "role=whatever" userdata to be able to distinguish the
different types of VMs that I have provisioned without having to have a
bunch of node blocks to distinguish them by name.
Maybe using cloudmonkey or some other tool is more effective, however I
guess I was just trying to figure out how to use the supposedly already
written puppet types to give me something out of the box that is "one
command" provisioning and configuring.
-Derek
On Wed, Dec 4, 2013 at 2:56 PM, Hiroki Ohashi <si...@gmail.com>wrote:
> Hi Derek
>
> When you use user-data, you have to provide user-data and deliver it
> to virtual router. Unfortunately, CloudStack only provides REST API to
> deliver user-data to virtual router now. This API is
> deployVirtualMachine that creates instance and set instance specific
> data.
>
> For example, I assume you want to deliver a shell script as user-data
> below.
>
> ----------------------------------------------------------------------
> #! /bin/sh
>
> echo "Hello user-data!"
> ----------------------------------------------------------------------
>
> In this case, you can call deployVirtualMachine by CloudMonkey like
> this.
>
> $ cloudmonkey deploy virtualmachine serviceofferingid=XXX
> templateid=XXX zoneid=XXX networkids=XXX
> userdata='IyEgL2Jpbi9zaAoKZWNobyAiSGVsbG8gdXNlci1kYXRhISIK'
>
> As a result of the command, a new instance is created with user-data
> delivered to virtual router, so you can access user-data from the
> instance.
>
> CloudMonkey is CLI tool to call CloudStack REST API easily and you can
> give parameters as key value pair. In the example, user-data is base64
> encoded shell script and provied as a parameter.
>
> Best regards
> Hiroki Ohashi
>
> 2013/12/5 Derek Cole <de...@gmail.com>:
> > I was kind of wondering about that, when I saw it earlier. It appears
> that
> > maybe the cloudstack_resources module has become out of date? I think you
> > are the preparer of a slide-show I was following here:
> > http://www.slideshare.net/PuppetLabs/cloudstack-18489665
> >
> > It seems like in those slides, Group was being used to hold the roles?
> This
> > is why I assumed that was OK. I was doing something like
> >
> > case $::role {
> > 'db': {
> > notify {"detected a fact named role with value
> db"
> > :}
> > }
> > }
> > because I thought that the custom fact here:
> >
> https://github.com/jasonhancock/puppet-facts/blob/master/cloudstack_userdata.rb
> >
> > Turned the cloudstack userdata into a fact where if you had
> > "role=db,foo=bar" you would get two custom facts, role and foo, with
> their
> > respective values.
> >
> > May I ask for some clarity if that is not the intended usage?
> >
> > In my case, obviously cloudstack_userdata is not generating the correct
> > facts, because the wget that is in that code returns nothing because
> > userdata is empty.
> >
> > Thanks,
> >
> > -Derek
> >
> >
> >
> > On Wed, Dec 4, 2013 at 12:57 PM, David Nalley <da...@gnsa.us> wrote:
> >
> >> Hi Derek:
> >>
> >> Glad that the puppet resources are useful.
> >>
> >> Setting user-data is different than setting group:
> >>
> >> Look at the options here:
> >>
> >>
> http://cloudstack.apache.org/docs/api/apidocs-4.2/user/deployVirtualMachine.html
> >>
> >> --David
> >>
> >> On Wed, Dec 4, 2013 at 11:54 AM, Derek Cole <de...@gmail.com>
> wrote:
> >> > Hello,
> >> >
> >> > I am attempting to use the cloudstack API to get user-data for my VMs.
> >> >
> >> > I am creating the VMs with the puppet cloudstack-resources module,
> and it
> >> > seems to populate it (From the cloudstack GUI, under "Group" I see my
> >> data)
> >> >
> >> > However, when I do a
> >> >
> >> > wget http://router/latest/user-data
> >> >
> >> > I get back an empty file. On the VR itself I have checked and there
> are a
> >> > bunch of /var/www/html/userdata/IP/user-data files, but every one of
> them
> >> > is empty. This same applies to VMs that I added the "Group" user data
> >> > manually.
> >> >
> >> > The VM's are all FreeBSD10, and I am using xenserver 6.2, and
> cloudstack
> >> 4.2
> >> >
> >> > Is this a bug? Is there a workaround or a fix?
> >> >
> >> > Thanks
> >>
>
>
>
> --
> 大橋 宏樹
> mail: silvernshell@gmail.com
>
Re: Cloudstack user-data empty for VMs
Posted by Hiroki Ohashi <si...@gmail.com>.
Hi Derek
When you use user-data, you have to provide user-data and deliver it
to virtual router. Unfortunately, CloudStack only provides REST API to
deliver user-data to virtual router now. This API is
deployVirtualMachine that creates instance and set instance specific
data.
For example, I assume you want to deliver a shell script as user-data
below.
----------------------------------------------------------------------
#! /bin/sh
echo "Hello user-data!"
----------------------------------------------------------------------
In this case, you can call deployVirtualMachine by CloudMonkey like
this.
$ cloudmonkey deploy virtualmachine serviceofferingid=XXX
templateid=XXX zoneid=XXX networkids=XXX
userdata='IyEgL2Jpbi9zaAoKZWNobyAiSGVsbG8gdXNlci1kYXRhISIK'
As a result of the command, a new instance is created with user-data
delivered to virtual router, so you can access user-data from the
instance.
CloudMonkey is CLI tool to call CloudStack REST API easily and you can
give parameters as key value pair. In the example, user-data is base64
encoded shell script and provied as a parameter.
Best regards
Hiroki Ohashi
2013/12/5 Derek Cole <de...@gmail.com>:
> I was kind of wondering about that, when I saw it earlier. It appears that
> maybe the cloudstack_resources module has become out of date? I think you
> are the preparer of a slide-show I was following here:
> http://www.slideshare.net/PuppetLabs/cloudstack-18489665
>
> It seems like in those slides, Group was being used to hold the roles? This
> is why I assumed that was OK. I was doing something like
>
> case $::role {
> 'db': {
> notify {"detected a fact named role with value db"
> :}
> }
> }
> because I thought that the custom fact here:
> https://github.com/jasonhancock/puppet-facts/blob/master/cloudstack_userdata.rb
>
> Turned the cloudstack userdata into a fact where if you had
> "role=db,foo=bar" you would get two custom facts, role and foo, with their
> respective values.
>
> May I ask for some clarity if that is not the intended usage?
>
> In my case, obviously cloudstack_userdata is not generating the correct
> facts, because the wget that is in that code returns nothing because
> userdata is empty.
>
> Thanks,
>
> -Derek
>
>
>
> On Wed, Dec 4, 2013 at 12:57 PM, David Nalley <da...@gnsa.us> wrote:
>
>> Hi Derek:
>>
>> Glad that the puppet resources are useful.
>>
>> Setting user-data is different than setting group:
>>
>> Look at the options here:
>>
>> http://cloudstack.apache.org/docs/api/apidocs-4.2/user/deployVirtualMachine.html
>>
>> --David
>>
>> On Wed, Dec 4, 2013 at 11:54 AM, Derek Cole <de...@gmail.com> wrote:
>> > Hello,
>> >
>> > I am attempting to use the cloudstack API to get user-data for my VMs.
>> >
>> > I am creating the VMs with the puppet cloudstack-resources module, and it
>> > seems to populate it (From the cloudstack GUI, under "Group" I see my
>> data)
>> >
>> > However, when I do a
>> >
>> > wget http://router/latest/user-data
>> >
>> > I get back an empty file. On the VR itself I have checked and there are a
>> > bunch of /var/www/html/userdata/IP/user-data files, but every one of them
>> > is empty. This same applies to VMs that I added the "Group" user data
>> > manually.
>> >
>> > The VM's are all FreeBSD10, and I am using xenserver 6.2, and cloudstack
>> 4.2
>> >
>> > Is this a bug? Is there a workaround or a fix?
>> >
>> > Thanks
>>
--
大橋 宏樹
mail: silvernshell@gmail.com
Re: Cloudstack user-data empty for VMs
Posted by Derek Cole <de...@gmail.com>.
I was kind of wondering about that, when I saw it earlier. It appears that
maybe the cloudstack_resources module has become out of date? I think you
are the preparer of a slide-show I was following here:
http://www.slideshare.net/PuppetLabs/cloudstack-18489665
It seems like in those slides, Group was being used to hold the roles? This
is why I assumed that was OK. I was doing something like
case $::role {
'db': {
notify {"detected a fact named role with value db"
:}
}
}
because I thought that the custom fact here:
https://github.com/jasonhancock/puppet-facts/blob/master/cloudstack_userdata.rb
Turned the cloudstack userdata into a fact where if you had
"role=db,foo=bar" you would get two custom facts, role and foo, with their
respective values.
May I ask for some clarity if that is not the intended usage?
In my case, obviously cloudstack_userdata is not generating the correct
facts, because the wget that is in that code returns nothing because
userdata is empty.
Thanks,
-Derek
On Wed, Dec 4, 2013 at 12:57 PM, David Nalley <da...@gnsa.us> wrote:
> Hi Derek:
>
> Glad that the puppet resources are useful.
>
> Setting user-data is different than setting group:
>
> Look at the options here:
>
> http://cloudstack.apache.org/docs/api/apidocs-4.2/user/deployVirtualMachine.html
>
> --David
>
> On Wed, Dec 4, 2013 at 11:54 AM, Derek Cole <de...@gmail.com> wrote:
> > Hello,
> >
> > I am attempting to use the cloudstack API to get user-data for my VMs.
> >
> > I am creating the VMs with the puppet cloudstack-resources module, and it
> > seems to populate it (From the cloudstack GUI, under "Group" I see my
> data)
> >
> > However, when I do a
> >
> > wget http://router/latest/user-data
> >
> > I get back an empty file. On the VR itself I have checked and there are a
> > bunch of /var/www/html/userdata/IP/user-data files, but every one of them
> > is empty. This same applies to VMs that I added the "Group" user data
> > manually.
> >
> > The VM's are all FreeBSD10, and I am using xenserver 6.2, and cloudstack
> 4.2
> >
> > Is this a bug? Is there a workaround or a fix?
> >
> > Thanks
>
Re: Cloudstack user-data empty for VMs
Posted by David Nalley <da...@gnsa.us>.
Hi Derek:
Glad that the puppet resources are useful.
Setting user-data is different than setting group:
Look at the options here:
http://cloudstack.apache.org/docs/api/apidocs-4.2/user/deployVirtualMachine.html
--David
On Wed, Dec 4, 2013 at 11:54 AM, Derek Cole <de...@gmail.com> wrote:
> Hello,
>
> I am attempting to use the cloudstack API to get user-data for my VMs.
>
> I am creating the VMs with the puppet cloudstack-resources module, and it
> seems to populate it (From the cloudstack GUI, under "Group" I see my data)
>
> However, when I do a
>
> wget http://router/latest/user-data
>
> I get back an empty file. On the VR itself I have checked and there are a
> bunch of /var/www/html/userdata/IP/user-data files, but every one of them
> is empty. This same applies to VMs that I added the "Group" user data
> manually.
>
> The VM's are all FreeBSD10, and I am using xenserver 6.2, and cloudstack 4.2
>
> Is this a bug? Is there a workaround or a fix?
>
> Thanks