You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@cloudstack.apache.org by Rohit Yadav <ro...@shapeblue.com> on 2014/08/08 19:50:41 UTC

[DISCUSS] Removing template URL format checking logic

Hi,

With reference to https://issues.apache.org/jira/browse/CLOUDSTACK-5512 Marcus and I think we should remove the template URL format checking logic because:

- It does not handle pre-signed URL (say something that does not end with .vhd etc, but has bunch of http params)
- One can game the system by say renaming any file to respective format
- We dumb down, take whatever URL user gives and use the format they specify in their register template API call

Marcus also notes that TemplateUtils utility would validate selected format.

Please discuss if you’ve any use-case that can get affected by this?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: [DISCUSS] Removing template URL format checking logic

Posted by Rohit Yadav <ro...@shapeblue.com>.

Hi Daan,

On 13-Aug-2014, at 12:34 pm, Daan Hoogland <da...@gmail.com> wrote:

> If this fixes the CLVM issue in KVM environments, I want it in 4.4.

Not related to CLVM directly, just template/url stuff.

>
> Rohit, can you expand on this?

Not a major bugfix, can be even viewed as a feature to support pre-signed urls (urls that don’t end with an extention).

You may follow it here: https://issues.apache.org/jira/browse/CLOUDSTACK-5512

Pick on your own discretion, probably not critical for 4.4.1

Cheers.

>
> On Tue, Aug 12, 2014 at 11:40 PM, Min Chen <mi...@citrix.com> wrote:
>> Completely remove the checkFormat() will cause trouble for debugging
>> issues later if the template format is wrong. I would prefer fail fast
>> approach instead of spending hours later to deal with wrong file format
>> issues.
>>
>> Thanks
>> -min
>>
>> On 8/12/14 2:37 PM, "Rohit Yadav" <ro...@shapeblue.com> wrote:
>>
>>> Hey,
>>>
>>> On 12-Aug-2014, at 8:53 pm, Min Chen <mi...@citrix.com> wrote:
>>>
>>>> A better fix to address this issue has been committed to master:
>>>>
>>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=e3564
>>>> 65
>>>> 8befaa72cbe5fd510bea2a25b40f108f5. This should address CLOUDSTACK-6940,
>>>> CLOUDSTACK-7312, CLOUDSTACK-5512 altogether, same logic to handle
>>>> registerTemplate, registerIso and uploadVolume.
>>>
>>> Thanks for the fix Min, I saw that. I was hoping that we completely
>>> remove the checkFormat() method (do we need it, let’s get rid of it?).
>>> Lastly, Daan can comment if we want that in 4.4? It looks more like a
>>> bugfix though :)
>>>
>>> Cheers.
>>>
>>>>
>>>> Thanks
>>>> -min
>>>>
>>>> On 8/11/14 5:41 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>>>
>>>>> Yes, its important to fail fast and suggest the user that he/she didn't
>>>>> register the correct format URI.
>>>>> Also we shouldn’t have boiler plate code for checking the url format
>>>>> for
>>>>> register iso/template/volume. It should be one util method.
>>>>> But would you know all the valid formats a URL can be - or will you
>>>>> just
>>>>> check for say 'vhd' in the URL ?
>>>>>
>>>>> Thanks,
>>>>> -Nitin
>>>>>
>>>>> On 11/08/14 5:26 PM, "Min Chen" <mi...@citrix.com> wrote:
>>>>>
>>>>>> Yes. But on second thought, I think that purely removing template url
>>>>>> format checking is too brute force and aggressive, and will cause hard
>>>>>> time in debugging issues thrown by later orchestration flow that
>>>>>> relies
>>>>>> on
>>>>>> template/volume in specific file format. A better fix to address the
>>>>>> issue
>>>>>> raised in CLOUDSTACK-5512 is to fix the code to parse download or
>>>>>> upload
>>>>>> URI properly to get correct path. I will check in such a fix soon.
>>>>>>
>>>>>> Thanks
>>>>>> -min
>>>>>>
>>>>>> On 8/11/14 5:16 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>>>>>
>>>>>>> Hi Min,
>>>>>>>
>>>>>>> We need to do this for Upload Volume as well - right ?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -Nitin
>>>>>>>
>>>>>>> On 11/08/14 3:01 PM, "Min Chen" <mi...@citrix.com> wrote:
>>>>>>>
>>>>>>>> Just checked in a fix to remove similar format check for ISO for
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7312.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -min
>>>>>>>>
>>>>>>>> On 8/11/14 9:59 AM, "Rohit Yadav" <ro...@shapeblue.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks for fixing that Min!
>>>>>>>>>
>>>>>>>>> I personally did not check the current state of code other than
>>>>>>>>> discussing it with Marcus on the issue.
>>>>>>>>>
>>>>>>>>> Cheers.
>>>>>>>>>
>>>>>>>>> On 11-Aug-2014, at 6:54 pm, Min Chen <mi...@citrix.com> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, I have filed and fixed that bug a month ago.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> -min
>>>>>>>>>>
>>>>>>>>>> On 8/10/14 10:39 PM, "Harikrishna Patnala"
>>>>>>>>>> <ha...@citrix.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Rohit,
>>>>>>>>>>>
>>>>>>>>>>> This was already fixed for templates
>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-6940
>>>>>>>>>>>
>>>>>>>>>>> We have to do the same for registering ISOs as well.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Harikrishna
>>>>>>>>>>>
>>>>>>>>>>> On 08-Aug-2014, at 11:20 pm, Rohit Yadav
>>>>>>>>>>> <ro...@shapeblue.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> With reference to
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-5512
>>>>>>>>>>>> Marcus and I think we should remove the template URL format
>>>>>>>>>>>> checking
>>>>>>>>>>>> logic because:
>>>>>>>>>>>>
>>>>>>>>>>>> - It does not handle pre-signed URL (say something that does not
>>>>>>>>>>>> end
>>>>>>>>>>>> with .vhd etc, but has bunch of http params)
>>>>>>>>>>>> - One can game the system by say renaming any file to respective
>>>>>>>>>>>> format
>>>>>>>>>>>> - We dumb down, take whatever URL user gives and use the format
>>>>>>>>>>>> they
>>>>>>>>>>>> specify in their register template API call
>>>>>>>>>>>>
>>>>>>>>>>>> Marcus also notes that TemplateUtils utility would validate
>>>>>>>>>>>> selected
>>>>>>>>>>>> format.
>>>>>>>>>>>>
>>>>>>>>>>>> Please discuss if you¹ve any use-case that can get affected by
>>>>>>>>>>>> this?
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Rohit Yadav
>>>>>>>>>>>> Software Architect, ShapeBlue
>>>>>>>>>>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>>>>>>>>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Find out more about ShapeBlue and our range of CloudStack
>>>>>>>>>>>> related
>>>>>>>>>>>> services
>>>>>>>>>>>>
>>>>>>>>>>>> IaaS Cloud Design &
>>>>>>>>>>>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>>>>>>>>>> CSForge  rapid IaaS deployment
>>>>>>>>>>>> framework<http://shapeblue.com/csforge/>
>>>>>>>>>>>> CloudStack
>>>>>>>>>>>> Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>>>>>>>>>> CloudStack Infrastructure
>>>>>>>>>>>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>>>>>>>>>> CloudStack Bootcamp Training
>>>>>>>>>>>> Courses<http://shapeblue.com/cloudstack-training/>
>>>>>>>>>>>>
>>>>>>>>>>>> This email and any attachments to it may be confidential and are
>>>>>>>>>>>> intended solely for the use of the individual to whom it is
>>>>>>>>>>>> addressed.
>>>>>>>>>>>> Any views or opinions expressed are solely those of the author
>>>>>>>>>>>> and
>>>>>>>>>>>> do
>>>>>>>>>>>> not necessarily represent those of Shape Blue Ltd or related
>>>>>>>>>>>> companies.
>>>>>>>>>>>> If you are not the intended recipient of this email, you must
>>>>>>>>>>>> neither
>>>>>>>>>>>> take any action based upon its contents, nor copy or show it to
>>>>>>>>>>>> anyone.
>>>>>>>>>>>> Please contact the sender if you believe you have received this
>>>>>>>>>>>> email
>>>>>>>>>>>> in
>>>>>>>>>>>> error. Shape Blue Ltd is a company incorporated in England &
>>>>>>>>>>>> Wales.
>>>>>>>>>>>> ShapeBlue Services India LLP is a company incorporated in India
>>>>>>>>>>>> and
>>>>>>>>>>>> is
>>>>>>>>>>>> operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>>>>>>>>>> Consultoria Ltda is a company incorporated in Brasil and is
>>>>>>>>>>>> operated
>>>>>>>>>>>> under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
>>>>>>>>>>>> company
>>>>>>>>>>>> registered by The Republic of South Africa and is traded under
>>>>>>>>>>>> license
>>>>>>>>>>>> from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Rohit Yadav
>>>>>>>>> Software Architect, ShapeBlue
>>>>>>>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>>>>>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Find out more about ShapeBlue and our range of CloudStack related
>>>>>>>>> services
>>>>>>>>>
>>>>>>>>> IaaS Cloud Design &
>>>>>>>>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>>>>>>> CSForge  rapid IaaS deployment
>>>>>>>>> framework<http://shapeblue.com/csforge/>
>>>>>>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>>>>>>> CloudStack Infrastructure
>>>>>>>>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>>>>>>> CloudStack Bootcamp Training
>>>>>>>>> Courses<http://shapeblue.com/cloudstack-training/>
>>>>>>>>>
>>>>>>>>> This email and any attachments to it may be confidential and are
>>>>>>>>> intended
>>>>>>>>> solely for the use of the individual to whom it is addressed. Any
>>>>>>>>> views
>>>>>>>>> or opinions expressed are solely those of the author and do not
>>>>>>>>> necessarily represent those of Shape Blue Ltd or related
>>>>>>>>> companies. If
>>>>>>>>> you are not the intended recipient of this email, you must neither
>>>>>>>>> take
>>>>>>>>> any action based upon its contents, nor copy or show it to anyone.
>>>>>>>>> Please
>>>>>>>>> contact the sender if you believe you have received this email in
>>>>>>>>> error.
>>>>>>>>> Shape Blue Ltd is a company incorporated in England & Wales.
>>>>>>>>> ShapeBlue
>>>>>>>>> Services India LLP is a company incorporated in India and is
>>>>>>>>> operated
>>>>>>>>> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria
>>>>>>>>> Ltda
>>>>>>>>> is
>>>>>>>>> a company incorporated in Brasil and is operated under license from
>>>>>>>>> Shape
>>>>>>>>> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
>>>>>>>>> Republic
>>>>>>>>> of
>>>>>>>>> South Africa and is traded under license from Shape Blue Ltd.
>>>>>>>>> ShapeBlue
>>>>>>>>> is a registered trademark.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>
>>> IaaS Cloud Design &
>>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>> CSForge  rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>> CloudStack Infrastructure
>>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>> CloudStack Bootcamp Training
>>> Courses<http://shapeblue.com/cloudstack-training/>
>>>
>>> This email and any attachments to it may be confidential and are intended
>>> solely for the use of the individual to whom it is addressed. Any views
>>> or opinions expressed are solely those of the author and do not
>>> necessarily represent those of Shape Blue Ltd or related companies. If
>>> you are not the intended recipient of this email, you must neither take
>>> any action based upon its contents, nor copy or show it to anyone. Please
>>> contact the sender if you believe you have received this email in error.
>>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
>>> Services India LLP is a company incorporated in India and is operated
>>> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is
>>> a company incorporated in Brasil and is operated under license from Shape
>>> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
>>> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue
>>> is a registered trademark.
>>
>
>
>
> --
> Daan

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.