You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "fabiomatavelli-f3 (via GitHub)" <gi...@apache.org> on 2024/02/16 11:08:22 UTC
[I] Trying to register a template with a long URL results in error [cloudstack]
fabiomatavelli-f3 opened a new issue, #8669:
URL: https://github.com/apache/cloudstack/issues/8669
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
UI, API
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main branch.
-->
~~~
4.18.1.0
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network, advanced networking, etc. N/A otherwise
-->
N/A
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
Linux Ubuntu 22.04
##### SUMMARY
<!-- Explain the problem/feature briefly -->
Trying to register a template with an S3 signed url (or any other really long URL) fails.
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
In the Cloudstack UI:
1. Go to `Images > Templates`
2. Try to register a template from URL
3. Use a long URL for the template (around 400~500 characters)
4. Cloudstack will fail:
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
Cloudstack to register the template without any problem.
##### ACTUAL RESULTS
<!-- What actually happened? -->
Cloudstack fails with the following error:
<!-- Paste verbatim command output between quotes below -->
~~~
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: INSERT INTO vm_template (vm_template.id, vm_template.format, vm_template.unique_name, vm_template.name, vm_template.public, vm_template.featured, vm_template.typ
e, vm_template.url, vm_template.hvm, vm_template.bits, vm_template.created, vm_template.account_id, vm_template.checksum, vm_template.display_text, vm_template.enable_password, vm_template.guest_os_id, vm_template.bootable, vm_template.prepopulate, vm_template.cross_zon
es, vm_template.hypervisor_type, vm_template.extractable, vm_template.source_template_id, vm_template.state, vm_template.template_tag, vm_template.uuid, vm_template.sort_key, vm_template.enable_sshkey, vm_template.size, vm_template.update_count, vm_template.updated, vm_
template.dynamically_scalable, vm_template.direct_download, vm_template.parent_template_id, vm_template.deploy_as_is, vm_template.user_data_id, vm_template.user_data_link_policy) VALUES (19344, 'QCOW2', x'31393334342D322D30333162623330652D326166332D336331632D396538362D6
13963393264313235373930', x'7562756E7475323030342D766D2D6B38732D32303234303231353131353434322D333332633961372D666D61746176656C6C692D636F6D70726573732D766D2D696D616765', 0, 0, 'USER', x'68747470733A2F2F6275636B65742E767063652D30333364633236656634316636373565302D396975307
2716A302E73332E65752D776573742D322E767063652E616D617A6F6E6177732E636F6D2F666F726D332D696D6167652D7061636B65722D65752D776573742D322F6F75747075742F766D2F666D61746176656C6C692D636F6D70726573732D766D2D696D6167652F7562756E7475323030342D766D2D6B38732D3230323430323135313135343
4322D333332633961372D666D61746176656C6C692D636F6D70726573732D766D2D696D6167652E71636F77322E677A3F582D416D7A2D416C676F726974686D3D415753342D484D41432D53484132353626582D416D7A2D43726564656E7469616C3D414B49415547514234325257485545504C4C4945253246323032343032313525324665752
D776573742D322532467333253246617773345F7265717565737426582D416D7A2D446174653D3230323430323135543138303332325A26582D416D7A2D457870697265733D39303026582D416D7A2D5369676E6564486561646572733D686F737426582D416D7A2D5369676E61747572653D35643039643562646562663738366536323435626
339303836613938376331613364613332316462313866313266326638303166656463316164633933643233', 0, 64, '2024-02-15 18:04:08', 2, null, x'7562756E7475323030342D766D2D6B38732D32303234303231353131353434322D333332633961372D666D61746176656C6C692D636F6D70726573732D766D2D696D616765'
, 0, 335, 1, 0, 1, 'KVM', 1, null, 'Active', null, x'31356530303839652D353735342D346361322D393264652D323666396233396634623963', 0, 0, null, 0, null, 0, 0, null, 0, null, null)
at com.cloud.utils.db.GenericDaoBase.persist(GenericDaoBase.java:1488)
at com.cloud.storage.dao.VMTemplateDaoImpl.addTemplateToZone(VMTemplateDaoImpl.java:467)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy108.addTemplateToZone(Unknown Source)
at com.cloud.template.TemplateAdapterBase.persistTemplate(TemplateAdapterBase.java:417)
at com.cloud.template.HypervisorTemplateAdapter.create(HypervisorTemplateAdapter.java:255)
at com.cloud.template.TemplateManagerImpl.registerTemplate(TemplateManagerImpl.java:355)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy222.registerTemplate(Unknown Source)
at org.apache.cloudstack.api.command.user.template.RegisterTemplateCmd.execute(RegisterTemplateCmd.java:315)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:163)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:782)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:603)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:347)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:154)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:151)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:105)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:645)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Data too long for column 'url' at row 1
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:104)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1092)
... 86 more
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-1949259798
> yeah @fabiomatavelli-f3 , I got the problem and the fix is easy but the question is in which version we could fix this. a dirty hack for you would just be `ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024)`, but you will possibly run into issues on upgrades, but this is the statement we would put in any of the upcoming upgrade files.
@DaanHoogland
the SQL can be executed several times. for example
```
mysql> ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024) DEFAULT NULL COMMENT 'the url where the template exists externally';
Query OK, 0 rows affected (0.10 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> mysql> ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024) DEFAULT NUENT 'the url where the template exists externally';
Query OK, 0 rows affected (0.14 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024) DEFAULT NULL COMMENT 'the url where the template exists externally';
Query OK, 0 rows affected (0.14 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024) DEFAULT NULL COMMENT 'the url where the template exists externally';
Query OK, 0 rows affected (0.10 sec)
Records: 0 Duplicates: 0 Warnings: 0
```
I think it can be put into 4.19.0->4.19.1 upgrade
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "fabiomatavelli-f3 (via GitHub)" <gi...@apache.org>.
fabiomatavelli-f3 commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-1948539004
indeed @DaanHoogland , I think even 1024 should be enough. Problem with S3 signed URLs is that they have the tokens/signature in the URL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-1948525108
so would changing this to 2048 be enough? an url should not be a blob or such, but some beyond 255 seems reasonable.
@JoaoJandre @sureshanaparti and all others, as this is a 4.18.1 bug (but a DB change), what version do you think we need to implement this for .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-1948573793
yeah @fabiomatavelli-f3 , I got the problem and the fix is easy but the question is in which version we could fix this. a dirty hack for you would just be `ALTER TABLE cloud.vm_template MODIFY COLUMN url VARCHAR(1024)`, but you will possibly run into issues on upgrades, but this is the statement we would put in any of the upcoming upgrade files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-2071622461
Closing this, as the fix PR got merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8669:
URL: https://github.com/apache/cloudstack/issues/8669#issuecomment-1948193009
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Trying to register a template with a long URL results in error [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti closed issue #8669: Trying to register a template with a long URL results in error
URL: https://github.com/apache/cloudstack/issues/8669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org