You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "detaras (via GitHub)" <gi...@apache.org> on 2024/11/26 18:22:32 UTC
[I] Database encryption engine migration script fails [cloudstack]
detaras opened a new issue, #9979:
URL: https://github.com/apache/cloudstack/issues/9979
##### ISSUE TYPE
* Bug Report
##### COMPONENT NAME
Database encryption engine migration script
~~~
##### CLOUDSTACK VERSION
4.18.2.3
~~~
##### CONFIGURATION
Database with legacy cryptographic cipher (MD5/DES)
~~~
##### OS / ENVIRONMENT
Centos 7
~~~
##### SUMMARY
When migrating from the old database encryption scheme (MD5/DES 56-bit) to the new one (AES-GCM 256-bit) using the supplied script (/usr/bin/cloudstack-migrate-databases) from any of the managers; the migration process starts, but suddenly, when working on on the 'user_vm_deploy_as_is_details' table, we get an error and the process fails.
~~~
##### STEPS TO REPRODUCE
# systemctl stop cloudstack-management cloudstack-usage
# /usr/bin/cloudstack-migrate-databases -d oldDBKey -m oldMSKey -e newDBKey -n newMSKey -v V2
...
...
Data Migration failed. Reverting db.properties
~~~
##### EXPECTED RESULTS
The script should end without error. The database content is migrated using the new cipher and the configuration files are updated as needed.
~~~
##### ACTUAL RESULTS
The database and configuration files remains untouched, the migration script fails with the following error:
Unable to update user_vm_deploy_as_is_details values due to: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'vm_instance.id = 20850 AND template_deploy_as_is_details.name = 'property-timezo' at line 1
Error during data migration
com.cloud.utils.exception.CloudRuntimeException: Unable to update user_vm_deploy_as_is_details values
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.throwCloudRuntimeException(EncryptionSecretKeyChanger.java:794)
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.migrateTemplateDeployAsIsDetails(EncryptionSecretKeyChanger.java:663)
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.migrateData(EncryptionSecretKeyChanger.java:485)
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.migratePropertiesAndDatabase(EncryptionSecretKeyChanger.java:343)
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.main(EncryptionSecretKeyChanger.java:142)
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'vm_instance.id = 20850 AND template_deploy_as_is_details.name = 'property-timezo' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:1003)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
at com.cloud.utils.crypt.EncryptionSecretKeyChanger.migrateTemplateDeployAsIsDetails(EncryptionSecretKeyChanger.java:649)
... 3 more
Data Migration failed. Reverting db.properties
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache closed issue #9979: Database encryption engine migration script fails
URL: https://github.com/apache/cloudstack/issues/9979
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "detaras (via GitHub)" <gi...@apache.org>.
detaras commented on issue #9979:
URL: https://github.com/apache/cloudstack/issues/9979#issuecomment-2501691167
I have found a workaround for this issue.
I took a look at the code of the EncryptionSecretKeyChanger class (located at framework/db/src/main/java/com/cloud/utils/crypt/EncryptionSecretKeyChanger.java) and found an error in the query created at lines 656-659:
~~~
String sqlTemplateDeployAsIsDetails = "SELECT template_deploy_as_is_details.value " +
"FROM template_deploy_as_is_details JOIN vm_instance " +
"WHERE template_deploy_as_is_details.template_id = vm_instance.vm_template_id " +
"vm_instance.id = %s AND template_deploy_as_is_details.name = '%s' LIMIT 1";
~~~
If we run the created query in MySQL CLI we get an error and that's because the last line shoud have a connector at the beginning (I think an 'AND' is needed there). After adding that connector and building ACS again, the migration script ended successfully.
The modified code I used is this:
~~~
String sqlTemplateDeployAsIsDetails = "SELECT template_deploy_as_is_details.value " +
"FROM template_deploy_as_is_details JOIN vm_instance " +
"WHERE template_deploy_as_is_details.template_id = vm_instance.vm_template_id " +
"AND vm_instance.id = %s AND template_deploy_as_is_details.name = '%s' LIMIT 1";
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9979:
URL: https://github.com/apache/cloudstack/issues/9979#issuecomment-2501689319
@detaras
thanks for reporting the issue, I will have a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9979:
URL: https://github.com/apache/cloudstack/issues/9979#issuecomment-2516585466
fixed by #10033
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9979:
URL: https://github.com/apache/cloudstack/issues/9979#issuecomment-2501704512
> I have found a workaround for this issue. I took a look at the code of the EncryptionSecretKeyChanger class (located at framework/db/src/main/java/com/cloud/utils/crypt/EncryptionSecretKeyChanger.java) and found an error in the query created at lines 656-659:
>
> ```
> String sqlTemplateDeployAsIsDetails = "SELECT template_deploy_as_is_details.value " +
> "FROM template_deploy_as_is_details JOIN vm_instance " +
> "WHERE template_deploy_as_is_details.template_id = vm_instance.vm_template_id " +
> "vm_instance.id = %s AND template_deploy_as_is_details.name = '%s' LIMIT 1";
> ```
>
> If we run the created query in MySQL CLI we get an error and that's because the last line shoud have a connector at the beginning (I think an 'AND' is needed there). After adding that connector and building ACS again, the migration script ended successfully.
>
> The modified code I used is this:
>
> ```
> String sqlTemplateDeployAsIsDetails = "SELECT template_deploy_as_is_details.value " +
> "FROM template_deploy_as_is_details JOIN vm_instance " +
> "WHERE template_deploy_as_is_details.template_id = vm_instance.vm_template_id " +
> "AND vm_instance.id = %s AND template_deploy_as_is_details.name = '%s' LIMIT 1";
> ```
good finding @detaras
it should be my mistake.
can you create a pull request against 4.19 branch ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Database encryption engine migration script fails [cloudstack]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9979:
URL: https://github.com/apache/cloudstack/issues/9979#issuecomment-2501643379
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org