You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/10/28 01:27:14 UTC
[GitHub] [cloudstack] div8cn opened a new issue #4433: Failed to create VOLUME from snapshot
div8cn opened a new issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and master branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
snapshot
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on master branch.
-->
~~~
4.14.0.0
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network, advanced networking, etc. N/A otherwise
-->
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
CentOS 7.7 KVM QEMU 2.12.0,libvirt 4.5.0
Ceph 14.2.10
##### SUMMARY
<!-- Explain the problem/feature briefly -->
snapshot.backup.to.secondary=false
kvm.snapshot.enabled=true
When creating a new VOLUME through a VOLUME snapshot,
Only the first snapshot of each VOLUME is successful;
No other snapshots of the same VOLUME can create a new VOLUME
"Path" is missing from CopyCommand: "snapshots/2/5445/4e130b18-a495-4220-8c8d-118d186e961f"
CopyCommand:
{
"org.apache.cloudstack.storage.command.CopyCommand": {
"srcTO": {
"org.apache.cloudstack.storage.to.SnapshotObjectTO": {
"volume": {
"uuid": "8c39f2f5-9b18-494e-b990-969d9318eeff",
"volumeType": "ROOT",
"dataStore": {
"org.apache.cloudstack.storage.to.PrimaryDataStoreTO": {
"uuid": "049f30ad-d4ec-33cb-a0e3-e34df5f2f757",
"id": 2,
"poolType": "RBD",
"host": "10.226.15.3,10.226.15.4,10.226.15.5",
"path": "cloudstack",
"port": 6789,
"url": "RBD://10.226.15.3,10.226.15.4,10.226.15.5/cloudstack/?ROLE=Primary&STOREUUID=049f30ad-d4ec-33cb-a0e3-e34df5f2f757",
"isManaged": false
}
},
"name": "ROOT-4158",
"size": 107374182400,
"path": "8c39f2f5-9b18-494e-b990-969d9318eeff",
"volumeId": 5445,
"vmName": "i-2-4158-VM",
"accountId": 2,
"format": "QCOW2",
"provisioningType": "THIN",
"id": 5445,
"deviceId": 0,
"hypervisorType": "KVM",
"directDownload": false
},
"dataStore": {
"com.cloud.agent.api.to.NfsTO": {
"_url": "nfs://10.226.15.21/nfs/secstorage",
"_role": "Image"
}
},
"vmName": "i-2-4158-VM",
"name": "aaaaa2",
"hypervisorType": "KVM",
"id": 27345,
"quiescevm": false,
"physicalSize": 0
}
},
"destTO": {
"org.apache.cloudstack.storage.to.VolumeObjectTO": {
"uuid": "32cc7365-9bc3-49c5-87f1-871a96fb0bf7",
"volumeType": "DATADISK",
"dataStore": {
"org.apache.cloudstack.storage.to.PrimaryDataStoreTO": {
"uuid": "049f30ad-d4ec-33cb-a0e3-e34df5f2f757",
"id": 2,
"poolType": "RBD",
"host": "10.226.15.3,10.226.15.4,10.226.15.5",
"path": "cloudstack",
"port": 6789,
"url": "RBD://10.226.15.3,10.226.15.4,10.226.15.5/cloudstack/?ROLE=Primary&STOREUUID=049f30ad-d4ec-33cb-a0e3-e34df5f2f757",
"isManaged": false
}
},
"name": "aaaaa2-Volume",
"size": 107374182400,
"volumeId": 5447,
"accountId": 2,
"format": "QCOW2",
"provisioningType": "THIN",
"id": 5447,
"hypervisorType": "KVM",
"directDownload": false
}
},
"executeInSequence": false,
"options": {},
"options2": {},
"wait": 10800
}
}
Log:
2020-10-28 09:03:55,535 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (API-Job-Executor-1:ctx-d3f6d1cd job-127275 ctx-f465d613) (logid:91387467) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@2b441866), no need to delete from object in store ref table
2020-10-28 09:03:55,539 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-1:ctx-d3f6d1cd job-127275 ctx-f465d613) (logid:91387467) Failed to create volume from snapshot:java.lang.NullPointerException
at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createVolumeFromSnapshot(KVMStorageProcessor.java:1599)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:110)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:57)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:89)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1665)
at com.cloud.agent.Agent.processRequest(Agent.java:644)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1069)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
1. Create a VOLUME snapshot;
2. Create VOLUME from the volume snapshot (successful);
3. Create a VOLUME snapshot;
4. Create VOLUME from volume snapshot (failure);
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
1. Create a VOLUME snapshot;
2. Create VOLUME from the volume snapshot (successful);
3. Create a VOLUME snapshot;
4. Create VOLUME from volume snapshot (successful);
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
~~~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] slavkap commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
slavkap commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-847635997
@Spaceman1984, I did the testing just out of curiosity and did not direct my comment to you, and I didn't mean to argue with you. And yes, it works with NFS because when creating volume from a snapshot, it first makes a backup on secondary.
But with NFS or Ceph, you cannot create a template from a snapshot when the snapshot is created only on primary storage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-720331118
@nvazquez can you advise steps on fixing this to @Spaceman1984 ? Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-908967207
Fixed in https://github.com/apache/cloudstack/pull/5349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] slavkap commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
slavkap commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-897430667
@nvazquez, yes, I could work on a fix if nobody wants to or didn't start working on it. But to mention that I don't have much experience with Ceph, and maybe I will have some questions about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-740459022
I think the original bug was from a Ceph env @nvazquez
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-843820313
Unable to reproduce on latest master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
nvazquez commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-897039057
My tests have failed on NFS but could be related to an environmental issue as @Spaceman1984 could not reproduce. @slavkap thanks for testing on Ceph, could you work on this issue or planning to work on it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-847666232
All good @slavkap, thanks for the info.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-846921411
@slavkap my tests were with NFS. I was trying to reproduce what @nvazquez wrote.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] div8cn commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
div8cn commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-720453369
We have tested the same problem in 4.14.0.0.
Now we have changed the operating architecture:
1. We run keepalived in each management node and generate a VIP
2. All kvm agents connect to VIP
Now the cluster is running well
When the master management node is powered off, KVM AGENT can quickly restore the connection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
nvazquez commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-897634565
@slavkap no problem, just checking if there was some work on it already to see if it will be included on the milestone or not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd closed issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd closed issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Spaceman1984 commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
Spaceman1984 commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-843823373
I was able to create snapshots from volumes multiple levels deep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] slavkap commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
slavkap commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-846091166
Hi all, I was able to reproduce the issue on the latest master also.
Same configuration:
```
snapshot.backup.to.secondary=false
kvm.snapshot.enabled=true
primary storage - Ceph
```
If the snapshot is created bypassing secondary storage, maybe I'll say something stupid, but there is no implementation to create Volume from the snapshot. In the code, the image store of the source should be an NFS instance (but the snapshot does not exist on secondary)
Also, there is another problem with the mgmt code. SnapshotDataFactoryImpl.getSnapshot should not search in snapshot_store_ref table by volume id and store role because, in some cases, it gets the wrong record. For example, if I want to create volume from a snapshot with id=11 (created only on primary), it gets the snapshot with id 5 because it looks for store_role = Image. Maybe there is a need to be added and snapshot.id also.
```
SELECT id,snapshot_id, store_role FROM cloud.snapshot_store_ref where volume_id=138;
+----+-------------+------------+
| id | snapshot_id | store_role |
+----+-------------+------------+
| 4 | 9 | Primary |
| 5 | 9 | Image |
| 7 | 11 | Primary |
+----+-------------+------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
nvazquez commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-897655076
@slavkap assigned you on this as discussed offline, many thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] div8cn removed a comment on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
div8cn removed a comment on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-720453369
We have tested the same problem in 4.14.0.0.
Now we have changed the operating architecture:
1. We run keepalived in each management node and generate a VIP
2. All kvm agents connect to VIP
Now the cluster is running well
When the master management node is powered off, KVM AGENT can quickly restore the connection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
nvazquez commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-718026184
I was able to reproduce on NFS storage pool, and the same setting values.
```
2020-10-28 15:35:59,214 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) getCommandHostDelegation: class org.apache.cloudstack.storage.command.CopyCommand
2020-10-28 15:35:59,215 DEBUG [c.c.a.t.Request] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Seq 1-6719370644036780103: Sending { Cmd , MgmtId: 32988956015047, via: 1(ref-trl-1821-k-M7-nicolas-vazquez-kvm1), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"88a6ce2c-de4c-475f-827e-696f0f0bdeb2","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9715ad47-244e-3d24-a382-7ad33a3dc0f1","id":1,"poolType":"NetworkFilesystem","host":"10.10.0.16","path":"/acs/primary/ref-trl-1821-k-M7-nicolas-vazquez/ref-trl-1821-k-M7-nicolas-vazquez-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.10.0.16/acs/primary/ref-trl-1821-k-M7-nicolas-vazquez/ref-trl-1821-k-M7-nicolas-vazquez-kvm-pri1/?ROLE=Primary&STOREUUID=9715ad47-244e-3d24-a382-7ad33a3dc0f1","isManaged":false}},"name":"ROOT-4","size":8589934592,"path":"88a6ce2c
-de4c-475f-827e-696f0f0bdeb2","volumeId":4,"vmName":"i-2-4-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":4,"deviceId":0,"hypervisorType":"KVM","directDownload":false},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"NFS://10.10.0.16/acs/secondary/ref-trl-1821-k-M7-nicolas-vazquez/ref-trl-1821-k-M7-nicolas-vazquez-sec1","_role":"Image"}},"vmName":"i-2-4-VM","name":"snap2","hypervisorType":"KVM","id":2,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"81832bc9-1c46-4db8-968d-198064a4cec3","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9715ad47-244e-3d24-a382-7ad33a3dc0f1","id":1,"poolType":"NetworkFilesystem","host":"10.10.0.16","path":"/acs/primary/ref-trl-1821-k-M7-nicolas-vazquez/ref-trl-1821-k-M7-nicolas-vazquez-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.10.0.16/acs/primary/ref-trl-1821-k-M7-nicolas-vazquez/ref-trl-1821-k-M7-nicolas-vazquez-
kvm-pri1/?ROLE=Primary&STOREUUID=9715ad47-244e-3d24-a382-7ad33a3dc0f1","isManaged":false}},"name":"diskfromsnap2","size":21474836480,"volumeId":7,"accountId":2,"format":"QCOW2","provisioningType":"THIN","id":7,"hypervisorType":"KVM","directDownload":false}},"executeInSequence":false,"options":{},"options2":{},"wait":10800}}] }
2020-10-28 15:35:59,224 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null) (logid:) Seq 1-6719370644036780103: Processing: { Ans: , MgmtId: 32988956015047, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createVolumeFromSnapshot(KVMStorageProcessor.java:1599)\n\tat com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:110)\n\tat com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:57)\n\tat com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)\n\tat com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)\n\tat com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWra
pper.execute(LibvirtRequestWrapper.java:78)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1637)\n\tat com.cloud.agent.Agent.processRequest(Agent.java:644)\n\tat com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1057)\n\tat com.cloud.utils.nio.Task.call(Task.java:83)\n\tat com.cloud.utils.nio.Task.call(Task.java:29)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n","wait":0}}] }
2020-10-28 15:35:59,225 DEBUG [c.c.a.t.Request] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Seq 1-6719370644036780103: Received: { Ans: , MgmtId: 32988956015047, via: 1(ref-trl-1821-k-M7-nicolas-vazquez-kvm1), Ver: v1, Flags: 10, { Answer } }
2020-10-28 15:35:59,232 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@6689d2a6), no need to delete from object in store ref table
2020-10-28 15:35:59,237 DEBUG [o.a.c.e.o.VolumeOrchestrator] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Failed to create volume from snapshot:java.lang.NullPointerException
at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createVolumeFromSnapshot(KVMStorageProcessor.java:1599)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:110)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:57)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1637)
at com.cloud.agent.Agent.processRequest(Agent.java:644)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1057)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-10-28 15:35:59,248 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1
2020-10-28 15:35:59,255 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-32:ctx-ee634337 job-44 ctx-540c9792) (logid:8bbe9b7f) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = 21474836480
2020-10-28 15:35:59,269 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-32:ctx-ee634337 job-44) (logid:8bbe9b7f) Unexpected exception while executing org.apache.cloudstack.api.command.admin.volume.CreateVolumeCmdByAdmin
com.cloud.utils.exception.CloudRuntimeException: Failed to create volume: 7
at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:837)
at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:184)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy212.createVolume(Unknown Source)
at org.apache.cloudstack.api.command.user.volume.CreateVolumeCmd.execute(CreateVolumeCmd.java:225)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:603)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:551)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to create volume from snapshot:java.lang.NullPointerException
at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createVolumeFromSnapshot(KVMStorageProcessor.java:1599)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:110)
at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:57)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1637)
at com.cloud.agent.Agent.processRequest(Agent.java:644)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1057)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.createVolumeFromSnapshot(VolumeOrchestrator.java:432)
at com.cloud.storage.VolumeApiServiceImpl.createVolumeFromSnapshot(VolumeApiServiceImpl.java:859)
at com.cloud.storage.VolumeApiServiceImpl.createVolume(VolumeApiServiceImpl.java:809)
... 31 more
2020-10-28 15:35:59,272 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-32:ctx-ee634337 job-44) (logid:8bbe9b7f) Complete async job-44, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to create volume: 7"}
```
The `KVMStorageProcessor` is expecting a path but its null.
DB:
I've created 2 snapshots, both of them have null path but the first worked and second failed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] andrijapanicsb commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-900060645
Keeping volume snaps on primary storage works ONLY for CEPH (and accidentally for XenServer), last time I tested (in 4.14) - the code needs more robustness, and it's better to check with @GabrielBrascher or @wido what is the expected "acs storage" feature set when keeping the volume snap on Primary Storage (i.e. not copying it to Secondary Storage, which is the default setup for volume snaps)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd commented on issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433#issuecomment-908967207
Fixed in https://github.com/apache/cloudstack/pull/5349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd closed issue #4433: Failed to create VOLUME from snapshot
Posted by GitBox <gi...@apache.org>.
rhtyd closed issue #4433:
URL: https://github.com/apache/cloudstack/issues/4433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org