You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "James DeFelice (JIRA)" <ji...@apache.org> on 2018/10/24 18:30:00 UTC
[jira] [Commented] (MESOS-9352) Data in persistent volume deleted
accidentally when using Docker container and Persistent volume
[ https://issues.apache.org/jira/browse/MESOS-9352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662650#comment-16662650 ]
James DeFelice commented on MESOS-9352:
---------------------------------------
Seemingly related MESOS-9049, MESOS-8830, MESOS-2408
> Data in persistent volume deleted accidentally when using Docker container and Persistent volume
> ------------------------------------------------------------------------------------------------
>
> Key: MESOS-9352
> URL: https://issues.apache.org/jira/browse/MESOS-9352
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization, docker
> Affects Versions: 1.5.1, 1.5.2
> Environment: DCOS 1.11.6
> Mesos 1.5.2
> Reporter: David Ko
> Priority: Critical
> Labels: mesosphere, persistent-volumes
> Attachments: image-2018-10-24-22-20-51-059.png, image-2018-10-24-22-21-13-399.png
>
>
> Using docker image w/ persistent volume to start a service, it will cause data in persistent volume deleted accidentally when task killed and restarted, also old mount points not unmounted, even the service already deleted.
> *The expected result should be data in persistent volume kept until task deleted completely, also dangling mount points should be unmounted correctly.*
>
> *Step 1:* Use below JSON config to create a Mysql server using Docker image and Persistent Volume
> {code:javascript}
> {
> "env": {
> "MYSQL_USER": "wordpress",
> "MYSQL_PASSWORD": "secret",
> "MYSQL_ROOT_PASSWORD": "supersecret",
> "MYSQL_DATABASE": "wordpress"
> },
> "id": "/mysqlgc",
> "backoffFactor": 1.15,
> "backoffSeconds": 1,
> "constraints": [
> [
> "hostname",
> "IS",
> "172.27.12.216"
> ]
> ],
> "container": {
> "portMappings": [
> {
> "containerPort": 3306,
> "hostPort": 0,
> "protocol": "tcp",
> "servicePort": 10000
> }
> ],
> "type": "DOCKER",
> "volumes": [
> {
> "persistent": {
> "type": "root",
> "size": 1000,
> "constraints": []
> },
> "mode": "RW",
> "containerPath": "mysqldata"
> },
> {
> "containerPath": "/var/lib/mysql",
> "hostPath": "mysqldata",
> "mode": "RW"
> }
> ],
> "docker": {
> "image": "mysql",
> "forcePullImage": false,
> "privileged": false,
> "parameters": []
> }
> },
> "cpus": 1,
> "disk": 0,
> "instances": 1,
> "maxLaunchDelaySeconds": 3600,
> "mem": 512,
> "gpus": 0,
> "networks": [
> {
> "mode": "container/bridge"
> }
> ],
> "residency": {
> "relaunchEscalationTimeoutSeconds": 3600,
> "taskLostBehavior": "WAIT_FOREVER"
> },
> "requirePorts": false,
> "upgradeStrategy": {
> "maximumOverCapacity": 0,
> "minimumHealthCapacity": 0
> },
> "killSelection": "YOUNGEST_FIRST",
> "unreachableStrategy": "disabled",
> "healthChecks": [],
> "fetch": []
> }
> {code}
> *Step 2:* Kill mysqld process to force rescheduling new Mysql task, but found 2 mount points to the same persistent volume, it means old mount point did not be unmounted immediately.
> !image-2018-10-24-22-20-51-059.png!
> *Step 3:* After GC, data in persistent volume was deleted accidentally, but mysqld (Mesos task) still running
> !image-2018-10-24-22-21-13-399.png!
> *Step 4:* Delete Mysql service from Marathon, all mount points unable to unmount, even the service already deleted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)