[GitHub] [accumulo-testing] dlmarion commented on pull request #252: Only install & enable Docker on manager node

[GitBox <gi...@apache.org>]

dlmarion commented on PR #252:
URL: https://github.com/apache/accumulo-testing/pull/252#issuecomment-1293714483

   The reason for this change is that `cloud-init` would intermittently fail with:
   ```
   (remote-exec): ('package-update-upgrade-install', ProcessExecutionError(u"Unexpected error while running command.\nCommand: ['yum', '-t', '-y', 'makecache']\nExit code: 1\nReason: -\nStdout: -\nStderr: -",))
   ```
   
   It would routinely take 3 or 4 attempts to create a cluster. The `cloud-init` log file contained:
   ```
   2022-10-27 11:53:30,436 - helpers.py[DEBUG]: Running update-sources using lock (<FileLock using file '/var/lib/cloud/instances/i-0ab76c54cf53177d8/sem/update_sources'>)
   2022-10-27 11:53:30,437 - rhel.py[DEBUG]: Using YUM for package management
   2022-10-27 11:53:30,437 - util.py[DEBUG]: Running command ['yum', '-t', '-y', 'makecache'] with allowed return codes [0] (shell=False, capture=False)
   2022-10-27 11:53:31,964 - util.py[WARNING]: Package update failed
   2022-10-27 11:53:31,965 - util.py[DEBUG]: Package update failed
   Traceback (most recent call last):
     File "/usr/lib/python2.7/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 85, in handle
       cloud.distro.update_package_sources()
     File "/usr/lib/python2.7/site-packages/cloudinit/distros/rhel.py", line 180, in update_package_sources
       ["makecache"], freq=PER_INSTANCE)
     File "/usr/lib/python2.7/site-packages/cloudinit/helpers.py", line 187, in run
       results = functor(*args)
     File "/usr/lib/python2.7/site-packages/cloudinit/distros/rhel.py", line 176, in package_command
       util.subp(cmd, capture=False)
     File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 2084, in subp
       cmd=args)
   ProcessExecutionError: Unexpected error while running command.
   Command: ['yum', '-t', '-y', 'makecache']
   Exit code: 1
   Reason: -
   Stdout: -
   Stderr: -
   ```
   
   And `/var/log/messages` contained:
   ```
   cloud-init: Error making cache directory: /var/cache/yum/x86_64/7/docker error was: [Errno 17] File exists: '/var/cache/yum/x86_64/7/docker'
   ```
   
   From what I could tell `cloud-init` was running at the same time as a `salt-minion` in my environment. I think the `salt-minion` (or the AMI) is conflicting with `cloud-init` sometimes, probably a race condition. This change installs the Docker repo on CentOS, but disables it, such that the `yum makecache` command that is run by the `cloud-init` process does not try to create the directory. Docker is only needed on the `manager` node, so I then install and enable it in the initialize accumulo script before the Docker containers are started.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org