You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by da...@apache.org on 2020/01/09 23:48:17 UTC
[couchdb-infra-cm] branch main updated (76f6e05 -> c26a384)
This is an automated email from the ASF dual-hosted git repository.
davisp pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb-infra-cm.git.
from 76f6e05 Configure TCP Keep-Alive parameters
new 109bf5c Only run docker tasks on x86_64 hosts
new c26a384 Make Ansible got faster
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
ansible.cfg | 4 +-
roles/ci_agent/tasks/{main.yml => docker.yml} | 67 +++----------------------
roles/ci_agent/tasks/main.yml | 56 +++------------------
ssh.cfg | 72 ++++++++++++++++++---------
tools/gen-config | 17 ++++---
5 files changed, 74 insertions(+), 142 deletions(-)
copy roles/ci_agent/tasks/{main.yml => docker.yml} (50%)
[couchdb-infra-cm] 01/02: Only run docker tasks on x86_64 hosts
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davisp pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb-infra-cm.git
commit 109bf5c0a8adb8294ab35a16bd065a9acf7a46b4
Author: Paul J. Davis <pa...@gmail.com>
AuthorDate: Thu Jan 9 17:16:30 2020 -0600
Only run docker tasks on x86_64 hosts
---
roles/ci_agent/tasks/{main.yml => docker.yml} | 67 +++------------------------
roles/ci_agent/tasks/main.yml | 56 +++-------------------
2 files changed, 13 insertions(+), 110 deletions(-)
diff --git a/roles/ci_agent/tasks/main.yml b/roles/ci_agent/tasks/docker.yml
similarity index 50%
copy from roles/ci_agent/tasks/main.yml
copy to roles/ci_agent/tasks/docker.yml
index 48064f2..a63f71f 100644
--- a/roles/ci_agent/tasks/main.yml
+++ b/roles/ci_agent/tasks/docker.yml
@@ -1,12 +1,3 @@
-- name: Configure TCP Keep-Alive
- copy:
- src: 50-ci-agent.sysctl.conf
- dest: /etc/sysctl.d/50-ci-agent.conf
- mode: 0644
-
-- name: Load TCP Keep-Alive configuration
- script: /sbin/sysctl --load /etc/sysctl.d/50-ci-agent.conf
-
- name: Install Docker gpg key for Apt
apt_key:
url: https://download.docker.com/linux/debian/gpg
@@ -37,29 +28,6 @@
- qemu
- qemu-user-static
-- name: Install Java 8
- apt:
- name: "{{ packages }}"
- state: latest
- vars:
- packages:
- - openjdk-8-jre-headless
-
-- name: Add group jenkins
- group:
- name: jenkins
- gid: 910
-
-- name: Add user jenkins
- user:
- name: jenkins
- uid: 910
- groups:
- - jenkins
- - docker
- state: present
- shell: /bin/bash
-
- name: Install kill-old-docker.sh
copy:
src: kill-old-docker.sh
@@ -78,32 +46,9 @@
hour: '19'
job: /usr/bin/docker system prune -a -f --filter "until=72h"
-- name: Create Jenkins runit service directory
- file:
- path: /etc/sv/jenkins
- state: directory
-
-- name: Create Jenkins runit log directory
- file:
- path: /etc/sv/jenkins/log/main
- state: directory
-
-- name : Create Jenkins runit run script
- template:
- src: runit-main.j2
- dest: /etc/sv/jenkins/run
- mode: 0755
-
-- name: Create Jenkins runit logs run script
- copy:
- src: runit-logs
- dest: /etc/sv/jenkins/log/run
- mode: 0755
-
-- name: Enable Jenkins runit service
- file:
- src: /etc/sv/jenkins
- dest: /etc/service/jenkins
- state: link
-
-
+- name: Add Jenkins user to docker group
+ user:
+ name: jenkins
+ groups:
+ - docker
+ append: yes
\ No newline at end of file
diff --git a/roles/ci_agent/tasks/main.yml b/roles/ci_agent/tasks/main.yml
index 48064f2..2f97ab8 100644
--- a/roles/ci_agent/tasks/main.yml
+++ b/roles/ci_agent/tasks/main.yml
@@ -3,39 +3,11 @@
src: 50-ci-agent.sysctl.conf
dest: /etc/sysctl.d/50-ci-agent.conf
mode: 0644
+ register: ciagentsysctl
- name: Load TCP Keep-Alive configuration
- script: /sbin/sysctl --load /etc/sysctl.d/50-ci-agent.conf
-
-- name: Install Docker gpg key for Apt
- apt_key:
- url: https://download.docker.com/linux/debian/gpg
- state: present
-
-- name: Setup Docker Apt repository
- apt_repository:
- repo: deb https://download.docker.com/linux/debian {{ ansible_distribution_release }} stable
- filename: docker
- state: present
-
-- name: Install Docker Packages
- apt:
- name: "{{ packages }}"
- vars:
- packages:
- - containerd.io
- - docker-ce
- - docker-ce-cli
-
-- name: Install multi-architecture support for Docker
- apt:
- name: "{{ packages }}"
- state: latest
- vars:
- packages:
- - binfmt-support
- - qemu
- - qemu-user-static
+ shell: /sbin/sysctl --load /etc/sysctl.d/50-ci-agent.conf
+ when: ciagentsysctl.changed
- name: Install Java 8
apt:
@@ -56,27 +28,13 @@
uid: 910
groups:
- jenkins
- - docker
state: present
shell: /bin/bash
-- name: Install kill-old-docker.sh
- copy:
- src: kill-old-docker.sh
- dest: /usr/local/bin/kill-old-docker.sh
- mode: 0755
-
-- name: Add kill-old-docker.sh cron entry
- cron:
- name: Kill old docker containers
- hour: '19'
- job: /usr/local/bin/kill-old-docker.sh
-
-- name: Add docker prune cron entry
- cron:
- name: Docker prune
- hour: '19'
- job: /usr/bin/docker system prune -a -f --filter "until=72h"
+- name: Setup Docker on x86_64 hosts
+ include_tasks: docker.yml
+ when:
+ - ansible_facts['architecture'] == 'x86_64'
- name: Create Jenkins runit service directory
file:
[couchdb-infra-cm] 02/02: Make Ansible got faster
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davisp pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb-infra-cm.git
commit c26a384e0194426c3161cd888cfea70bc52b2dc1
Author: Paul J. Davis <pa...@gmail.com>
AuthorDate: Thu Jan 9 17:47:46 2020 -0600
Make Ansible got faster
Still not what I would call fast though.
---
ansible.cfg | 4 +++-
ssh.cfg | 72 +++++++++++++++++++++++++++++++++++++-------------------
tools/gen-config | 17 +++++++------
3 files changed, 61 insertions(+), 32 deletions(-)
diff --git a/ansible.cfg b/ansible.cfg
index d5122a1..de128bb 100644
--- a/ansible.cfg
+++ b/ansible.cfg
@@ -1,7 +1,9 @@
[defaults]
inventory = ./production
vault_password_file = ~/.couchdb-ansible-vault
+forks = 10
[ssh_connection]
ssh_args = -F ./ssh.cfg
-control_path = ~/.ssh/ansible-%%r@%%h:%%p
+control_path = /tmp/ansible-%%r@%%h:%%p
+pipelining = True
\ No newline at end of file
diff --git a/ssh.cfg b/ssh.cfg
index 95da36a..02ab8fe 100644
--- a/ssh.cfg
+++ b/ssh.cfg
@@ -4,54 +4,78 @@ Host couchdb-bastion-x86-64-debian-dal-1-1
ForwardAgent yes
StrictHostKeyChecking no
ControlMaster auto
- ControlPath ~/.ssh/ansible-%r@%h:%p
+ ControlPath /tmp/ansible-%r@%h:%p
ControlPersist 30m
-Host couchdb-worker-x86-64-debian-dal-1-08
- Hostname 10.240.0.16
+Host couchdb-worker-x86-64-debian-dal-1-01
+ Hostname 10.240.0.4
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
-Host couchdb-worker-x86-64-debian-dal-1-05
- Hostname 10.240.0.10
+Host couchdb-worker-x86-64-debian-dal-1-02
+ Hostname 10.240.0.5
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
+
+Host couchdb-worker-x86-64-debian-dal-1-03
+ Hostname 10.240.0.6
+ User root
+ StrictHostKeyChecking no
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
Host couchdb-worker-x86-64-debian-dal-1-04
Hostname 10.240.0.9
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
-Host couchdb-worker-x86-64-debian-dal-1-07
- Hostname 10.240.0.15
+Host couchdb-worker-x86-64-debian-dal-1-05
+ Hostname 10.240.0.10
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
Host couchdb-worker-x86-64-debian-dal-1-06
Hostname 10.240.0.14
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
-
-Host couchdb-worker-x86-64-debian-dal-1-01
- Hostname 10.240.0.4
- User root
- StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
-Host couchdb-worker-x86-64-debian-dal-1-03
- Hostname 10.240.0.6
+Host couchdb-worker-x86-64-debian-dal-1-07
+ Hostname 10.240.0.15
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
-Host couchdb-worker-x86-64-debian-dal-1-02
- Hostname 10.240.0.5
+Host couchdb-worker-x86-64-debian-dal-1-08
+ Hostname 10.240.0.16
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@169.48.153.153
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@169.48.153.153
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
diff --git a/tools/gen-config b/tools/gen-config
index 1f39bd9..ffc59d3 100755
--- a/tools/gen-config
+++ b/tools/gen-config
@@ -175,11 +175,11 @@ def assign_bastions(bastions, ci_agents):
def write_inventory(fname, bastions, ci_agents):
inventory = {"all": {
"children": {
- "ci_agents": {
- "hosts": ci_agents
- },
"bastions": {
"hosts": bastions
+ },
+ "ci_agents": {
+ "hosts": ci_agents
}
}
}}
@@ -196,7 +196,7 @@ def write_ssh_cfg(filename, bastions, ci_agents):
ForwardAgent yes
StrictHostKeyChecking no
ControlMaster auto
- ControlPath ~/.ssh/ansible-%r@%h:%p
+ ControlPath /tmp/ansible-%r@%h:%p
ControlPersist 30m
""")
@@ -205,18 +205,21 @@ def write_ssh_cfg(filename, bastions, ci_agents):
Hostname {ip_addr}
User root
StrictHostKeyChecking no
- ProxyCommand /usr/bin/ssh -W %h:%p -q root@{bastion_ip}
+ ProxyCommand /usr/bin/ssh -F ./ssh.cfg -W %h:%p -q root@{bastion_ip}
+ ControlMaster auto
+ ControlPath /tmp/ansible-%r@%h:%p
+ ControlPersist 30m
""")
with open(filename, "w") as handle:
- for host, info in bastions.items():
+ for host, info in sorted(bastions.items()):
args = {
"host": host,
"ip_addr": info["ip_addrs"]["public"]
}
entry = bastion_tmpl.format(**args)
handle.write(entry)
- for host, info in ci_agents.items():
+ for host, info in sorted(ci_agents.items()):
args = {
"host": host,
"ip_addr": info["ip_addrs"]["private"],