You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by bh...@apache.org on 2017/12/04 05:15:47 UTC
[cloudstack] branch debian9-systemvmtemplate updated (f590e65 ->
cc3113a)
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a change to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git.
discard f590e65 CLOUDSTACK-10013: Make the generated VR/json files unique (ports #1470)
new 584e69d CLOUDSTACK-10013: Make the generated VR/json files unique (ports #1470)
new 11031ac password server improvements and postinit reintroduction
new 32fb7e4 fix typo
new cc3113a refactor crufty configuration code
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f590e65)
\
N -- N -- N refs/heads/debian9-systemvmtemplate (cc3113a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../kvm/resource/LibvirtComputingResource.java | 11 +-
.../patches/debian/etc/init.d/cloud-early-config | 6 +-
.../patches/debian/etc/init.d/cloud-passwd-srvr | 118 ---------------------
systemvm/patches/{vpn => debian}/etc/ipsec.conf | 0
.../patches/{vpn => debian}/etc/ipsec.d/l2tp.conf | 0
systemvm/patches/{vpn => debian}/etc/ipsec.secrets | 0
.../patches/{vpn => debian}/etc/ppp/options.xl2tpd | 0
systemvm/patches/debian/etc/rc.local | 9 --
.../systemd/system/cloud-password-server@.service | 11 ++
.../etc/systemd/system/cloud-postinit.service | 13 +++
.../debian/etc/systemd/system/cloud.service | 13 +++
.../patches/{vpn => debian}/etc/xl2tpd/xl2tpd.conf | 0
systemvm/patches/debian/opt/cloud/bin/configure.py | 99 ++++++++---------
.../patches/debian/opt/cloud/bin/cs/CsAddress.py | 8 +-
systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py | 14 +--
.../patches/debian/opt/cloud/bin/cs/CsRedundant.py | 5 +-
.../patches/debian/opt/cloud/bin/setup/common.sh | 6 +-
.../debian/opt/cloud/bin/setup/consoleproxy.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/elbvm.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/ilbvm.sh | 2 +-
.../debian/opt/cloud/bin/setup/patchsystemvm.sh | 3 +-
.../cloud/bin/setup/{default.sh => postinit.sh} | 17 ++-
.../debian/opt/cloud/bin/setup/secstorage.sh | 2 +-
.../patches/debian/opt/cloud/bin/update_config.py | 19 ++--
.../{vpn => debian}/opt/cloud/bin/vpn_l2tp.sh | 0
systemvm/pom.xml | 3 -
.../configure_systemvm_services.sh | 36 ++-----
tools/appliance/shar_cloud_scripts.sh | 7 +-
29 files changed, 138 insertions(+), 270 deletions(-)
delete mode 100755 systemvm/patches/debian/etc/init.d/cloud-passwd-srvr
rename systemvm/patches/{vpn => debian}/etc/ipsec.conf (100%)
rename systemvm/patches/{vpn => debian}/etc/ipsec.d/l2tp.conf (100%)
rename systemvm/patches/{vpn => debian}/etc/ipsec.secrets (100%)
rename systemvm/patches/{vpn => debian}/etc/ppp/options.xl2tpd (100%)
create mode 100644 systemvm/patches/debian/etc/systemd/system/cloud-password-server@.service
create mode 100644 systemvm/patches/debian/etc/systemd/system/cloud-postinit.service
create mode 100644 systemvm/patches/debian/etc/systemd/system/cloud.service
rename systemvm/patches/{vpn => debian}/etc/xl2tpd/xl2tpd.conf (100%)
copy systemvm/patches/debian/opt/cloud/bin/setup/{default.sh => postinit.sh} (76%)
rename systemvm/patches/{vpn => debian}/opt/cloud/bin/vpn_l2tp.sh (100%)
--
To stop receiving notification emails like this one, please contact
['"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>'].
[cloudstack] 03/04: fix typo
Posted by bh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git
commit 32fb7e4ce903a23b1775a66639bb81d3db4959d0
Author: Rohit Yadav <ro...@shapeblue.com>
AuthorDate: Sun Dec 3 23:28:53 2017 +0530
fix typo
Signed-off-by: Rohit Yadav <ro...@shapeblue.com>
---
.../definitions/systemvmtemplate/configure_systemvm_services.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh b/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
index b4aaec9..51ffc86 100644
--- a/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
+++ b/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
@@ -36,7 +36,7 @@ function install_cloud_scripts() {
rsync -av ./cloud_scripts/ /
chmod +x /opt/cloud/bin/* \
/root/{clearUsageRules.sh,reconfigLB.sh,monitorServices.py} \
- /etc/init.d/{cloud-early-config} \
+ /etc/init.d/cloud-early-config \
/etc/profile.d/cloud.sh
cat > /etc/systemd/system/cloud-early-config.service << EOF
--
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>.
[cloudstack] 02/04: password server improvements and postinit
reintroduction
Posted by bh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git
commit 11031acfbc64ad4c40a40f5dd1806c718a9ff6eb
Author: Rohit Yadav <ro...@shapeblue.com>
AuthorDate: Sun Dec 3 22:32:13 2017 +0530
password server improvements and postinit reintroduction
merge vpn to debian/
disable rvr setup for now
Signed-off-by: Rohit Yadav <ro...@shapeblue.com>
---
.../kvm/resource/LibvirtComputingResource.java | 11 +-
.../patches/debian/etc/init.d/cloud-early-config | 6 +-
.../patches/debian/etc/init.d/cloud-passwd-srvr | 118 ---------------------
systemvm/patches/{vpn => debian}/etc/ipsec.conf | 0
.../patches/{vpn => debian}/etc/ipsec.d/l2tp.conf | 0
systemvm/patches/{vpn => debian}/etc/ipsec.secrets | 0
.../patches/{vpn => debian}/etc/ppp/options.xl2tpd | 0
systemvm/patches/debian/etc/rc.local | 9 ++
.../systemd/system/cloud-password-server@.service | 11 ++
.../etc/systemd/system/cloud-postinit.service | 13 +++
.../debian/etc/systemd/system/cloud.service | 13 +++
.../patches/{vpn => debian}/etc/xl2tpd/xl2tpd.conf | 0
.../patches/debian/opt/cloud/bin/cs/CsAddress.py | 8 +-
systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py | 14 +--
.../patches/debian/opt/cloud/bin/cs/CsRedundant.py | 5 +-
.../patches/debian/opt/cloud/bin/setup/common.sh | 6 +-
.../debian/opt/cloud/bin/setup/consoleproxy.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/elbvm.sh | 2 +-
.../patches/debian/opt/cloud/bin/setup/ilbvm.sh | 2 +-
.../debian/opt/cloud/bin/setup/patchsystemvm.sh | 3 +-
.../opt/cloud/bin/setup/{ilbvm.sh => postinit.sh} | 32 ++----
.../debian/opt/cloud/bin/setup/secstorage.sh | 2 +-
.../patches/debian/opt/cloud/bin/update_config.py | 17 ++-
.../{vpn => debian}/opt/cloud/bin/vpn_l2tp.sh | 0
systemvm/pom.xml | 3 -
.../configure_systemvm_services.sh | 36 ++-----
tools/appliance/shar_cloud_scripts.sh | 7 +-
28 files changed, 99 insertions(+), 223 deletions(-)
diff --git a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
index e139ecb..76f5e4c 100644
--- a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
+++ b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
@@ -341,17 +341,18 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
@Override
public ExecutionResult createFileInVR(final String routerIp, final String path, final String filename, final String content) {
final File permKey = new File("/root/.ssh/id_rsa.cloud");
- String error = null;
-
- s_logger.debug("Creating file in VR, with ip: " + routerIp + ", file: " + filename);
+ boolean success = true;
+ String details = "Creating file in VR, with ip: " + routerIp + ", file: " + filename;
+ s_logger.debug(details);
try {
SshHelper.scpTo(routerIp, 3922, "root", permKey, null, path, content.getBytes(), filename, null);
} catch (final Exception e) {
s_logger.warn("Fail to create file " + path + filename + " in VR " + routerIp, e);
- error = e.getMessage();
+ details = e.getMessage();
+ success = false;
}
- return new ExecutionResult(error == null, error);
+ return new ExecutionResult(success, details);
}
@Override
diff --git a/systemvm/patches/debian/etc/init.d/cloud-early-config b/systemvm/patches/debian/etc/init.d/cloud-early-config
index 9964f90..5e3f7b6 100755
--- a/systemvm/patches/debian/etc/init.d/cloud-early-config
+++ b/systemvm/patches/debian/etc/init.d/cloud-early-config
@@ -192,10 +192,10 @@ patch() {
tar xzf $patchfile -C /
echo ${newmd5} > ${md5file}
log_it "Patched scripts using $patchfile"
- fi
- log_it "Patching cloud service"
- /opt/cloud/bin/setup/patchsystemvm.sh $PATCH_MOUNT $TYPE
+ log_it "Patching cloud service"
+ /opt/cloud/bin/setup/patchsystemvm.sh $PATCH_MOUNT $TYPE
+ fi
[ -f $privkey ] && cp -f $privkey /root/.ssh/ && chmod go-rwx /root/.ssh/authorized_keys
umount $PATCH_MOUNT
diff --git a/systemvm/patches/debian/etc/init.d/cloud-passwd-srvr b/systemvm/patches/debian/etc/init.d/cloud-passwd-srvr
deleted file mode 100755
index ffd0ec8..0000000
--- a/systemvm/patches/debian/etc/init.d/cloud-passwd-srvr
+++ /dev/null
@@ -1,118 +0,0 @@
-#!/bin/bash
-### BEGIN INIT INFO
-# Provides: cloud-passwd-srvr
-# Required-Start: $local_fs cloud-early-config
-# Required-Stop: $local_fs
-# Default-Start: 3 4 5
-# Default-Stop: 0 1 6
-# Short-Description: Web server that sends passwords to User VMs
-### END INIT INFO
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-add_iptables_rules()
-{
- #Delete any old iptables rule for port 8080 on eth0
- remove_iptables_rules
-
- #For all cidrs on eth0 for port 8080 accept only if source is withing that cidr
- for cidr in $(ip addr | grep eth0 | grep inet | awk '{print $2}');
- do
- count=1
- #Try for 10 times, if it still fails then bail
- while [ $count -le 10 ];
- do
- (( count++ ))
- iptables -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp -s $cidr --dport 8080 -j ACCEPT
- if [ `iptables -L INPUT -n -v | grep eth0 | grep 8080 | grep ACCEPT | wc -l` -gt 0 ]
- then
- break
- else
- sleep 2
- fi
- done
- done
- echo "Added cloud-passwd-srvr iptables rules" && return 0
-}
-
-remove_iptables_rules()
-{
- #Change the Internal Field Separator so the for loop, loops on lines and not spaces
- OIFS="${IFS}"
- NIFS=$'\n'
- IFS="${NIFS}"
-
- #Removed all iptable rules for port 8080 on eth0, they were added in start()
- for srcdest in `iptables -L -n -v | grep eth0 | grep 8080 | grep ACCEPT | awk '{print "--source "$8" --destination "$9}'`;
- do
- eval "iptables -D INPUT -i eth0 -p tcp -m state --state NEW -m tcp $srcdest --dport 8080 -j ACCEPT";
- done
-
- #Restore IFS
- IFS="${OIFS}"
-
- echo "Removed cloud-passwd-srvr iptables rules" && return 0
-}
-
-start() {
- pid=$(getpid)
- [ "$pid" != "" ] && echo "Password server is already running (pid=$pid)" && return 0
- add_iptables_rules
- nohup bash /opt/cloud/bin/passwd_server &
-}
-
-getpid() {
- pid=$(ps -ef | grep passwd_server_ip | grep -v grep | awk '{print $2}')
- echo $pid
-}
-
-stop_password_server() {
- spid=$(pidof -s python passwd_server_ip.py)
- [ "$spid" != "" ] && kill -9 $spid && echo "Killed password server (pid=$spid)"
- return 0
-}
-
-stop () {
- stop_password_server
- pid=$(getpid)
- [ "$pid" != "" ] && kill -9 $pid && remove_iptables_rules && echo "Stopped password server (pid=$pid)" && stop_password_server && return 0
-
- echo "Password server is not running" && return 0
-}
-
-status () {
- pid=$(getpid)
- [ "$pid" != "" ] && echo "Password server is running (pid=$pid)" && return 0
- echo "Password server is not running" && return 0
-}
-
-case "$1" in
- start) start
- ;;
- stop) stop
- ;;
- status) status
- ;;
- restart) stop
- start
- ;;
- *) echo "Usage: $0 {start|stop|status|restart}"
- exit 1
- ;;
-esac
-
-exit 0
diff --git a/systemvm/patches/vpn/etc/ipsec.conf b/systemvm/patches/debian/etc/ipsec.conf
similarity index 100%
rename from systemvm/patches/vpn/etc/ipsec.conf
rename to systemvm/patches/debian/etc/ipsec.conf
diff --git a/systemvm/patches/vpn/etc/ipsec.d/l2tp.conf b/systemvm/patches/debian/etc/ipsec.d/l2tp.conf
similarity index 100%
rename from systemvm/patches/vpn/etc/ipsec.d/l2tp.conf
rename to systemvm/patches/debian/etc/ipsec.d/l2tp.conf
diff --git a/systemvm/patches/vpn/etc/ipsec.secrets b/systemvm/patches/debian/etc/ipsec.secrets
similarity index 100%
rename from systemvm/patches/vpn/etc/ipsec.secrets
rename to systemvm/patches/debian/etc/ipsec.secrets
diff --git a/systemvm/patches/vpn/etc/ppp/options.xl2tpd b/systemvm/patches/debian/etc/ppp/options.xl2tpd
similarity index 100%
rename from systemvm/patches/vpn/etc/ppp/options.xl2tpd
rename to systemvm/patches/debian/etc/ppp/options.xl2tpd
diff --git a/systemvm/patches/debian/etc/rc.local b/systemvm/patches/debian/etc/rc.local
index 895c120..bb4e251 100755
--- a/systemvm/patches/debian/etc/rc.local
+++ b/systemvm/patches/debian/etc/rc.local
@@ -67,3 +67,12 @@ then
python /opt/cloud/bin/baremetal-vr.py &
logger -t cloud "Started baremetal-vr service"
fi
+
+if [ "$TYPE" == "router" ] || [ "$TYPE" == "vpcrouter" ] || [ "$TYPE" == "dhcpsrvr" ]
+then
+ if [ -x /opt/cloud/bin/update_config.py ]
+ then
+ /opt/cloud/bin/update_config.py cmd_line.json
+ logger -t cloud "Updated config: cmd_line.json"
+ fi
+fi
diff --git a/systemvm/patches/debian/etc/systemd/system/cloud-password-server@.service b/systemvm/patches/debian/etc/systemd/system/cloud-password-server@.service
new file mode 100644
index 0000000..8ee9e79
--- /dev/null
+++ b/systemvm/patches/debian/etc/systemd/system/cloud-password-server@.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Cloud password server on %I
+After=network.target local-fs.target
+
+[Install]
+WantedBy=multi-user.target
+
+[Service]
+Type=simple
+WorkingDirectory=/opt/cloud/bin
+ExecStart=/opt/cloud/bin/passwd_server_ip.py %I
diff --git a/systemvm/patches/debian/etc/systemd/system/cloud-postinit.service b/systemvm/patches/debian/etc/systemd/system/cloud-postinit.service
new file mode 100644
index 0000000..a5ca9b0
--- /dev/null
+++ b/systemvm/patches/debian/etc/systemd/system/cloud-postinit.service
@@ -0,0 +1,13 @@
+[Unit]
+Description=cloud-postinit: post init script
+After=cloud-early-config.service network.target local-fs.target
+Before=ssh.service
+Requires=network.service
+
+[Install]
+WantedBy=multi-user.target
+
+[Service]
+Type=oneshot
+ExecStart=/opt/cloud/bin/setup/postinit.sh
+RemainAfterExit=true
diff --git a/systemvm/patches/debian/etc/systemd/system/cloud.service b/systemvm/patches/debian/etc/systemd/system/cloud.service
new file mode 100644
index 0000000..9cd0f49
--- /dev/null
+++ b/systemvm/patches/debian/etc/systemd/system/cloud.service
@@ -0,0 +1,13 @@
+[Unit]
+Description=cloud: startup cloud service
+After=cloud-early-config.service network.target local-fs.target
+
+[Install]
+WantedBy=multi-user.target
+
+[Service]
+Type=simple
+WorkingDirectory=/usr/local/cloud/systemvm
+ExecStart=/usr/local/cloud/systemvm/_run.sh
+Restart=always
+RestartSec=5
diff --git a/systemvm/patches/vpn/etc/xl2tpd/xl2tpd.conf b/systemvm/patches/debian/etc/xl2tpd/xl2tpd.conf
similarity index 100%
rename from systemvm/patches/vpn/etc/xl2tpd/xl2tpd.conf
rename to systemvm/patches/debian/etc/xl2tpd/xl2tpd.conf
diff --git a/systemvm/patches/debian/opt/cloud/bin/cs/CsAddress.py b/systemvm/patches/debian/opt/cloud/bin/cs/CsAddress.py
index d190a72..3980b18 100755
--- a/systemvm/patches/debian/opt/cloud/bin/cs/CsAddress.py
+++ b/systemvm/patches/debian/opt/cloud/bin/cs/CsAddress.py
@@ -565,9 +565,11 @@ class CsIP:
app.setup()
cmdline = self.config.cmdline()
- # If redundant then this is dealt with by the master backup functions
- if self.get_type() in ["guest"] and not cmdline.is_redundant():
- pwdsvc = CsPasswdSvc(self.address['public_ip']).start()
+ # Start password server on non-redundant routers and on master rVR
+ if self.get_type() in ["guest"] and (not cmdline.is_redundant() or cmdline.is_master()):
+ CsPasswdSvc(self.address['public_ip']).start()
+ else:
+ CsPasswdSvc(self.address['public_ip']).stop()
if self.get_type() == "public" and self.config.is_vpc() and method == "add":
if self.address["source_nat"]:
diff --git a/systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py b/systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py
index 496a0e7..2085088 100755
--- a/systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py
+++ b/systemvm/patches/debian/opt/cloud/bin/cs/CsApp.py
@@ -16,7 +16,6 @@
# specific language governing permissions and limitations
# under the License.
import os
-import CsHelper
from CsFile import CsFile
from CsProcess import CsProcess
import CsHelper
@@ -74,20 +73,13 @@ class CsPasswdSvc():
self.ip = ip
def start(self):
- proc = CsProcess(["dummy"])
- if proc.grep("passwd_server_ip %s" % self.ip) == -1:
- proc.start("/opt/cloud/bin/passwd_server_ip %s >> /var/log/cloud.log 2>&1" % self.ip, "&")
+ CsHelper.service("cloud-password-server@%s" % self.ip, "start")
def stop(self):
- proc = CsProcess(["Password Service"])
- pid = proc.grep("passwd_server_ip %s" % self.ip)
- proc.kill(pid)
- pid = proc.grep("8080,reuseaddr,fork,crnl,bind=%s" % self.ip)
- proc.kill(pid)
+ CsHelper.service("cloud-password-server@%s" % self.ip, "stop")
def restart(self):
- self.stop()
- self.start()
+ CsHelper.service("cloud-password-server@%s" % self.ip, "restart")
class CsDnsmasq(CsApp):
diff --git a/systemvm/patches/debian/opt/cloud/bin/cs/CsRedundant.py b/systemvm/patches/debian/opt/cloud/bin/cs/CsRedundant.py
index 99c7458..71c4eba 100755
--- a/systemvm/patches/debian/opt/cloud/bin/cs/CsRedundant.py
+++ b/systemvm/patches/debian/opt/cloud/bin/cs/CsRedundant.py
@@ -76,10 +76,11 @@ class CsRedundant(object):
CsHelper.service("keepalived", "stop")
#CsHelper.umount_tmpfs(self.CS_RAMDISK_DIR)
#CsHelper.rmdir(self.CS_RAMDISK_DIR)
- CsHelper.rm(self.CONNTRACKD_CONF)
- CsHelper.rm(self.KEEPALIVED_CONF)
+ #CsHelper.rm(self.CONNTRACKD_CONF)
+ #CsHelper.rm(self.KEEPALIVED_CONF)
def _redundant_on(self):
+ return
guest = self.address.get_guest_if()
# No redundancy if there is no guest network
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/common.sh b/systemvm/patches/debian/opt/cloud/bin/setup/common.sh
index e476745..4fc883d 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/common.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/common.sh
@@ -596,18 +596,16 @@ routing_svcs() {
echo "cloud nfs-common portmap" > /var/cache/cloud/disabled_svcs
if [ $RROUTER -eq 1 ]
then
- systemctl disable --now cloud-passwd-srvr
systemctl disable --now dnsmasq
systemctl enable conntrackd
systemctl enable keepalived
echo "keepalived conntrackd" >> /var/cache/cloud/enabled_svcs
- echo "dnsmasq cloud-passwd-srvr" >> /var/cache/cloud/disabled_svcs
+ echo "dnsmasq" >> /var/cache/cloud/disabled_svcs
else
systemctl disable --now conntrackd
systemctl disable --now keepalived
- systemctl enable cloud-passwd-srvr
systemctl enable dnsmasq
- echo "dnsmasq cloud-passwd-srvr " >> /var/cache/cloud/enabled_svcs
+ echo "dnsmasq" >> /var/cache/cloud/enabled_svcs
echo "keepalived conntrackd " >> /var/cache/cloud/disabled_svcs
fi
}
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/consoleproxy.sh b/systemvm/patches/debian/opt/cloud/bin/setup/consoleproxy.sh
index 36c1fa4..ad23381 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/consoleproxy.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/consoleproxy.sh
@@ -20,7 +20,7 @@
consoleproxy_svcs() {
echo "cloud ssh" > /var/cache/cloud/enabled_svcs
- echo "cloud-passwd-srvr haproxy dnsmasq apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
+ echo "haproxy dnsmasq apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
mkdir -p /var/log/cloud
}
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh b/systemvm/patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh
index 1e28ab4..d9a9c1a 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/dhcpsrvr.sh
@@ -19,7 +19,7 @@
. /opt/cloud/bin/setup/common.sh
dhcpsrvr_svcs() {
- echo "ssh dnsmasq cloud-passwd-srvr apache2" > /var/cache/cloud/enabled_svcs
+ echo "ssh dnsmasq apache2" > /var/cache/cloud/enabled_svcs
echo "cloud nfs-common conntrackd keepalived haproxy portmap" > /var/cache/cloud/disabled_svcs
}
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/elbvm.sh b/systemvm/patches/debian/opt/cloud/bin/setup/elbvm.sh
index 23a2607..fbae405 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/elbvm.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/elbvm.sh
@@ -20,7 +20,7 @@
elbvm_svcs() {
echo "ssh haproxy" > /var/cache/cloud/enabled_svcs
- echo "cloud dnsmasq conntrackd keepalived cloud-passwd-srvr apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
+ echo "cloud dnsmasq conntrackd keepalived apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
}
setup_elbvm() {
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh b/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh
index 4cd599f..809be09 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh
@@ -20,7 +20,7 @@
ilbvm_svcs() {
echo "ssh haproxy" > /var/cache/cloud/enabled_svcs
- echo "cloud dnsmasq conntrackd keepalived cloud-passwd-srvr apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
+ echo "cloud dnsmasq conntrackd keepalived apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
}
setup_ilbvm() {
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/patchsystemvm.sh b/systemvm/patches/debian/opt/cloud/bin/setup/patchsystemvm.sh
index d09cc03..2e9634d 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/patchsystemvm.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/patchsystemvm.sh
@@ -1,4 +1,4 @@
-#/bin/bash
+#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -55,3 +55,4 @@ then
exit 1
fi
fi
+systemctl daemon-reload
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh b/systemvm/patches/debian/opt/cloud/bin/setup/postinit.sh
similarity index 55%
copy from systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh
copy to systemvm/patches/debian/opt/cloud/bin/setup/postinit.sh
index 4cd599f..6024329 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/ilbvm.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/postinit.sh
@@ -18,31 +18,11 @@
. /opt/cloud/bin/setup/common.sh
-ilbvm_svcs() {
- echo "ssh haproxy" > /var/cache/cloud/enabled_svcs
- echo "cloud dnsmasq conntrackd keepalived cloud-passwd-srvr apache2 nfs-common portmap" > /var/cache/cloud/disabled_svcs
-}
-
-setup_ilbvm() {
- log_it "Setting up Internal Load Balancer system vm"
- setup_common eth0 eth1
- #eth0 = guest network, eth1=control network
-
- sed -i /$NAME/d /etc/hosts
- echo "$ETH0_IP $NAME" >> /etc/hosts
-
- cp /etc/iptables/iptables-ilbvm /etc/iptables/rules.v4
- cp /etc/iptables/iptables-ilbvm /etc/iptables/rules
- setup_sshd $ETH1_IP "eth1"
-
- enable_fwding 0
- enable_irqbalance 1
-}
-
-ilbvm_svcs
-if [ $? -gt 0 ]
+if [ "$TYPE" == "router" ] || [ "$TYPE" == "vpcrouter" ] || [ "$TYPE" == "dhcpsrvr" ]
then
- log_it "Failed to execute ilbvm svcs"
- exit 1
+ if [ -x /opt/cloud/bin/update_config.py ]
+ then
+ /opt/cloud/bin/update_config.py cmd_line.json
+ logger -t cloud "Updated config: cmd_line.json"
+ fi
fi
-setup_ilbvm
diff --git a/systemvm/patches/debian/opt/cloud/bin/setup/secstorage.sh b/systemvm/patches/debian/opt/cloud/bin/setup/secstorage.sh
index 327353c..5cdc4bb 100755
--- a/systemvm/patches/debian/opt/cloud/bin/setup/secstorage.sh
+++ b/systemvm/patches/debian/opt/cloud/bin/setup/secstorage.sh
@@ -20,7 +20,7 @@
secstorage_svcs() {
echo "apache2 cloud ssh nfs-common portmap" > /var/cache/cloud/enabled_svcs
- echo "cloud-passwd-srvr conntrackd keepalived haproxy dnsmasq" > /var/cache/cloud/disabled_svcs
+ echo "conntrackd keepalived haproxy dnsmasq" > /var/cache/cloud/disabled_svcs
mkdir -p /var/log/cloud
}
diff --git a/systemvm/patches/debian/opt/cloud/bin/update_config.py b/systemvm/patches/debian/opt/cloud/bin/update_config.py
index e36980a..9427bc8 100755
--- a/systemvm/patches/debian/opt/cloud/bin/update_config.py
+++ b/systemvm/patches/debian/opt/cloud/bin/update_config.py
@@ -24,11 +24,12 @@ from subprocess import PIPE, STDOUT
import os
import os.path
import configure
+import glob
import json
OCCURRENCES = 1
-logging.basicConfig(filename='/var/log/cloud.log', level=logging.INFO, format='%(asctime)s %(filename)s %(funcName)s:%(lineno)d %(message)s')
+logging.basicConfig(filename='/var/log/cloud.log', level=logging.DEBUG, format='%(asctime)s %(filename)s %(funcName)s:%(lineno)d %(message)s')
# first commandline argument should be the file to process
if (len(sys.argv) != 2):
@@ -41,14 +42,6 @@ jsonCmdConfigPath = jsonPath % sys.argv[1]
currentGuestNetConfig = "/etc/cloudstack/guestnetwork.json"
-# If the command line json file is unprocessed process it
-# This is important or, the control interfaces will get deleted!
-if os.path.isfile(jsonPath % "cmd_line.json"):
- qf = QueueFile()
- qf.setFile("cmd_line.json")
- qf.load(None)
-
-
def finish_config():
# Converge
returncode = configure.main(sys.argv)
@@ -82,7 +75,8 @@ def is_guestnet_configured(guestnet_dict, keys):
print "[WARN] update_config.py :: Reconfiguring guest network..."
return False
- file = open(jsonCmdConfigPath)
+ filename = min(glob.iglob(jsonCmdConfigPath + '*'), key=os.path.getctime)
+ file = open(filename)
new_guestnet_dict = json.load(file)
if not new_guestnet_dict['add']:
@@ -121,7 +115,8 @@ def is_guestnet_configured(guestnet_dict, keys):
return exists
-filename = jsonCmdConfigPath
+
+filename = min(glob.iglob(jsonCmdConfigPath + '*'), key=os.path.getctime)
if not (os.path.isfile(filename) and os.access(filename, os.R_OK)):
print "[ERROR] update_config.py :: You are telling me to process %s, but i can't access it" % jsonCmdConfigPath
sys.exit(1)
diff --git a/systemvm/patches/vpn/opt/cloud/bin/vpn_l2tp.sh b/systemvm/patches/debian/opt/cloud/bin/vpn_l2tp.sh
similarity index 100%
rename from systemvm/patches/vpn/opt/cloud/bin/vpn_l2tp.sh
rename to systemvm/patches/debian/opt/cloud/bin/vpn_l2tp.sh
diff --git a/systemvm/pom.xml b/systemvm/pom.xml
index 0ae3e02..c22f71c 100644
--- a/systemvm/pom.xml
+++ b/systemvm/pom.xml
@@ -115,9 +115,6 @@
<fileset dir="${basedir}/patches/debian/">
<include name="**/*" />
</fileset>
- <fileset dir="${basedir}/patches/vpn/">
- <include name="**/*" />
- </fileset>
</copy>
<copy overwrite="true" todir="${basedir}/target/build-patch/opt/cloud/bin/">
<fileset dir="${basedir}/../scripts/util/">
diff --git a/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh b/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
index 791f2ea..b4aaec9 100644
--- a/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
+++ b/tools/appliance/definitions/systemvmtemplate/configure_systemvm_services.sh
@@ -36,7 +36,7 @@ function install_cloud_scripts() {
rsync -av ./cloud_scripts/ /
chmod +x /opt/cloud/bin/* \
/root/{clearUsageRules.sh,reconfigLB.sh,monitorServices.py} \
- /etc/init.d/{cloud-early-config,cloud-passwd-srvr} \
+ /etc/init.d/{cloud-early-config} \
/etc/profile.d/cloud.sh
cat > /etc/systemd/system/cloud-early-config.service << EOF
@@ -62,42 +62,26 @@ TimeoutStartSec=5min
EOF
- cat > /etc/systemd/system/cloud.service << EOF
+ cat > /etc/systemd/system/cloud-postinit.service << EOF
[Unit]
-Description=cloud: startup cloud service
+Description=cloud-postinit: post init script
After=cloud-early-config.service network.target local-fs.target
+Before=ssh.service
+Requires=network.service
[Install]
WantedBy=multi-user.target
[Service]
-Type=simple
-WorkingDirectory=/usr/local/cloud/systemvm
-ExecStart=/usr/local/cloud/systemvm/_run.sh
-Restart=always
-RestartSec=5
-EOF
-
- cat > /etc/systemd/system/cloud-passwd-srvr.service << EOF
-[Unit]
-Description=cloud-passwd-srvr: cloud password server
-After=network.target local-fs.target
-
-[Install]
-WantedBy=multi-user.target
-
-[Service]
-Type=forking
-ExecStart=/etc/init.d/cloud-passwd-srvr start
-ExecStop=/etc/init.d/cloud-passwd-srvr stop
-Restart=always
-RestartSec=5
+Type=oneshot
+ExecStart=/opt/cloud/bin/setup/postinit.sh
+RemainAfterExit=true
+TimeoutStartSec=1min
EOF
systemctl daemon-reload
systemctl enable cloud-early-config
- systemctl disable cloud
- systemctl disable cloud-passwd-srvr
+ systemctl enable cloud-postinit
}
function do_signature() {
diff --git a/tools/appliance/shar_cloud_scripts.sh b/tools/appliance/shar_cloud_scripts.sh
index 5504d16..d67f889 100755
--- a/tools/appliance/shar_cloud_scripts.sh
+++ b/tools/appliance/shar_cloud_scripts.sh
@@ -37,14 +37,11 @@ TEMP_DIR=`mktemp -d ${TMPDIR}/shar_cloud.XXXXXXXX`
cd ${TEMP_DIR}
mkdir cloud_scripts
mkdir -p cloud_scripts/opt/cloudstack
-cp -r ${CLOUDSTACK_DIR}/systemvm/patches/debian/config/* cloud_scripts/
-cp -r ${CLOUDSTACK_DIR}/systemvm/patches/debian/vpn/* cloud_scripts/
+cp -r ${CLOUDSTACK_DIR}/systemvm/patches/debian/* cloud_scripts/
mkdir -p cloud_scripts/usr/share/cloud
-cd ${CLOUDSTACK_DIR}/systemvm/patches/debian/config
+cd ${CLOUDSTACK_DIR}/systemvm/patches/debian
tar -cf ${TEMP_DIR}/cloud_scripts/usr/share/cloud/cloud-scripts.tar *
-cd ${CLOUDSTACK_DIR}/systemvm/patches/debian/vpn
-tar -rf ${TEMP_DIR}/cloud_scripts/usr/share/cloud/cloud-scripts.tar *
cd ${TEMP_DIR}
shar `find . -print` > ${CURR_DIR}/cloud_scripts_shar_archive.sh
--
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>.
[cloudstack] 04/04: refactor crufty configuration code
Posted by bh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git
commit cc3113a010266a52d21fea54719125e88ef1e46d
Author: Rohit Yadav <ro...@shapeblue.com>
AuthorDate: Mon Dec 4 10:44:34 2017 +0530
refactor crufty configuration code
Signed-off-by: Rohit Yadav <ro...@shapeblue.com>
---
systemvm/patches/debian/etc/rc.local | 9 ---
systemvm/patches/debian/opt/cloud/bin/configure.py | 89 +++++++++++-----------
.../patches/debian/opt/cloud/bin/update_config.py | 16 ++--
3 files changed, 49 insertions(+), 65 deletions(-)
diff --git a/systemvm/patches/debian/etc/rc.local b/systemvm/patches/debian/etc/rc.local
index bb4e251..895c120 100755
--- a/systemvm/patches/debian/etc/rc.local
+++ b/systemvm/patches/debian/etc/rc.local
@@ -67,12 +67,3 @@ then
python /opt/cloud/bin/baremetal-vr.py &
logger -t cloud "Started baremetal-vr service"
fi
-
-if [ "$TYPE" == "router" ] || [ "$TYPE" == "vpcrouter" ] || [ "$TYPE" == "dhcpsrvr" ]
-then
- if [ -x /opt/cloud/bin/update_config.py ]
- then
- /opt/cloud/bin/update_config.py cmd_line.json
- logger -t cloud "Updated config: cmd_line.json"
- fi
-fi
diff --git a/systemvm/patches/debian/opt/cloud/bin/configure.py b/systemvm/patches/debian/opt/cloud/bin/configure.py
index 3b39e6d..ed67ec6 100755
--- a/systemvm/patches/debian/opt/cloud/bin/configure.py
+++ b/systemvm/patches/debian/opt/cloud/bin/configure.py
@@ -17,15 +17,15 @@
# specific language governing permissions and limitations
# under the License.
import sys
+import os
import base64
+import time
from collections import OrderedDict
import logging
import re
-import os.path
-import os
from fcntl import flock, LOCK_EX, LOCK_UN
from cs.CsDatabag import CsDataBag
@@ -39,7 +39,6 @@ from cs.CsConfig import CsConfig
from cs.CsProcess import CsProcess
from cs.CsStaticRoutes import CsStaticRoutes
-OCCURRENCES = 1
class CsPassword(CsDataBag):
@@ -197,12 +196,12 @@ class CsAcl(CsDataBag):
self.rule['action'] = "ACCEPT"
egressIpsetStr=''
- if sflag == True and dflag == True:
+ if sflag and dflag :
egressIpsetStr = " -m set --match-set %s src " % sourceIpsetName + \
" -m set --match-set %s dst " % destIpsetName
- elif sflag == True:
+ elif sflag:
egressIpsetStr = " -m set --match-set %s src " % sourceIpsetName
- elif dflag == True:
+ elif dflag:
egressIpsetStr = " -m set --match-set %s dst " % destIpsetName
if rule['protocol'] == "icmp":
@@ -666,8 +665,7 @@ class CsRemoteAccessVpn(CsDataBag):
# Enable remote access vpn
if vpnconfig['create']:
- shutdownIpsec = False
- logging.debug("Enabling remote access vpn on "+ public_ip)
+ logging.debug("Enabling remote access vpn on "+ public_ip)
CsHelper.start_if_stopped("ipsec")
self.configure_l2tpIpsec(public_ip, self.dbag[public_ip])
@@ -995,6 +993,8 @@ def main(argv):
logging.debug("No file was received, do not go on processing the other actions. Just leave for now.")
return
+ json_type = os.path.basename(process_file).split('.json')[0]
+
# The "GLOBAL" Configuration object
config = CsConfig()
@@ -1008,50 +1008,47 @@ def main(argv):
config.address().compare()
config.address().process()
- databag_map = OrderedDict([("guest_network.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("vm_password.json", {"process_iptables" : False, "executor" : CsPassword("vmpassword", config)}),
- ("vm_metadata.json", {"process_iptables" : False, "executor" : CsVmMetadata('vmdata', config)}),
- ("network_acl.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("firewall_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("forwarding_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("staticnat_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("site_2_site_vpn.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("remote_access_vpn.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("vpn_user_list.json", {"process_iptables" : False, "executor" : CsVpnUser("vpnuserlist", config)}),
- ("vm_dhcp_entry.json", {"process_iptables" : False, "executor" : CsDhcp("dhcpentry", config)}),
- ("dhcp.json", {"process_iptables" : False, "executor" : CsDhcp("dhcpentry", config)}),
- ("load_balancer.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
- ("monitor_service.json", {"process_iptables" : False, "executor" : CsMonitor("monitorservice", config)}),
- ("static_routes.json", {"process_iptables" : False, "executor" : CsStaticRoutes("staticroutes", config)})
+ databag_map = OrderedDict([("guest_network", {"process_iptables" : True, "executor" : []}),
+ ("vm_password", {"process_iptables" : False, "executor" : [CsPassword("vmpassword", config)]}),
+ ("vm_metadata", {"process_iptables" : False, "executor" : [CsVmMetadata('vmdata', config)]}),
+ ("network_acl", {"process_iptables" : True, "executor" : []}),
+ ("firewall_rules", {"process_iptables" : True, "executor" : []}),
+ ("forwarding_rules", {"process_iptables" : True, "executor" : []}),
+ ("staticnat_rules", {"process_iptables" : True, "executor" : []}),
+ ("site_2_site_vpn", {"process_iptables" : True, "executor" : []}),
+ ("remote_access_vpn", {"process_iptables" : True, "executor" : []}),
+ ("vpn_user_list", {"process_iptables" : False, "executor" : [CsVpnUser("vpnuserlist", config)]}),
+ ("vm_dhcp_entry", {"process_iptables" : False, "executor" : [CsDhcp("dhcpentry", config)]}),
+ ("dhcp", {"process_iptables" : False, "executor" : [CsDhcp("dhcpentry", config)]}),
+ ("load_balancer", {"process_iptables" : True, "executor" : []}),
+ ("monitor_service", {"process_iptables" : False, "executor" : [CsMonitor("monitorservice", config)]}),
+ ("static_routes", {"process_iptables" : False, "executor" : [CsStaticRoutes("staticroutes", config)]})
])
- if process_file.count("cmd_line.json") == OCCURRENCES:
- logging.debug("cmd_line.json changed. All other files will be processed as well.")
-
- while databag_map:
- item = databag_map.popitem(last = False)
- item_name = item[0]
- item_dict = item[1]
- if not item_dict["process_iptables"]:
- executor = item_dict["executor"]
- executor.process()
+ def execDatabag(key, db):
+ if key not in db.keys() or 'executor' not in db[key]:
+ logging.warn("Unable to find config or executor(s) for the databag type %s" % key)
+ return
+ for executor in db[key]['executor']:
+ logging.debug("Processing for databag type: %s" % key)
+ executor.process()
+ def execIptables(config):
+ logging.debug("Processing iptables rules")
iptables_executor = IpTablesExecutor(config)
iptables_executor.process()
- else:
- while databag_map:
- item = databag_map.popitem(last = False)
- item_name = item[0]
- item_dict = item[1]
- if process_file.count(item_name) == OCCURRENCES:
- executor = item_dict["executor"]
- executor.process()
-
- if item_dict["process_iptables"]:
- iptables_executor = IpTablesExecutor(config)
- iptables_executor.process()
- break
+ if json_type == "cmd_line":
+ logging.debug("cmd_line.json changed. All other files will be processed as well.")
+ for key in databag_map.keys():
+ execDatabag(key, databag_map)
+ execIptables(config)
+ elif json_type in databag_map.keys():
+ execDatabag(json_type, databag_map)
+ if databag_map[json_type]['process_iptables']:
+ execIptables(config)
+ else:
+ logging.warn("Unable to find and process databag for file: %s, for json type=%s" % (process_file, json_type))
red = CsRedundant(config)
red.set()
diff --git a/systemvm/patches/debian/opt/cloud/bin/update_config.py b/systemvm/patches/debian/opt/cloud/bin/update_config.py
index 9427bc8..e4e70de 100755
--- a/systemvm/patches/debian/opt/cloud/bin/update_config.py
+++ b/systemvm/patches/debian/opt/cloud/bin/update_config.py
@@ -24,11 +24,8 @@ from subprocess import PIPE, STDOUT
import os
import os.path
import configure
-import glob
import json
-OCCURRENCES = 1
-
logging.basicConfig(filename='/var/log/cloud.log', level=logging.DEBUG, format='%(asctime)s %(filename)s %(funcName)s:%(lineno)d %(message)s')
# first commandline argument should be the file to process
@@ -38,7 +35,8 @@ if (len(sys.argv) != 2):
# FIXME we should get this location from a configuration class
jsonPath = "/var/cache/cloud/%s"
-jsonCmdConfigPath = jsonPath % sys.argv[1]
+jsonFilename = sys.argv[1]
+jsonConfigFile = jsonPath % jsonFilename
currentGuestNetConfig = "/etc/cloudstack/guestnetwork.json"
@@ -75,8 +73,7 @@ def is_guestnet_configured(guestnet_dict, keys):
print "[WARN] update_config.py :: Reconfiguring guest network..."
return False
- filename = min(glob.iglob(jsonCmdConfigPath + '*'), key=os.path.getctime)
- file = open(filename)
+ file = open(jsonConfigFile)
new_guestnet_dict = json.load(file)
if not new_guestnet_dict['add']:
@@ -116,13 +113,12 @@ def is_guestnet_configured(guestnet_dict, keys):
return exists
-filename = min(glob.iglob(jsonCmdConfigPath + '*'), key=os.path.getctime)
-if not (os.path.isfile(filename) and os.access(filename, os.R_OK)):
- print "[ERROR] update_config.py :: You are telling me to process %s, but i can't access it" % jsonCmdConfigPath
+if not (os.path.isfile(jsonConfigFile) and os.access(jsonConfigFile, os.R_OK)):
+ print "[ERROR] update_config.py :: Unable to read and access %s to process it" % jsonConfigFile
sys.exit(1)
# If the guest network is already configured and have the same IP, do not try to configure it again otherwise it will break
-if sys.argv[1] and sys.argv[1].count("guest_network.json") == OCCURRENCES:
+if jsonFilename.startswith("guest_network.json"):
if os.path.isfile(currentGuestNetConfig):
file = open(currentGuestNetConfig)
guestnet_dict = json.load(file)
--
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>.
[cloudstack] 01/04: CLOUDSTACK-10013: Make the generated VR/json
files unique (ports #1470)
Posted by bh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a commit to branch debian9-systemvmtemplate
in repository https://gitbox.apache.org/repos/asf/cloudstack.git
commit 584e69d92c55a24900e3cb92c19cc8ed8222c010
Author: Remi Bergsma <gi...@remi.nl>
AuthorDate: Sat Dec 2 23:19:14 2017 +0530
CLOUDSTACK-10013: Make the generated VR/json files unique (ports #1470)
This ports PR #1470 by @remibergsma.
Make the generated json files unique to prevent concurrency issues:
The json files now have UUIDs to prevent them from getting overwritten
before they've been executed. Prevents config to be pushed to the wrong
router.
2016-02-25 18:32:23,797 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) (logid:) Seq 2-4684025087442026584: Processing: { Ans: , MgmtId: 90520732674657, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.routing.GroupA
nswer":{"results":["null - success: null","null - success: [INFO] update_config.py :: Processing incoming file => vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b\n[INFO] Processing JSON file vm_dhcp_entry.json.4ea4506
1-2efb-4467-8eaa-db3d77fb0a7b\n"],"result":true,"wait":0}}] }
On the router:
2016-02-25 18:32:23,416 merge.py __moveFile:298 Processed file written to /var/cache/cloud/processed/vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b.gz
Signed-off-by: Rohit Yadav <ro...@shapeblue.com>
---
.../facade/AbstractConfigItemFacade.java | 21 +-
.../kvm/resource/LibvirtComputingResource.java | 5 +
systemvm/patches/debian/etc/rc.local | 9 -
systemvm/patches/debian/opt/cloud/bin/configure.py | 220 +++++++++------------
systemvm/patches/debian/opt/cloud/bin/merge.py | 37 ++--
.../patches/debian/opt/cloud/bin/update_config.py | 22 ++-
6 files changed, 159 insertions(+), 155 deletions(-)
diff --git a/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java b/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
index f017384..a083012 100644
--- a/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
+++ b/core/src/com/cloud/agent/resource/virtualnetwork/facade/AbstractConfigItemFacade.java
@@ -22,6 +22,8 @@ package com.cloud.agent.resource.virtualnetwork.facade;
import java.util.Hashtable;
import java.util.LinkedList;
import java.util.List;
+import java.util.UUID;
+import org.apache.log4j.Logger;
import com.cloud.agent.api.BumpUpPriorityCommand;
import com.cloud.agent.api.SetupGuestNetworkCommand;
@@ -58,6 +60,8 @@ import com.google.gson.GsonBuilder;
public abstract class AbstractConfigItemFacade {
+ private static final Logger s_logger = Logger.getLogger(AbstractConfigItemFacade.class);
+
private final static Gson gson;
private static Hashtable<Class<? extends NetworkElementCommand>, AbstractConfigItemFacade> flyweight = new Hashtable<Class<? extends NetworkElementCommand>, AbstractConfigItemFacade>();
@@ -104,13 +108,26 @@ public abstract class AbstractConfigItemFacade {
return instance;
}
+ private static String appendUuidToJsonFiles(final String filename) {
+ String remoteFileName = new String(filename);
+ if (remoteFileName.endsWith("json")) {
+ remoteFileName += "." + UUID.randomUUID().toString();
+ }
+ return remoteFileName;
+ }
+
protected List<ConfigItem> generateConfigItems(final ConfigBase configuration) {
final List<ConfigItem> cfg = new LinkedList<>();
- final ConfigItem configFile = new FileConfigItem(VRScripts.CONFIG_PERSIST_LOCATION, destinationFile, gson.toJson(configuration));
+ final String remoteFilename = appendUuidToJsonFiles(destinationFile);
+ if (s_logger.isDebugEnabled()) {
+ s_logger.debug("Transformed filename: " + destinationFile + " to: " + remoteFilename);
+ }
+
+ final ConfigItem configFile = new FileConfigItem(VRScripts.CONFIG_PERSIST_LOCATION, remoteFilename, gson.toJson(configuration));
cfg.add(configFile);
- final ConfigItem updateCommand = new ScriptConfigItem(VRScripts.UPDATE_CONFIG, destinationFile);
+ final ConfigItem updateCommand = new ScriptConfigItem(VRScripts.UPDATE_CONFIG, remoteFilename);
cfg.add(updateCommand);
return cfg;
diff --git a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
index 7794249..e139ecb 100644
--- a/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
+++ b/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
@@ -332,6 +332,9 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
if (details == null) {
details = parser.getLines();
}
+
+ s_logger.debug("Executing script in VR: " + script);
+
return new ExecutionResult(command.getExitValue() == 0, details);
}
@@ -340,6 +343,8 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
final File permKey = new File("/root/.ssh/id_rsa.cloud");
String error = null;
+ s_logger.debug("Creating file in VR, with ip: " + routerIp + ", file: " + filename);
+
try {
SshHelper.scpTo(routerIp, 3922, "root", permKey, null, path, content.getBytes(), filename, null);
} catch (final Exception e) {
diff --git a/systemvm/patches/debian/etc/rc.local b/systemvm/patches/debian/etc/rc.local
index bb4e251..895c120 100755
--- a/systemvm/patches/debian/etc/rc.local
+++ b/systemvm/patches/debian/etc/rc.local
@@ -67,12 +67,3 @@ then
python /opt/cloud/bin/baremetal-vr.py &
logger -t cloud "Started baremetal-vr service"
fi
-
-if [ "$TYPE" == "router" ] || [ "$TYPE" == "vpcrouter" ] || [ "$TYPE" == "dhcpsrvr" ]
-then
- if [ -x /opt/cloud/bin/update_config.py ]
- then
- /opt/cloud/bin/update_config.py cmd_line.json
- logger -t cloud "Updated config: cmd_line.json"
- fi
-fi
diff --git a/systemvm/patches/debian/opt/cloud/bin/configure.py b/systemvm/patches/debian/opt/cloud/bin/configure.py
index b8a3e02..3b39e6d 100755
--- a/systemvm/patches/debian/opt/cloud/bin/configure.py
+++ b/systemvm/patches/debian/opt/cloud/bin/configure.py
@@ -17,33 +17,29 @@
# specific language governing permissions and limitations
# under the License.
import sys
-import os
import base64
-from merge import DataBag
-from pprint import pprint
-import subprocess
+from collections import OrderedDict
+
import logging
import re
-import time
-import shutil
+
import os.path
import os
from fcntl import flock, LOCK_EX, LOCK_UN
-from cs.CsDatabag import CsDataBag, CsCmdLine
-import cs.CsHelper
+from cs.CsDatabag import CsDataBag
from cs.CsNetfilter import CsNetfilters
from cs.CsDhcp import CsDhcp
from cs.CsRedundant import *
from cs.CsFile import CsFile
-from cs.CsApp import CsApache, CsDnsmasq
from cs.CsMonitor import CsMonitor
from cs.CsLoadBalancer import CsLoadBalancer
from cs.CsConfig import CsConfig
from cs.CsProcess import CsProcess
from cs.CsStaticRoutes import CsStaticRoutes
+OCCURRENCES = 1
class CsPassword(CsDataBag):
@@ -668,16 +664,11 @@ class CsRemoteAccessVpn(CsDataBag):
continue
vpnconfig=self.dbag[public_ip]
- #Enable remote access vpn
+ # Enable remote access vpn
if vpnconfig['create']:
shutdownIpsec = False
logging.debug("Enabling remote access vpn on "+ public_ip)
- dev = CsHelper.get_device(public_ip)
- if dev == "":
- logging.error("Request for ipsec to %s not possible because ip is not configured", public_ip)
- continue
-
CsHelper.start_if_stopped("ipsec")
self.configure_l2tpIpsec(public_ip, self.dbag[public_ip])
logging.debug("Remote accessvpn data bag %s", self.dbag)
@@ -960,16 +951,49 @@ class CsForwardingRules(CsDataBag):
self.fw.append(["nat", "front", "-A POSTROUTING -s %s -d %s -j SNAT -o eth0 --to-source %s" % (self.getNetworkByIp(rule['internal_ip']),rule["internal_ip"], self.getGuestIp())])
+class IpTablesExecutor:
+
+ config = None
+
+ def __init__(self, config):
+ self.config = config
+
+ def process(self):
+ acls = CsAcl('networkacl', self.config)
+ acls.process()
+
+ acls = CsAcl('firewallrules', self.config)
+ acls.process()
+
+ fwd = CsForwardingRules("forwardingrules", self.config)
+ fwd.process()
+
+ vpns = CsSite2SiteVpn("site2sitevpn", self.config)
+ vpns.process()
+
+ rvpn = CsRemoteAccessVpn("remoteaccessvpn", self.config)
+ rvpn.process()
+
+ lb = CsLoadBalancer("loadbalancer", self.config)
+ lb.process()
+
+ logging.debug("Configuring iptables rules")
+ nf = CsNetfilters()
+ nf.compare(self.config.get_fw())
+
+ logging.debug("Configuring iptables rules done ...saving rules")
+
+ # Save iptables configuration - will be loaded on reboot by the iptables-restore that is configured on /etc/rc.local
+ CsHelper.save_iptables("iptables-save", "/etc/iptables/router_rules.v4")
+ CsHelper.save_iptables("ip6tables-save", "/etc/iptables/router_rules.v6")
+
def main(argv):
# The file we are currently processing, if it is "cmd_line.json" everything will be processed.
process_file = argv[1]
- # process_file can be None, if so assume cmd_line.json
if process_file is None:
- process_file = "cmd_line.json"
-
- # Track if changes need to be committed to NetFilter
- iptables_change = False
+ logging.debug("No file was received, do not go on processing the other actions. Just leave for now.")
+ return
# The "GLOBAL" Configuration object
config = CsConfig()
@@ -977,108 +1001,60 @@ def main(argv):
logging.basicConfig(filename=config.get_logger(),
level=config.get_level(),
format=config.get_format())
- try:
- # Load stored ip adresses from disk to CsConfig()
- config.set_address()
-
- logging.debug("Configuring ip addresses")
- config.address().compare()
- config.address().process()
-
- if process_file in ["cmd_line.json", "guest_network.json"]:
- logging.debug("Configuring Guest Network")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "vm_password.json"]:
- logging.debug("Configuring vmpassword")
- password = CsPassword("vmpassword", config)
- password.process()
-
- if process_file in ["cmd_line.json", "vm_metadata.json"]:
- logging.debug("Configuring vmdata")
- metadata = CsVmMetadata('vmdata', config)
- metadata.process()
-
- if process_file in ["cmd_line.json", "network_acl.json"]:
- logging.debug("Configuring networkacl")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "firewall_rules.json"]:
- logging.debug("Configuring firewall rules")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "forwarding_rules.json", "staticnat_rules.json"]:
- logging.debug("Configuring PF rules")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "site_2_site_vpn.json"]:
- logging.debug("Configuring s2s vpn")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "remote_access_vpn.json"]:
- logging.debug("Configuring remote access vpn")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "vpn_user_list.json"]:
- logging.debug("Configuring vpn users list")
- vpnuser = CsVpnUser("vpnuserlist", config)
- vpnuser.process()
-
- if process_file in ["cmd_line.json", "vm_dhcp_entry.json", "dhcp.json"]:
- logging.debug("Configuring dhcp entry")
- dhcp = CsDhcp("dhcpentry", config)
- dhcp.process()
-
- if process_file in ["cmd_line.json", "load_balancer.json"]:
- logging.debug("Configuring load balancer")
- iptables_change = True
-
- if process_file in ["cmd_line.json", "monitor_service.json"]:
- logging.debug("Configuring monitor service")
- mon = CsMonitor("monitorservice", config)
- mon.process()
-
- # If iptable rules have changed, apply them.
- if iptables_change:
- acls = CsAcl('networkacl', config)
- acls.process()
-
- acls = CsAcl('firewallrules', config)
- acls.flushAllowAllEgressRules()
- acls.process()
-
- fwd = CsForwardingRules("forwardingrules", config)
- fwd.process()
-
- vpns = CsSite2SiteVpn("site2sitevpn", config)
- vpns.process()
-
- rvpn = CsRemoteAccessVpn("remoteaccessvpn", config)
- rvpn.process()
-
- lb = CsLoadBalancer("loadbalancer", config)
- lb.process()
-
- logging.debug("Configuring iptables rules")
- nf = CsNetfilters()
- nf.compare(config.get_fw())
-
- logging.debug("Configuring iptables rules done ...saving rules")
-
- # Save iptables configuration - will be loaded on reboot by the iptables-restore that is configured on /etc/rc.local
- CsHelper.save_iptables("iptables-save", "/etc/iptables/router_rules.v4")
- CsHelper.save_iptables("ip6tables-save", "/etc/iptables/router_rules.v6")
-
- red = CsRedundant(config)
- red.set()
-
- if process_file in ["cmd_line.json", "static_routes.json"]:
- logging.debug("Configuring static routes")
- static_routes = CsStaticRoutes("staticroutes", config)
- static_routes.process()
- except Exception:
- logging.exception("Exception while configuring router")
- return 1
+ # Load stored ip adresses from disk to CsConfig()
+ config.set_address()
+
+ logging.debug("Configuring ip addresses")
+ config.address().compare()
+ config.address().process()
+
+ databag_map = OrderedDict([("guest_network.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("vm_password.json", {"process_iptables" : False, "executor" : CsPassword("vmpassword", config)}),
+ ("vm_metadata.json", {"process_iptables" : False, "executor" : CsVmMetadata('vmdata', config)}),
+ ("network_acl.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("firewall_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("forwarding_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("staticnat_rules.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("site_2_site_vpn.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("remote_access_vpn.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("vpn_user_list.json", {"process_iptables" : False, "executor" : CsVpnUser("vpnuserlist", config)}),
+ ("vm_dhcp_entry.json", {"process_iptables" : False, "executor" : CsDhcp("dhcpentry", config)}),
+ ("dhcp.json", {"process_iptables" : False, "executor" : CsDhcp("dhcpentry", config)}),
+ ("load_balancer.json", {"process_iptables" : True, "executor" : IpTablesExecutor(config)}),
+ ("monitor_service.json", {"process_iptables" : False, "executor" : CsMonitor("monitorservice", config)}),
+ ("static_routes.json", {"process_iptables" : False, "executor" : CsStaticRoutes("staticroutes", config)})
+ ])
+
+ if process_file.count("cmd_line.json") == OCCURRENCES:
+ logging.debug("cmd_line.json changed. All other files will be processed as well.")
+
+ while databag_map:
+ item = databag_map.popitem(last = False)
+ item_name = item[0]
+ item_dict = item[1]
+ if not item_dict["process_iptables"]:
+ executor = item_dict["executor"]
+ executor.process()
+
+ iptables_executor = IpTablesExecutor(config)
+ iptables_executor.process()
+ else:
+ while databag_map:
+ item = databag_map.popitem(last = False)
+ item_name = item[0]
+ item_dict = item[1]
+ if process_file.count(item_name) == OCCURRENCES:
+ executor = item_dict["executor"]
+ executor.process()
+
+ if item_dict["process_iptables"]:
+ iptables_executor = IpTablesExecutor(config)
+ iptables_executor.process()
+
+ break
+
+ red = CsRedundant(config)
+ red.set()
if __name__ == "__main__":
main(sys.argv)
diff --git a/systemvm/patches/debian/opt/cloud/bin/merge.py b/systemvm/patches/debian/opt/cloud/bin/merge.py
index 1c9adf2..ef20d0b 100755
--- a/systemvm/patches/debian/opt/cloud/bin/merge.py
+++ b/systemvm/patches/debian/opt/cloud/bin/merge.py
@@ -18,8 +18,10 @@
import json
import os
-import time
+import uuid
import logging
+import gzip
+import shutil
import cs_ip
import cs_guestnetwork
import cs_cmdline
@@ -36,8 +38,6 @@ import cs_remoteaccessvpn
import cs_vpnusers
import cs_staticroutes
-from pprint import pprint
-
class DataBag:
@@ -282,22 +282,26 @@ class QueueFile:
if data is not None:
self.data = data
self.type = self.data["type"]
- proc = updateDataBag(self)
+ updateDataBag(self)
return
- fn = self.configCache + '/' + self.fileName
+ filename = '{cache_location}/{json_file}'.format(cache_location = self.configCache, json_file = self.fileName)
try:
- handle = open(fn)
- except IOError:
- logging.error("Could not open %s", fn)
+ handle = open(filename)
+ except IOError as exception:
+ error_message = ("Exception occurred with the following exception error '{error}'. Could not open '{file}'. "
+ "It seems that the file has already been moved.".format(error = exception, file = filename))
+ logging.error(error_message)
else:
+ logging.info("Continuing with the processing of file '{file}'".format(file = filename))
+
self.data = json.load(handle)
self.type = self.data["type"]
handle.close()
if self.keep:
- self.__moveFile(fn, self.configCache + "/processed")
+ self.__moveFile(filename, self.configCache + "/processed")
else:
- os.remove(fn)
- proc = updateDataBag(self)
+ os.remove(filename)
+ updateDataBag(self)
def setFile(self, name):
self.fileName = name
@@ -314,8 +318,15 @@ class QueueFile:
def __moveFile(self, origPath, path):
if not os.path.exists(path):
os.makedirs(path)
- timestamp = str(int(round(time.time())))
- os.rename(origPath, path + "/" + self.fileName + "." + timestamp)
+ originalName = os.path.basename(origPath)
+ if originalName.count(".") == 1:
+ originalName += "." + str(uuid.uuid4())
+ zipped_file_name = path + "/" + originalName + ".gz"
+ with open(origPath, 'rb') as f_in, gzip.open(zipped_file_name, 'wb') as f_out:
+ shutil.copyfileobj(f_in, f_out)
+ os.remove(origPath)
+
+ logging.debug("Processed file written to %s", zipped_file_name)
class PrivateGatewayHack:
diff --git a/systemvm/patches/debian/opt/cloud/bin/update_config.py b/systemvm/patches/debian/opt/cloud/bin/update_config.py
index ab08e03..e36980a 100755
--- a/systemvm/patches/debian/opt/cloud/bin/update_config.py
+++ b/systemvm/patches/debian/opt/cloud/bin/update_config.py
@@ -26,6 +26,8 @@ import os.path
import configure
import json
+OCCURRENCES = 1
+
logging.basicConfig(filename='/var/log/cloud.log', level=logging.INFO, format='%(asctime)s %(filename)s %(funcName)s:%(lineno)d %(message)s')
# first commandline argument should be the file to process
@@ -39,6 +41,14 @@ jsonCmdConfigPath = jsonPath % sys.argv[1]
currentGuestNetConfig = "/etc/cloudstack/guestnetwork.json"
+# If the command line json file is unprocessed process it
+# This is important or, the control interfaces will get deleted!
+if os.path.isfile(jsonPath % "cmd_line.json"):
+ qf = QueueFile()
+ qf.setFile("cmd_line.json")
+ qf.load(None)
+
+
def finish_config():
# Converge
returncode = configure.main(sys.argv)
@@ -111,19 +121,13 @@ def is_guestnet_configured(guestnet_dict, keys):
return exists
-if not (os.path.isfile(jsonCmdConfigPath) and os.access(jsonCmdConfigPath, os.R_OK)):
+filename = jsonCmdConfigPath
+if not (os.path.isfile(filename) and os.access(filename, os.R_OK)):
print "[ERROR] update_config.py :: You are telling me to process %s, but i can't access it" % jsonCmdConfigPath
sys.exit(1)
-# If the command line json file is unprocessed process it
-# This is important or, the control interfaces will get deleted!
-if os.path.isfile(jsonPath % "cmd_line.json"):
- qf = QueueFile()
- qf.setFile("cmd_line.json")
- qf.load(None)
-
# If the guest network is already configured and have the same IP, do not try to configure it again otherwise it will break
-if sys.argv[1] == "guest_network.json":
+if sys.argv[1] and sys.argv[1].count("guest_network.json") == OCCURRENCES:
if os.path.isfile(currentGuestNetConfig):
file = open(currentGuestNetConfig)
guestnet_dict = json.load(file)
--
To stop receiving notification emails like this one, please contact
"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>.