You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/08/16 08:49:07 UTC
[GitHub] [cloudstack] davidjumani opened a new pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
davidjumani opened a new pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329
## Description
Adding AutoScaling support for cks
Kubernetes PR : https://github.com/kubernetes/autoscaler/pull/3629
Also replaces CoreOS with Debian
Fixes https://github.com/apache/cloudstack/issues/4198
TODO: Remove the templateid and template name from KubernetesClusterResponse and DB since the templates can vary after acs upgrades
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## How Has This Been Tested?
TODO
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-766833246
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2590
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-808687155
<b>Trillian test result (tid-275)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56017 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t275-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_storage_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 78 look OK, 8 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_basic_lifecycle_kubernetes_cluster | `Failure` | 49.69 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 44.12 | test_kubernetes_clusters.py
test_03_invalid_upgrade_kubernetes_cluster | `Failure` | 46.11 | test_kubernetes_clusters.py
test_04_scale_kubernetes_cluster | `Failure` | 43.08 | test_kubernetes_clusters.py
test_05_autoscale_kubernetes_cluster | `Failure` | 55.31 | test_kubernetes_clusters.py
test_06_deploy_kubernetes_ha_cluster | `Failure` | 63.43 | test_kubernetes_clusters.py
test_07_upgrade_kubernetes_ha_cluster | `Failure` | 50.16 | test_kubernetes_clusters.py
test_08_delete_kubernetes_ha_cluster | `Failure` | 54.21 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 275.94 | test_kubernetes_clusters.py
ContextSuite context=TestVMWareStoragePolicies>:setup | `Error` | 0.00 | test_storage_policy.py
test_02_create_template_with_checksum_sha1 | `Error` | 65.39 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 65.39 | test_templates.py
test_04_create_template_with_checksum_md5 | `Error` | 65.39 | test_templates.py
test_05_create_template_with_no_checksum | `Error` | 65.37 | test_templates.py
test_04_extract_template | `Failure` | 129.30 | test_templates.py
ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
test_01_volume_usage | `Failure` | 672.50 | test_usage.py
ContextSuite context=TestVAppsVM>:setup | `Error` | 43.41 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | `Failure` | 1510.50 | test_vm_life_cycle.py
test_06_download_detached_volume | `Failure` | 322.83 | test_volumes.py
test_10_list_volumes | `Failure` | 366.22 | test_volumes.py
ContextSuite context=TestVPCNics>:setup | `Error` | 0.00 | test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517213136
##########
File path: systemvm/debian/opt/cloud/bin/setup/CKSNode.sh
##########
@@ -0,0 +1,61 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+. /opt/cloud/bin/setup/common.sh
+
+setup_k8s_node() {
+ log_it "Setting up k8s node"
+
+ # set default ssh port and restart sshd service
+ sed -i 's/3922/22/g' /etc/ssh/sshd_config
+
+ swapoff -a
+ sudo sed -i '/ swap / s/^/#/' /etc/fstab
+ log_it "Swap disabled"
+
+ log_it "Setting up interfaces"
+ setup_common eth0
+ setup_system_rfc1918_internal
+
+ log_it "Setting up entry in hosts"
+ sed -i /$NAME/d /etc/hosts
+ echo "$ETH0_IP $NAME" >> /etc/hosts
+
+ public_ip=`getPublicIp`
+ echo "$public_ip $NAME" >> /etc/hosts
+
+ echo "export PATH='$PATH:/opt/bin/'">> ~/.bashrc
+
+ disable_rpfilter
+ enable_fwding 1
+ enable_irqbalance 0
+ setup_ntp
+ dhclient
Review comment:
sudo systemctl disable dhcpcd.service ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-804901269
Packaging result: :heavy_multiplication_x: centos7 :heavy_check_mark: centos8 :heavy_multiplication_x: debian. SL-JID 211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-725199767
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517214885
##########
File path: systemvm/debian/opt/cloud/bin/setup/postinit.sh
##########
@@ -33,6 +42,10 @@ then
fi
fi
+if [ "$TYPE" == "CKSNode" ]; then
+ pkill -9 dhclient
Review comment:
Note - if we've guarded how to call/run the dhclient, we may not need to do this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517219223
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,29 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
-
- apt-get -y autoremove --purge
- apt-get clean
- apt-get autoclean
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@Pearl1594 check if we can use - `--no-install-recommends `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-828962341
TODO: Remove the templateid and template name from KubernetesClusterResponse since the templates can vary after acs upgrades
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-904227186
<b>Trillian test result (tid-1744)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 65991 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1744-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 78 look OK, 11 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestScaleVm>:setup | `Error` | 0.00 | test_scale_vm.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 1511.28 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903699829
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 974
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697264025
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,778 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.ClusterDaoImpl;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.DataCenterDaoImpl;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.DataStoreRole;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateStorageResourceAssoc;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateDaoImpl;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.upgrade.dao.BasicTemplateDataStoreDaoImpl;
+import com.cloud.user.Account;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.vm.dao.VMInstanceDaoImpl;
+import org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDao;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreVO;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreVO;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.inject.Inject;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/systemvm/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ public static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
Review comment:
@Pearl1594 just check/revisit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906841143
<b>Trillian test result (tid-1783)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33209 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1783-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_arping_in_ssvm | `Failure` | 5.16 | test_diagnostics.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-766790071
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911560119
<b>Trillian test result (tid-1921)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38681 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1921-vmware-65u2.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_list_snapshots_with_removed_data_store | `Error` | 54.88 | test_snapshots.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695793257
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -405,6 +405,10 @@
static final ConfigKey<Boolean> HaVmRestartHostUp = new ConfigKey<Boolean>("Advanced", Boolean.class, "ha.vm.restart.hostup", "true",
"If an out-of-band stop of a VM is detected and its host is up, then power on the VM", true);
+ static final ConfigKey<Long> SystemVmRootDiskSize = new ConfigKey<Long>("Advanced",
+ Long.class, "systemvm.root.disk.size", "-1",
Review comment:
Can be done as part of another phase?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697268660
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,778 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.ClusterDaoImpl;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.DataCenterDaoImpl;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.DataStoreRole;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateStorageResourceAssoc;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateDaoImpl;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.upgrade.dao.BasicTemplateDataStoreDaoImpl;
+import com.cloud.user.Account;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.vm.dao.VMInstanceDaoImpl;
+import org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDao;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreVO;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreVO;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.inject.Inject;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/systemvm/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ public static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARTIAL_TEMPLATE_FOLDER = String.format("/template/tmpl/%d/", Account.ACCOUNT_ID_SYSTEM);
+ private static final String FETCH_REGISTERED_TEMPLATE_INSTALL_PATH = "SELECT install_path FROM `cloud`.`template_store_ref` where template_id = ? LIMIT 1";
+ private static final String storageScriptsDir = "scripts/storage/secondary";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ private static final Integer TOKEN_LENGTH = 10;
+
+ public static String CS_MAJOR_VERSION = null;
+ public static String CS_TINY_VERSION = null;
+
+ @Inject
+ DataCenterDao dataCenterDao;
+ @Inject
+ VMTemplateDao vmTemplateDao;
+ @Inject
+ TemplateDataStoreDao templateDataStoreDao;
+ @Inject
+ VMInstanceDao vmInstanceDao;
+ @Inject
+ ImageStoreDao imageStoreDao;
+ @Inject
+ ClusterDao clusterDao;
+ @Inject
+ ConfigurationDao configurationDao;
+
+ public SystemVmTemplateRegistration() {
+ dataCenterDao = new DataCenterDaoImpl();
+ vmTemplateDao = new VMTemplateDaoImpl();
+ templateDataStoreDao = new BasicTemplateDataStoreDaoImpl();
+ vmInstanceDao = new VMInstanceDaoImpl();
+ imageStoreDao = new ImageStoreDaoImpl();
+ clusterDao = new ClusterDaoImpl();
+ configurationDao = new ConfigurationDaoImpl();
+ }
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> FileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
Review comment:
Put a comment to explain the guest OS ID
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894056334
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-897382067
<b>Trillian Build Failed (tid-1615)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-781876755
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-695923972
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-921424687
<b>Trillian test result (tid-2094)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39790 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t2094-xenserver-71.zip
Smoke tests completed. 87 look OK, 2 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_create_pvlan_network | `Error` | 0.05 | test_pvlan.py
test_02_list_snapshots_with_removed_data_store | `Error` | 55.78 | test_snapshots.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-892221412
<b>Trillian test result (tid-1459)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 54947 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1459-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 86 look OK, 3 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_isolate_network_password_server | `Failure` | 216.61 | test_password_server.py
test_02_upgrade_kubernetes_cluster | `Failure` | 3605.11 | test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster | `Error` | 35.93 | test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster | `Failure` | 3616.99 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.21 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 0.08 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 46.63 | test_kubernetes_clusters.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-720282326
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-724463596
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-925380383
<b>Trillian test result (tid-2163)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40616 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t2163-xenserver-71.zip
Smoke tests completed. 87 look OK, 2 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_extract_template | `Failure` | 4.18 | test_templates.py
test_02_cancel_host_maintenace_with_migration_jobs | `Error` | 1512.71 | test_host_maintenance.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517248971
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java
##########
@@ -380,4 +411,59 @@ protected boolean stateTransitTo(long kubernetesClusterId, KubernetesCluster.Eve
return false;
}
}
+
+ protected boolean createCloudStackSecret(String[] keys) {
+ File pkFile = getManagementServerSshPublicKeyFile();
+ Pair<String, Integer> publicIpSshPort = getKubernetesClusterServerIpSshPort(null);
+ publicIpAddress = publicIpSshPort.first();
+ sshPort = publicIpSshPort.second();
+
+ try {
+ Pair<Boolean, String> result = SshHelper.sshExecute(publicIpAddress, sshPort, CLUSTER_NODE_VM_USER,
+ pkFile, null, String.format("/opt/bin/deploy-cloudstack-secret -u '%s' -k '%s' -s '%s'",
Review comment:
@davidjumani - our systemvmtemplate patching also uses `/opt/bin` --> data iso patching doesn't overwrite any scripts or folders?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-921018952
<b>Trillian Build Failed (tid-2095)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911745712
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929108809
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1424
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] vladimirpetrov commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
vladimirpetrov commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929199636
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 closed pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 closed pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-725200086
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895170645
@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-907018875
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894096005
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896470359
<b>Trillian test result (tid-1575)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48806 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1575-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Failure` | 337.06 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Failure` | 432.01 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 125.30 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 167.03 | test_internal_lb.py
test_02_upgrade_kubernetes_cluster | `Error` | 4042.41 | test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster | `Failure` | 351.69 | test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster | `Failure` | 3625.68 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.04 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 0.03 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 94.95 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911377793
<b>Trillian test result (tid-1920)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35034 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1920-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-897368695
<b>Trillian test result (tid-1596)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 57395 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1596-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Smoke tests completed. 85 look OK, 4 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Failure` | 570.15 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Failure` | 603.05 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 260.06 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 326.72 | test_internal_lb.py
test_01_invalid_upgrade_kubernetes_cluster | `Failure` | 153.20 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 3608.99 | test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster | `Failure` | 2081.06 | test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster | `Failure` | 0.04 | test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster | `Failure` | 3.23 | test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster | `Failure` | 0.04 | test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster | `Failure` | 1.20 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.03 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 1.11 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 367.56 | test_kubernetes_clusters.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
test_02_vpc_privategw_static_routes | `Failure` | 142.56 | test_privategw_acl.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695805956
##########
File path: plugins/integrations/kubernetes-service/src/main/resources/conf/k8s-control-node-add.yml
##########
@@ -17,12 +17,16 @@
# under the License.
---
-ssh_authorized_keys:
- {{ k8s.ssh.pub.key }}
-
-write-files:
+users:
+ - name: core
Review comment:
`core` is the username used in coreOS templates, hence the same user is created for backward compatibility
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697286446
##########
File path: server/src/main/java/org/apache/cloudstack/diagnostics/DiagnosticsServiceImpl.java
##########
@@ -341,7 +341,8 @@ private Answer deleteDiagnosticsZipFileInsystemVm(VMInstanceVO vmInstance, Strin
boolean existsInSecondaryStore = dataDirectory.exists() || dataDirectory.mkdir();
if (existsInSecondaryStore) {
// scp from system VM to mounted sec storage directory
- File permKey = new File("/var/cloudstack/management/.ssh/id_rsa");
+ String homeDir = System.getProperty("user.home");
+ File permKey = new File(homeDir + "/.ssh/id_rsa");
Review comment:
Tests advised - run diagnostic tests across all supported distros
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-849285279
Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 98
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894168404
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian. SL-JID 783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-712641997
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2209
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-811138457
Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 301
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-778166255
Packaging result: ✖centos7 ✖centos8 ✔debian. JID-2692
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: [WIP] Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-717823085
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r514204424
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterStartWorker.java
##########
@@ -378,17 +376,18 @@ private void setupKubernetesClusterNetworkRules(Network network, List<UserVm> cl
throw new ManagementServerException(String.format("No source NAT IP addresses found for network : %s, Kubernetes cluster : %s",
network.getName(), kubernetesCluster.getName()));
}
-
+ // Firewall rule fo API access for master node VMs
Review comment:
this comment could be a method name and the bit below factored out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-695923552
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r563556732
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@davidjumani
we do not need docker installed in systemvm template.
it would be better to create separated folder in tools/appliance/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929078700
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929253531
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1425
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] vladimirpetrov commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
vladimirpetrov commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929928897
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-934072111
Ping @alexandremattioli or @andrijapanicsb are you lgtm on this as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-924775032
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894149015
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903434849
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-900853235
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 closed pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 closed pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-801084902
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906491176
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895111302
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian. SL-JID 806
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895135854
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian. SL-JID 808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-898738209
<b>Trillian test result (tid-1625)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 50625 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1625-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Failure` | 366.62 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Failure` | 368.14 | test_routers_network_ops.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 607.92 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 449.74 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Failure` | 391.89 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695799897
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,801 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String HASH_ALGORITHM = "MD5";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ long templateId = -1;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement("select id from `cloud`.`vm_template` where name = ? and removed is null order by id desc limit 1");
+ // Get systemvm template id for corresponding hypervisor
+ pstmt.setString(1, hypervisorAndTemplateName.second());
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ templateId = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "updateSystemVmTemplates: Exception caught while getting ids of SystemVM templates";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ } catch (SQLException e) {
+ String errorMessage = "Unable to upgrade the database";
+ LOGGER.error(errorMessage, e);
+ throw new CloudRuntimeException(errorMessage, e);
+ }
+ return templateId;
+ }
+
+ private static String fetchTemplatesPath() {
+ String filePath = RELATIVE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ File metaFile = new File(filePath);
+ String templatePath = null;
+ if (metaFile.exists()) {
+ templatePath = RELATIVE_TEMPLATE_PATH;
+ }
+ if (templatePath == null) {
+ filePath = ABSOLUTE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ metaFile = new File(filePath);
+ templatePath = ABSOLUTE_TEMPLATE_PATH;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ if (!metaFile.exists()) {
+ String errMsg = String.format("Unable to locate metadata file in your setup at %s", filePath.toString());
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+ }
+ return templatePath;
+ }
+
+ private static String getHypervisorName(String name) {
+ if (name.equals("xenserver")) {
+ return "xen";
+ }
+ if (name.equals("ovm3")) {
+ return "ovm";
+ }
+ return name;
+
+ }
+
+ private static Hypervisor.HypervisorType getHypervisorType(String hypervisor) {
+ if (hypervisor.equalsIgnoreCase("xen")) {
+ hypervisor = "xenserver";
+ } else if (hypervisor.equalsIgnoreCase("ovm")) {
+ hypervisor = "ovm3";
+ }
+ return Hypervisor.HypervisorType.getType(hypervisor);
+ }
+
+ private static List<Long> getEligibleZoneIds(Connection conn) {
+ List<Long> zones = new ArrayList<Long>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_ELIGIBLE_ZONES);
+ ResultSet rs = pstmt.executeQuery();
+ while (rs.next()) {
+ zones.add(rs.getLong(1));
+ }
+ } catch (SQLException e) {
+ String errMsg = "Failed to fetch eligible zones for SystemVM template registration";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return zones;
+ }
+
+ private static Pair<String, Long> getNfsStoreInZone(Connection conn, Long zoneId) {
+ String url = null;
+ Long storeId = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_IMAGE_STORE_PER_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ url = resultSet.getString(1);
+ storeId = resultSet.getLong(2);
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch NFS store in zone = %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ if (url == null || storeId == null) {
+ throw new CloudRuntimeException(String.format("Failed to get an NFS store in zone: %s", zoneId));
+ }
+ return new Pair<>(url, storeId);
+ }
+
+ public static void mountStore(String storeUrl) {
+ try {
+ if (storeUrl != null) {
+ URI uri = new URI(UriUtils.encodeURIComponent(storeUrl));
+ String host = uri.getHost();
+ String mountPath = uri.getPath();
+ Script.runSimpleBashScript("mkdir -p " + TEMPORARY_SECONDARY_STORE);
+ String mount = String.format(MOUNT_COMMAND, host + ":" + mountPath, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(mount);
+ }
+ } catch (Exception e) {
+ String msg = "NFS Store URL is not in the correct format";
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(msg, e);
+
+ }
+ }
+
+ private static List<String> fetchAllHypervisors(Connection conn, Long zoneId) {
+ List<String> hypervisorList = new ArrayList<>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_HYPERVISORS_IN_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ hypervisorList.add(resultSet.getString(1));
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch distinct hypervisors in zone: %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return hypervisorList;
+ }
+
+ private static Long createTemplateObjectInDB(Connection conn, SystemVMTemplateDetails details) {
+ Long id = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_VM_TEMPLATE_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getName());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.setString(i++, details.getChecksum());
+ pstmt.setString(i++, details.getName());
+ pstmt.setString(i++, details.getFormat().toString());
+ pstmt.setLong(i++, details.getGuestOsId());
+ pstmt.setString(i++, details.getHypervisorType().toString());
+ pstmt.setBoolean(i++, details.getHypervisorType() == Hypervisor.HypervisorType.VMware);
+ pstmt.executeUpdate();
+
+ pstmt = conn.prepareStatement("SELECT id FROM vm_template ORDER BY id DESC LIMIT 1");
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ id = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "Failed to fetch template id ";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create vm_template record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ return id;
+ }
+
+ private static void createTemplateStoreRefEntry(Connection conn, SystemVMTemplateDetails details) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setLong(i++, details.getStoreId());
+ pstmt.setLong(i++, details.getId());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getInstallPath());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create template_store_ref record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateDb(Connection conn, SystemVMTemplateDetails details, boolean updateTemplateDetails) {
+ try {
+ int i = 1;
+ PreparedStatement pstmt = null;
+ if (updateTemplateDetails) {
+ pstmt = conn.prepareStatement(UPDATE_VM_TEMPLATE_ENTRY);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ }
+ i = 1;
+ pstmt = conn.prepareStatement(UPDATE_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getPhysicalSize());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to update template_store_ref record for the systemVM template registered for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateSystemVMEntries(Connection conn, Long templateId, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ // update template ID of system Vms
+ try {
+ PreparedStatement update_templ_id_pstmt = conn
+ .prepareStatement("update `cloud`.`vm_instance` set vm_template_id = ? where type <> 'User' and hypervisor_type = ? and removed is NULL");
+ update_templ_id_pstmt.setLong(1, templateId);
+ update_templ_id_pstmt.setString(2, hypervisorAndTemplateName.first().toString());
+ update_templ_id_pstmt.executeUpdate();
+ } catch (SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates:Exception while setting template for %s to %s",hypervisorAndTemplateName.first().toString(), templateId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void updateConfigurationParams(Connection conn, Map<String, String> configParams) {
+ String key = null;
+ String value = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_CONFIGURATION_TABLE);
+ for (Map.Entry<String, String> config : configParams.entrySet()) {
+ key = config.getKey();
+ value = config.getValue();
+ pstmt.setString(1, value);
+ pstmt.setString(2, key);
+ pstmt.executeUpdate();
+ }
+
+ } catch (final SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates: Exception while setting %s to %s ", key, value);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ private static void readTemplateProperties(String path, SystemVMTemplateDetails details) {
+ File tmpFile = new File(path);
+ Long size = null;
+ Long physicalSize = 0L;
+ try (FileReader fr = new FileReader(tmpFile); BufferedReader brf = new BufferedReader(fr);) {
+ String line = null;
+ while ((line = brf.readLine()) != null) {
+ if (line.startsWith("size=")) {
+ physicalSize = Long.parseLong(line.split("=")[1]);
+ } else if (line.startsWith("virtualsize=")) {
+ size = Long.parseLong(line.split("=")[1]);
+ }
+ if (size == null) {
+ size = physicalSize;
+ }
+ }
+ } catch (IOException ex) {
+ LOGGER.warn("Failed to read from template.properties", ex);
+ }
+ details.setSize(size);
+ details.setPhysicalSize(physicalSize);
+ }
+
+ private static void updateTemplateTablesOnFailure(Connection conn, long templateId) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_TEMPLATE_TABLE_ON_FAILURE);
+ if (pstmt != null) {
+ Date removedTime = new Date(DateUtil.currentGMTTime().getTime());
+ pstmt.setDate(1, removedTime);
+ pstmt.setLong(2, templateId);
+ pstmt.executeUpdate();
+ }
+
+ PreparedStatement pstmt1 = conn.prepareStatement(DELETE_TEMPLATE_REF_RECORD_ON_FAILURE);
+ if (pstmt1 != null) {
+ pstmt1.setLong(1, templateId);
+ pstmt1.executeUpdate();
+ }
+ } catch (Exception e) {
+ String errMsg = "updateSystemVmTemplates: Exception while updating vm_template and template_store_ref tables on failure";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void unmountStore() {
+ try {
+ LOGGER.info("Unmounting store");
+ String umountCmd = String.format(UMOUNT_COMMAND, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(umountCmd);
Review comment:
I haven't faced any permission issue during mount/umount - so adding sudo may not be required
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911775818
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929807548
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-910816552
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-930437268
<b>Trillian test result (tid-2234)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36943 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t2234-vmware-65u2.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_03_live_migrate_VM_with_two_data_disks | `Error` | 67.74 | test_vm_life_cycle.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-930436565
<b>Trillian test result (tid-2232)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36825 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t2232-xenserver-71.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_extract_template | `Failure` | 128.33 | test_templates.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-724478054
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2347
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517221576
##########
File path: tools/appliance/systemvmtemplate/template.json
##########
@@ -33,11 +33,11 @@
[ "-smp", "1" ]
],
"format": "qcow2",
- "disk_size": 2500,
+ "disk_size": 10000,
"disk_interface": "virtio",
"net_device": "virtio-net",
- "iso_url": "https://download.cloudstack.org/systemvm/debian/debian-10.5.0-amd64-netinst.iso",
- "iso_checksum": "0a6aee1d9aafc1ed095105c052f9fdd65ed00ea9274188c9cd0072c8e6838ab40e246d45a1e6956d74ef1b04a1fc042151762f25412e9ff0cbf49418eef7992e",
+ "iso_url": "http://sbjenkins-stagingrepo.jenkins.lon/flatcar/debian-10.6.0-amd64-netinst.iso",
Review comment:
Can you fix the url to download.cloudstack.org for 10.6 iso?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-811870650
<b>Trillian test result (tid-321)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31328 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t321-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_scale_vm | `Failure` | 11.28 | test_scale_vm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-804924285
Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 212
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-907425240
<b>Trillian test result (tid-1792)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37144 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1792-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
shwstppr commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517160205
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterManagerImpl.java
##########
@@ -261,60 +269,6 @@ private void logAndThrow(final Level logLevel, final String message, final Excep
logTransitStateAndThrow(logLevel, message, null, null, ex);
}
- private boolean isKubernetesServiceTemplateConfigured(DataCenter zone) {
- // Check Kubernetes VM template for zone
- boolean isHyperVAvailable = false;
- boolean isKVMAvailable = false;
- boolean isVMwareAvailable = false;
- boolean isXenserverAvailable = false;
- List<ClusterVO> clusters = clusterDao.listByZoneId(zone.getId());
- for (ClusterVO clusterVO : clusters) {
- if (Hypervisor.HypervisorType.Hyperv.equals(clusterVO.getHypervisorType())) {
- isHyperVAvailable = true;
- }
- if (Hypervisor.HypervisorType.KVM.equals(clusterVO.getHypervisorType())) {
- isKVMAvailable = true;
- }
- if (Hypervisor.HypervisorType.VMware.equals(clusterVO.getHypervisorType())) {
- isVMwareAvailable = true;
- }
- if (Hypervisor.HypervisorType.XenServer.equals(clusterVO.getHypervisorType())) {
- isXenserverAvailable = true;
- }
- }
- List<Pair<String, String>> templatePairs = new ArrayList<>();
- if (isHyperVAvailable) {
- templatePairs.add(new Pair<>(KubernetesClusterHyperVTemplateName.key(), KubernetesClusterHyperVTemplateName.value()));
- }
- if (isKVMAvailable) {
Review comment:
@Pearl1594 have we put any check or something for people who will upgrade to 4.16 and have existing k8s cluster? They try to scale their cluster cc @davidjumani
##########
File path: tools/appliance/systemvmtemplate/template.json
##########
@@ -33,11 +33,11 @@
[ "-smp", "1" ]
],
"format": "qcow2",
- "disk_size": 2500,
+ "disk_size": 10000,
"disk_interface": "virtio",
"net_device": "virtio-net",
- "iso_url": "https://download.cloudstack.org/systemvm/debian/debian-10.5.0-amd64-netinst.iso",
- "iso_checksum": "0a6aee1d9aafc1ed095105c052f9fdd65ed00ea9274188c9cd0072c8e6838ab40e246d45a1e6956d74ef1b04a1fc042151762f25412e9ff0cbf49418eef7992e",
+ "iso_url": "http://sbjenkins-stagingrepo.jenkins.lon/flatcar/debian-10.6.0-amd64-netinst.iso",
Review comment:
@Pearl1594 @davidjumani you don't need to to put this link here. It can be set in Trillian override code.
##########
File path: server/src/main/java/com/cloud/vm/UserVmManager.java
##########
@@ -55,6 +55,10 @@
static final int MAX_USER_DATA_LENGTH_BYTES = 2048;
+ public static enum UserVmType {
+ CKSNode
+ }
Review comment:
Don't we need to have some default type?
##########
File path: tools/marvin/marvin/config/test_data.py
##########
@@ -2036,46 +2030,11 @@
},
"1.16.3": {
"semanticversion": "1.16.3",
- "url": "http://download.cloudstack.org/cks/setup-1.16.3.iso",
+ "url": "http://sbjenkins-stagingrepo.jenkins.lon/cks/binaries-iso/as-1.16.3.iso",
Review comment:
@Pearl1594 @davidjumani you don't need to to put this link here. It can be set in Trillian override code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901545758
<b>Trillian test result (tid-1700)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38337 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1700-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903453288
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906103246
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903449152
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895785899
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896777232
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901843009
@Pearl1594 a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-773285177
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-778166255
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695792882
##########
File path: debian/rules
##########
@@ -66,17 +66,21 @@ override_dh_auto_install:
mkdir -p $(DESTDIR)/usr/share/$(PACKAGE)-management
mkdir -p $(DESTDIR)/usr/share/$(PACKAGE)-management/lib
mkdir -p $(DESTDIR)/usr/share/$(PACKAGE)-management/setup
+ mkdir -p $(DESTDIR)/usr/share/$(PACKAGE)-management/templates
mkdir $(DESTDIR)/var/log/$(PACKAGE)/management
mkdir $(DESTDIR)/var/cache/$(PACKAGE)/management
mkdir $(DESTDIR)/var/log/$(PACKAGE)/ipallocator
mkdir $(DESTDIR)/var/lib/$(PACKAGE)/management
mkdir $(DESTDIR)/var/lib/$(PACKAGE)/mnt
+
cp -r client/target/utilities/scripts/db/* $(DESTDIR)/usr/share/$(PACKAGE)-management/setup/
cp -r client/target/classes/META-INF/webapp $(DESTDIR)/usr/share/$(PACKAGE)-management/webapp
cp server/target/conf/* $(DESTDIR)/$(SYSCONFDIR)/$(PACKAGE)/server/
cp client/target/conf/* $(DESTDIR)/$(SYSCONFDIR)/$(PACKAGE)/management/
cp client/target/cloud-client-ui-$(VERSION).jar $(DESTDIR)/usr/share/$(PACKAGE)-management/lib/cloudstack-$(VERSION).jar
cp client/target/lib/*jar $(DESTDIR)/usr/share/$(PACKAGE)-management/lib/
+ cp -r engine/schema/dist/systemvm-templates/* $(DESTDIR)/usr/share/$(PACKAGE)-management/templates/
+ rm -rf $(DESTDIR)/usr/share/$(PACKAGE)-management/templates/md5sum.txt
Review comment:
yes - the metatdata.ini file will be present
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695807606
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2638,6 +2643,29 @@ public String getName() {
return null;
}
+ private String getValidTemplateName(Long zoneId, HypervisorType hType) {
+ String templateName = null;
+ switch (hType) {
+ case XenServer:
+ templateName = VirtualNetworkApplianceManager.RouterTemplateXen.valueIn(zoneId);
+ break;
+ case KVM:
+ templateName = VirtualNetworkApplianceManager.RouterTemplateKvm.valueIn(zoneId);
+ break;
+ case VMware:
+ templateName = VirtualNetworkApplianceManager.RouterTemplateVmware.valueIn(zoneId);
+ break;
+ case Hyperv:
+ templateName = VirtualNetworkApplianceManager.RouterTemplateHyperV.valueIn(zoneId);
+ break;
+ case LXC:
+ templateName = VirtualNetworkApplianceManager.RouterTemplateLxc.valueIn(zoneId);
Review comment:
There is different global setting for LXC's template name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895899377
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r514205528
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterStartWorker.java
##########
@@ -565,6 +564,18 @@ public boolean startKubernetesClusterOnCreate() {
if (!isKubernetesClusterDashboardServiceRunning(true, startTimeoutTime)) {
logTransitStateAndThrow(Level.ERROR, String.format("Failed to setup Kubernetes cluster : %s in usable state as unable to get Dashboard service running for the cluster", kubernetesCluster.getName()), kubernetesCluster.getId(),KubernetesCluster.Event.OperationFailed);
}
+ retrieveScriptFiles();
+ for (int i = 0; i < clusterVMs.size(); ++i) {
+ try {
+ copyAutoscalerScripts(publicIpAddress, CLUSTER_NODES_DEFAULT_START_SSH_PORT + i);
+ } catch (Exception e) {
+ throw new CloudRuntimeException(e);
+ }
+ }
+ if (!createCloudStackSecret(keys)) {
+ logTransitStateAndThrow(Level.ERROR, String.format("Failed to setup keys for Kubernetes cluster %s",
+ kubernetesCluster.getName()), kubernetesCluster.getId(),KubernetesCluster.Event.OperationFailed);
+ }
Review comment:
one or two extra methods
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517206804
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41400to41500.sql
##########
@@ -833,3 +833,17 @@ INSERT INTO `cloud`.`guest_os_hypervisor` (uuid,hypervisor_type, hypervisor_vers
-- Fix OS category for Guest OS 'Other PV Virtio-SCSI (64-bit)'
UPDATE `cloud`.`guest_os` SET category_id = 7 WHERE id = 275 AND display_name = 'Other PV Virtio-SCSI (64-bit)';
+
+-- TODO : Move to 4.16
+ALTER TABLE `cloud`.`user_vm` ADD COLUMN `user_vm_type` varchar(255) DEFAULT "UserVM" COMMENT 'Defines the type of UserVM';
+
+UPDATE `cloud`.`vm_template` set deploy_as_is = 1 where id = 8;
Review comment:
Add a comment to explain why this is done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 removed a comment on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 removed a comment on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-908957028
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-909865265
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896469944
<b>Trillian test result (tid-1576)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48834 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1576-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Smoke tests completed. 85 look OK, 4 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Failure` | 582.88 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Failure` | 707.63 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 170.91 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 262.52 | test_internal_lb.py
test_01_invalid_upgrade_kubernetes_cluster | `Failure` | 8.82 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 5.57 | test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster | `Failure` | 829.66 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 84.74 | test_kubernetes_clusters.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 454.78 | test_privategw_acl.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 646.19 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Error` | 419.17 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Failure` | 453.05 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 520.45 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Error` | 520.46 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906442537
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-932896150
The vmware failure is same as in the health check PR, so not caused by this PR. Tests LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-924821993
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] alexandremattioli commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
alexandremattioli commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-933491250
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929807548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-804936657
<b>Trillian Build Failed (tid-235)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-934158120
Ah okay thanks @Pearl1594; just a note to @alexandremattioli - please use Github's review -> LGTM that way it's easy to track approvals.
Let's wait for @andrijapanicsb to confirm as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901120831
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-900991633
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929200581
@vladimirpetrov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-811120496
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-914971289
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901175330
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-887170345
Hi @davidjumani can you please fix the conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-891098266
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894217690
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-781876298
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517214229
##########
File path: systemvm/debian/opt/cloud/bin/setup/CKSNode.sh
##########
@@ -0,0 +1,61 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+. /opt/cloud/bin/setup/common.sh
+
+setup_k8s_node() {
+ log_it "Setting up k8s node"
+
+ # set default ssh port and restart sshd service
+ sed -i 's/3922/22/g' /etc/ssh/sshd_config
+
+ swapoff -a
+ sudo sed -i '/ swap / s/^/#/' /etc/fstab
+ log_it "Swap disabled"
+
+ log_it "Setting up interfaces"
+ setup_common eth0
+ setup_system_rfc1918_internal
+
+ log_it "Setting up entry in hosts"
+ sed -i /$NAME/d /etc/hosts
+ echo "$ETH0_IP $NAME" >> /etc/hosts
+
+ public_ip=`getPublicIp`
+ echo "$public_ip $NAME" >> /etc/hosts
+
+ echo "export PATH='$PATH:/opt/bin/'">> ~/.bashrc
+
+ disable_rpfilter
+ enable_fwding 1
+ enable_irqbalance 0
+ setup_ntp
+ dhclient
Review comment:
note - should we call dhclient with some guarding parameters (timeouts, no. of retries...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517204970
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41400to41500.java
##########
@@ -140,9 +140,9 @@ private void updateSystemVmTemplates(final Connection conn) {
final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>() {
{
- put(Hypervisor.HypervisorType.KVM, "81b3e48bb934784a13555a43c5ef5ffb");
- put(Hypervisor.HypervisorType.XenServer, "1b178a5dbdbe090555515340144c6017");
- put(Hypervisor.HypervisorType.VMware, "e6a88e518c57d6f36c096c4204c3417f");
+ put(Hypervisor.HypervisorType.KVM, "0d95bb3d9385097dec8b485e46eae34b");
Review comment:
Note: after 4.15 is cut; we need to create a new upgrade path from 4.15->4.16 and move these there. cc @Pearl1594
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-763642296
@Pearl1594 @davidjumani pl merge latest master and handle db upgrade paths.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-721053073
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697270693
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,778 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.ClusterDaoImpl;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.DataCenterDaoImpl;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.DataStoreRole;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateStorageResourceAssoc;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateDaoImpl;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.upgrade.dao.BasicTemplateDataStoreDaoImpl;
+import com.cloud.user.Account;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.vm.dao.VMInstanceDaoImpl;
+import org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDao;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreVO;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreVO;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.inject.Inject;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/systemvm/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ public static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARTIAL_TEMPLATE_FOLDER = String.format("/template/tmpl/%d/", Account.ACCOUNT_ID_SYSTEM);
+ private static final String FETCH_REGISTERED_TEMPLATE_INSTALL_PATH = "SELECT install_path FROM `cloud`.`template_store_ref` where template_id = ? LIMIT 1";
+ private static final String storageScriptsDir = "scripts/storage/secondary";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ private static final Integer TOKEN_LENGTH = 10;
+
+ public static String CS_MAJOR_VERSION = null;
+ public static String CS_TINY_VERSION = null;
+
+ @Inject
+ DataCenterDao dataCenterDao;
+ @Inject
+ VMTemplateDao vmTemplateDao;
+ @Inject
+ TemplateDataStoreDao templateDataStoreDao;
+ @Inject
+ VMInstanceDao vmInstanceDao;
+ @Inject
+ ImageStoreDao imageStoreDao;
+ @Inject
+ ClusterDao clusterDao;
+ @Inject
+ ConfigurationDao configurationDao;
+
+ public SystemVmTemplateRegistration() {
+ dataCenterDao = new DataCenterDaoImpl();
+ vmTemplateDao = new VMTemplateDaoImpl();
+ templateDataStoreDao = new BasicTemplateDataStoreDaoImpl();
+ vmInstanceDao = new VMInstanceDaoImpl();
+ imageStoreDao = new ImageStoreDaoImpl();
+ clusterDao = new ClusterDaoImpl();
+ configurationDao = new ConfigurationDaoImpl();
+ }
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> FileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static String generateToken(int length) {
Review comment:
Explore use of Files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 closed pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 closed pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-924821476
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-931446901
<b>Trillian test result (tid-2252)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37957 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t2252-vmware-67u3.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_03_live_migrate_VM_with_two_data_disks | `Error` | 63.03 | test_vm_life_cycle.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929077406
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695810139
##########
File path: tools/appliance/systemvmtemplate/scripts/configure_conntrack.sh
##########
@@ -34,8 +34,6 @@ function load_conntrack_modules() {
grep nf_conntrack_ipv4 /etc/modules && return
cat >> /etc/modules << EOF
-nf_conntrack_ipv4
-nf_conntrack_ipv6
Review comment:
there is only an nf_conntrack module, it is because of these unknown modules that we see an error during systemvm bootup

##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -69,23 +75,33 @@ function install_packages() {
radvd \
sharutils genisoimage \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ conntrack apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common
+
+ apt-get install -y python3-json-pointer python3-jsonschema cloud-init
# python2-netaddr workaround
wget https://github.com/shapeblue/cloudstack-nonoss/raw/main/python-netaddr_0.7.19-1_all.deb
dpkg -i python-netaddr_0.7.19-1_all.deb
- apt-get -y autoremove --purge
- apt-get clean
- apt-get autoclean
+ apt_clean
- #32 bit architecture support for vhd-util: not required for 32 bit template
+ # 32 bit architecture support for vhd-util
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
fi
+ # Install docker and containerd for CKS
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+ add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
+1 - will work on it in the next phase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901090371
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903832115
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-900853235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r696281994
##########
File path: plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtStartCommandWrapper.java
##########
@@ -88,14 +89,7 @@ public Answer execute(final StartCommand command, final LibvirtComputingResource
libvirtComputingResource.applyDefaultNetworkRules(conn, vmSpec, false);
// pass cmdline info to system vms
- if (vmSpec.getType() != VirtualMachine.Type.User) {
- String controlIp = null;
- for (final NicTO nic : vmSpec.getNics()) {
- if (nic.getType() == TrafficType.Control) {
- controlIp = nic.getIp();
- break;
- }
- }
+ if (vmSpec.getType() != VirtualMachine.Type.User || (vmSpec.getBootArgs() != null && vmSpec.getBootArgs().contains(UserVmManager.CKS_NODE))) {
// try to patch and SSH into the systemvm for up to 5 minutes
for (int count = 0; count < 10; count++) {
// wait and try passCmdLine for 30 seconds at most for CLOUDSTACK-2823
Review comment:
Currently, yes, we pass the cmdline args to CKS nodes as we do with systemVMs - and this has been restricted to other userVMs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-908957425
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani edited a comment on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
davidjumani edited a comment on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-828962341
TODO: Remove the templateid and template name from KubernetesClusterResponse and DB since the templates can vary after acs upgrades
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-914910557
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-723851902
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2342
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901842700
@blueorangutan test suse15 kvm-suse15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r693909292
##########
File path: plugins/integrations/kubernetes-service/src/main/java/org/apache/cloudstack/api/command/user/kubernetes/cluster/CreateKubernetesClusterCmd.java
##########
@@ -55,6 +57,7 @@
public class CreateKubernetesClusterCmd extends BaseAsyncCreateCmd {
public static final Logger LOGGER = Logger.getLogger(CreateKubernetesClusterCmd.class.getName());
public static final String APINAME = "createKubernetesCluster";
+ private static final Long DEFAULT_NODE_ROOT_DISK_SIZE = 8L;
Review comment:
We've set it as 8GB as this was the disk size for core os as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896804513
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 842
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903959032
<b>Trillian Build Failed (tid-1751)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r693909881
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterVO.java
##########
@@ -333,6 +369,16 @@ public KubernetesClusterVO(String name, String description, long zoneId, long ku
this.checkForGc = false;
}
+ public KubernetesClusterVO(String name, String description, long zoneId, long kubernetesVersionId, long serviceOfferingId, long templateId,
Review comment:
I'm not sure of it's usage probably @davidjumani can advice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r693910937
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
+ private static final String relativeTemplatePath = "./engine/schema/dist/systemvm-templates/";
+ private static final String AbsolutetemplatesPath = "/usr/share/cloudstack-management/templates/";
+ private static final String templatesPath = fetchTemplatesPath();
+ private static final String metadataFileName = "metadata.ini";
+ private static final String metadataFile = templatesPath + metadataFileName;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
Review comment:
it's only a placeholder - this will automatically be updated during MS restart - if the code version has changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-914970631
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_multiplication_x: suse15. SL-JID 1168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-921014175
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-910818643
@nvazquez a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-811119741
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-808317894
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-712627209
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-724464089
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901131838
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695801672
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,801 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String HASH_ALGORITHM = "MD5";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ long templateId = -1;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement("select id from `cloud`.`vm_template` where name = ? and removed is null order by id desc limit 1");
+ // Get systemvm template id for corresponding hypervisor
+ pstmt.setString(1, hypervisorAndTemplateName.second());
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ templateId = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "updateSystemVmTemplates: Exception caught while getting ids of SystemVM templates";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ } catch (SQLException e) {
+ String errorMessage = "Unable to upgrade the database";
+ LOGGER.error(errorMessage, e);
+ throw new CloudRuntimeException(errorMessage, e);
+ }
+ return templateId;
+ }
+
+ private static String fetchTemplatesPath() {
+ String filePath = RELATIVE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ File metaFile = new File(filePath);
+ String templatePath = null;
+ if (metaFile.exists()) {
+ templatePath = RELATIVE_TEMPLATE_PATH;
+ }
+ if (templatePath == null) {
+ filePath = ABSOLUTE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ metaFile = new File(filePath);
+ templatePath = ABSOLUTE_TEMPLATE_PATH;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ if (!metaFile.exists()) {
+ String errMsg = String.format("Unable to locate metadata file in your setup at %s", filePath.toString());
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+ }
+ return templatePath;
+ }
+
+ private static String getHypervisorName(String name) {
+ if (name.equals("xenserver")) {
+ return "xen";
+ }
+ if (name.equals("ovm3")) {
+ return "ovm";
+ }
+ return name;
+
+ }
+
+ private static Hypervisor.HypervisorType getHypervisorType(String hypervisor) {
+ if (hypervisor.equalsIgnoreCase("xen")) {
+ hypervisor = "xenserver";
+ } else if (hypervisor.equalsIgnoreCase("ovm")) {
+ hypervisor = "ovm3";
+ }
+ return Hypervisor.HypervisorType.getType(hypervisor);
+ }
+
+ private static List<Long> getEligibleZoneIds(Connection conn) {
+ List<Long> zones = new ArrayList<Long>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_ELIGIBLE_ZONES);
+ ResultSet rs = pstmt.executeQuery();
+ while (rs.next()) {
+ zones.add(rs.getLong(1));
+ }
+ } catch (SQLException e) {
+ String errMsg = "Failed to fetch eligible zones for SystemVM template registration";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return zones;
+ }
+
+ private static Pair<String, Long> getNfsStoreInZone(Connection conn, Long zoneId) {
+ String url = null;
+ Long storeId = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_IMAGE_STORE_PER_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ url = resultSet.getString(1);
+ storeId = resultSet.getLong(2);
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch NFS store in zone = %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ if (url == null || storeId == null) {
+ throw new CloudRuntimeException(String.format("Failed to get an NFS store in zone: %s", zoneId));
+ }
+ return new Pair<>(url, storeId);
+ }
+
+ public static void mountStore(String storeUrl) {
+ try {
+ if (storeUrl != null) {
+ URI uri = new URI(UriUtils.encodeURIComponent(storeUrl));
+ String host = uri.getHost();
+ String mountPath = uri.getPath();
+ Script.runSimpleBashScript("mkdir -p " + TEMPORARY_SECONDARY_STORE);
+ String mount = String.format(MOUNT_COMMAND, host + ":" + mountPath, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(mount);
+ }
+ } catch (Exception e) {
+ String msg = "NFS Store URL is not in the correct format";
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(msg, e);
+
+ }
+ }
+
+ private static List<String> fetchAllHypervisors(Connection conn, Long zoneId) {
+ List<String> hypervisorList = new ArrayList<>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_HYPERVISORS_IN_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ hypervisorList.add(resultSet.getString(1));
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch distinct hypervisors in zone: %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return hypervisorList;
+ }
+
+ private static Long createTemplateObjectInDB(Connection conn, SystemVMTemplateDetails details) {
+ Long id = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_VM_TEMPLATE_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getName());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.setString(i++, details.getChecksum());
+ pstmt.setString(i++, details.getName());
+ pstmt.setString(i++, details.getFormat().toString());
+ pstmt.setLong(i++, details.getGuestOsId());
+ pstmt.setString(i++, details.getHypervisorType().toString());
+ pstmt.setBoolean(i++, details.getHypervisorType() == Hypervisor.HypervisorType.VMware);
+ pstmt.executeUpdate();
+
+ pstmt = conn.prepareStatement("SELECT id FROM vm_template ORDER BY id DESC LIMIT 1");
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ id = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "Failed to fetch template id ";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create vm_template record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ return id;
+ }
+
+ private static void createTemplateStoreRefEntry(Connection conn, SystemVMTemplateDetails details) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setLong(i++, details.getStoreId());
+ pstmt.setLong(i++, details.getId());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getInstallPath());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create template_store_ref record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateDb(Connection conn, SystemVMTemplateDetails details, boolean updateTemplateDetails) {
+ try {
+ int i = 1;
+ PreparedStatement pstmt = null;
+ if (updateTemplateDetails) {
+ pstmt = conn.prepareStatement(UPDATE_VM_TEMPLATE_ENTRY);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ }
+ i = 1;
+ pstmt = conn.prepareStatement(UPDATE_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getPhysicalSize());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to update template_store_ref record for the systemVM template registered for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateSystemVMEntries(Connection conn, Long templateId, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ // update template ID of system Vms
+ try {
+ PreparedStatement update_templ_id_pstmt = conn
+ .prepareStatement("update `cloud`.`vm_instance` set vm_template_id = ? where type <> 'User' and hypervisor_type = ? and removed is NULL");
+ update_templ_id_pstmt.setLong(1, templateId);
+ update_templ_id_pstmt.setString(2, hypervisorAndTemplateName.first().toString());
+ update_templ_id_pstmt.executeUpdate();
+ } catch (SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates:Exception while setting template for %s to %s",hypervisorAndTemplateName.first().toString(), templateId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void updateConfigurationParams(Connection conn, Map<String, String> configParams) {
+ String key = null;
+ String value = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_CONFIGURATION_TABLE);
+ for (Map.Entry<String, String> config : configParams.entrySet()) {
+ key = config.getKey();
+ value = config.getValue();
+ pstmt.setString(1, value);
+ pstmt.setString(2, key);
+ pstmt.executeUpdate();
+ }
+
+ } catch (final SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates: Exception while setting %s to %s ", key, value);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ private static void readTemplateProperties(String path, SystemVMTemplateDetails details) {
+ File tmpFile = new File(path);
+ Long size = null;
+ Long physicalSize = 0L;
+ try (FileReader fr = new FileReader(tmpFile); BufferedReader brf = new BufferedReader(fr);) {
+ String line = null;
+ while ((line = brf.readLine()) != null) {
+ if (line.startsWith("size=")) {
+ physicalSize = Long.parseLong(line.split("=")[1]);
+ } else if (line.startsWith("virtualsize=")) {
+ size = Long.parseLong(line.split("=")[1]);
+ }
+ if (size == null) {
+ size = physicalSize;
+ }
+ }
+ } catch (IOException ex) {
+ LOGGER.warn("Failed to read from template.properties", ex);
+ }
+ details.setSize(size);
+ details.setPhysicalSize(physicalSize);
+ }
+
+ private static void updateTemplateTablesOnFailure(Connection conn, long templateId) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_TEMPLATE_TABLE_ON_FAILURE);
+ if (pstmt != null) {
+ Date removedTime = new Date(DateUtil.currentGMTTime().getTime());
+ pstmt.setDate(1, removedTime);
+ pstmt.setLong(2, templateId);
+ pstmt.executeUpdate();
+ }
+
+ PreparedStatement pstmt1 = conn.prepareStatement(DELETE_TEMPLATE_REF_RECORD_ON_FAILURE);
+ if (pstmt1 != null) {
+ pstmt1.setLong(1, templateId);
+ pstmt1.executeUpdate();
+ }
+ } catch (Exception e) {
+ String errMsg = "updateSystemVmTemplates: Exception while updating vm_template and template_store_ref tables on failure";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void unmountStore() {
+ try {
+ LOGGER.info("Unmounting store");
+ String umountCmd = String.format(UMOUNT_COMMAND, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(umountCmd);
+ } catch (Exception e) {
+ String msg = String.format("Failed to unmount store mounted at %s", TEMPORARY_SECONDARY_STORE);
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(msg, e);
+ }
+ }
+
+ private static void setupTemplate(String templateName, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName,
+ String destTempFolder) throws CloudRuntimeException{
+ String storageScriptsDir = "scripts/storage/secondary";
+ String setupTmpltScript = Script.findScript(storageScriptsDir, "setup-sysvm-tmplt");
+ if (setupTmpltScript == null) {
+ throw new CloudRuntimeException("Unable to find the createtmplt.sh");
+ }
+ Script scr = new Script(setupTmpltScript, SCRIPT_TIMEOUT, LOGGER);
Review comment:
It isn't required that the script be run by root user - the necessary command - "touch" has been added to the `/etc/sudoers.d/cloudstack-management` file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695798513
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,801 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String HASH_ALGORITHM = "MD5";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ long templateId = -1;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement("select id from `cloud`.`vm_template` where name = ? and removed is null order by id desc limit 1");
+ // Get systemvm template id for corresponding hypervisor
+ pstmt.setString(1, hypervisorAndTemplateName.second());
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ templateId = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "updateSystemVmTemplates: Exception caught while getting ids of SystemVM templates";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ } catch (SQLException e) {
+ String errorMessage = "Unable to upgrade the database";
+ LOGGER.error(errorMessage, e);
+ throw new CloudRuntimeException(errorMessage, e);
+ }
+ return templateId;
+ }
+
+ private static String fetchTemplatesPath() {
+ String filePath = RELATIVE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ File metaFile = new File(filePath);
+ String templatePath = null;
+ if (metaFile.exists()) {
+ templatePath = RELATIVE_TEMPLATE_PATH;
+ }
+ if (templatePath == null) {
+ filePath = ABSOLUTE_TEMPLATE_PATH + METADATA_FILE_NAME;
+ metaFile = new File(filePath);
+ templatePath = ABSOLUTE_TEMPLATE_PATH;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ if (!metaFile.exists()) {
+ String errMsg = String.format("Unable to locate metadata file in your setup at %s", filePath.toString());
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+ }
+ return templatePath;
+ }
+
+ private static String getHypervisorName(String name) {
+ if (name.equals("xenserver")) {
+ return "xen";
+ }
+ if (name.equals("ovm3")) {
+ return "ovm";
+ }
+ return name;
+
+ }
+
+ private static Hypervisor.HypervisorType getHypervisorType(String hypervisor) {
+ if (hypervisor.equalsIgnoreCase("xen")) {
+ hypervisor = "xenserver";
+ } else if (hypervisor.equalsIgnoreCase("ovm")) {
+ hypervisor = "ovm3";
+ }
Review comment:
the reason for doing this is that templates names don't have "xenserver" / "ovm3" but rather "xen"/"ovm" - hence using this to fetch the template names while parsing the metadata file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895169982
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-713532001
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2251
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-907514094
<b>Trillian test result (tid-1794)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49745 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1794-vmware-65u2.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_07_deploy_kubernetes_ha_cluster | `Failure` | 3615.32 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.06 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 0.04 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 42.04 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901805122
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-713504796
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695804072
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41510to41600.sql
##########
@@ -19,6 +19,19 @@
-- Schema upgrade from 4.15.1.0 to 4.16.0.0
--;
+ALTER TABLE `cloud`.`user_vm` ADD COLUMN `user_vm_type` varchar(255) DEFAULT "UserVM" COMMENT 'Defines the type of UserVM';
Review comment:
The type field is to state if it's a User / System or any other type of VM - CKS Node is a type of user vm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906129515
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-893455081
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-898604358
<b>Trillian test result (tid-1626)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33386 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1626-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-927742341
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-779873133
> is this ready for review and test ?
Sorry for the delay in response @weizhouapache. This PR is ready for testing wrt KVM and Xen, however, we encountered an issue wrt resizing of Volumes with VMware on 4.15, which impacts this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r709840764
##########
File path: engine/schema/src/main/java/com/cloud/dc/dao/ClusterDao.java
##########
@@ -34,6 +35,8 @@
List<HypervisorType> getAvailableHypervisorInZone(Long zoneId);
+ Set<HypervisorType> getDistictAvailableHypervisorsAcrossClusters();
Review comment:
```suggestion
Set<HypervisorType> getDistinctAvailableHypervisorsAcrossClusters();
```
typo ^^^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-891047963
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 727
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906204068
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895205584
<b>Trillian Build Failed (tid-1550)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896489854
test failures wrt k8s/cks pertain to failure in provisioning node VMs as the cluster's memory utilization has exceeded the threshold `cluster.memory.allocated.capacity.disablethreshold ` :
```
Cannot allocate cluster list [1] for vm creation since their allocated percentage crosses the disable capacity threshold defined at each cluster/ at global value for capacity Type : 0, skipping these clusters
....
2021-08-10 18:37:01,372 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-18:ctx-63501faa job-1014/job-1015) (logid:aba1589c) Done executing com.cloud.vm.VmWorkStart for job-1015
2021-08-10 18:37:01,373 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-18:ctx-63501faa job-1014/job-1015) (logid:aba1589c) Remove job-1015 from job monitoring
2021-08-10 18:37:01,384 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-8:ctx-57f0feba job-1014 ctx-30ce85b6) (logid:aba1589c) Scaling failed for Kubernetes cluster : testcluster-LHSA7V, unable to provision node VM in the cluster
com.cloud.exception.ManagementServerException: Failed to start VM in the Kubernetes cluster : testcluster-LHSA7V
at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterResourceModifierActionWorker.startKubernetesVM(KubernetesClusterResourceModifierActionWorker.java:322)
at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterResourceModifierActionWorker.provisionKubernetesClusterNodeVms(KubernetesClusterResourceModifierActionWorker.java:340)
at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleUpKubernetesClusterSize(KubernetesClusterScaleWorker.java:361)
at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleKubernetesClusterSize(KubernetesClusterScaleWorker.java:403)
at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterScaleWorker.scaleCluster(KubernetesClusterScaleWorker.java:437)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895166918
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian. SL-JID 810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903663613
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-905530149
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-778145692
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-721809568
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-808318116
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-804865609
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895873875
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r519632820
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterManagerImpl.java
##########
@@ -261,60 +269,6 @@ private void logAndThrow(final Level logLevel, final String message, final Excep
logTransitStateAndThrow(logLevel, message, null, null, ex);
}
- private boolean isKubernetesServiceTemplateConfigured(DataCenter zone) {
- // Check Kubernetes VM template for zone
- boolean isHyperVAvailable = false;
- boolean isKVMAvailable = false;
- boolean isVMwareAvailable = false;
- boolean isXenserverAvailable = false;
- List<ClusterVO> clusters = clusterDao.listByZoneId(zone.getId());
- for (ClusterVO clusterVO : clusters) {
- if (Hypervisor.HypervisorType.Hyperv.equals(clusterVO.getHypervisorType())) {
- isHyperVAvailable = true;
- }
- if (Hypervisor.HypervisorType.KVM.equals(clusterVO.getHypervisorType())) {
- isKVMAvailable = true;
- }
- if (Hypervisor.HypervisorType.VMware.equals(clusterVO.getHypervisorType())) {
- isVMwareAvailable = true;
- }
- if (Hypervisor.HypervisorType.XenServer.equals(clusterVO.getHypervisorType())) {
- isXenserverAvailable = true;
- }
- }
- List<Pair<String, String>> templatePairs = new ArrayList<>();
- if (isHyperVAvailable) {
- templatePairs.add(new Pair<>(KubernetesClusterHyperVTemplateName.key(), KubernetesClusterHyperVTemplateName.value()));
- }
- if (isKVMAvailable) {
Review comment:
@shwstppr On an upgraded env, the existing cluster will continue to operate as is, and scaling will result in creation of a new debian-based worker node
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-718668386
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-898227884
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-915019809
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1171
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-910733129
<b>Trillian test result (tid-1888)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56535 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1888-vmware-65u2.zip
Smoke tests completed. 89 look OK, 4 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_deploy_vm_start_failure | `Error` | 109.57 | test_deploy_vm.py
test_deploy_vm_volume_creation_failure | `Error` | 135.25 | test_deploy_vm.py
test_vm_ha | `Error` | 89.86 | test_vm_ha.py
test_vm_sync | `Error` | 169.73 | test_vm_sync.py
test_07_reboot_ssvm | `Failure` | 119.47 | test_ssvm.py
test_08_reboot_cpvm | `Failure` | 27.74 | test_ssvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-720344442
@davidjumani can you check Travis job 1, for possible unit test, lint checks and license failures.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-718710699
@davidjumani in the test class tearDownClass method `cleanup_resources(cls.apiclient, cls._cleanup)` should read `cleanup_resources(cls.apiclient, reversed(cls._cleanup))` this can result in garbage left behind and exceptions at the end of the test run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd merged pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd merged pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-920597151
@Pearl1594 can you fix merge conflict and kick pkg + test matrix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-801065181
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r709841952
##########
File path: engine/schema/src/main/java/com/cloud/dc/dao/ClusterDaoImpl.java
##########
@@ -154,6 +161,17 @@ public ClusterVO findBy(String name, long podId) {
return hypers;
}
+ @Override
+ public Set<HypervisorType> getDistictAvailableHypervisorsAcrossClusters() {
+ SearchCriteria<ClusterVO> sc = ClusterSearch.create();
+ List<ClusterVO> clusters = listBy(sc);
+ Set<HypervisorType> hypers = new HashSet<>();
Review comment:
```suggestion
Set<HypervisorType> hypervisorTypes = new HashSet<>();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-920626356
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-811929181
<b>Trillian test result (tid-320)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37358 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t320-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_10_traceroute_in_vr | `Failure` | 61.27 | test_diagnostics.py
ContextSuite context=TestVAppsVM>:setup | `Error` | 45.41 | test_vm_life_cycle.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929929349
@vladimirpetrov a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-801112143
Packaging result: :heavy_check_mark: centos7 :heavy_multiplication_x: centos8 :heavy_check_mark: debian. SL-JID 161
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-773285177
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-720282102
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906153172
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1017
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r696283994
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,801 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String HASH_ALGORITHM = "MD5";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
Review comment:
this has been refactored and handled - thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-891086867
<b>Trillian test result (tid-1453)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 450 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1453-vmware-67u3.zip
Smoke tests completed. 0 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896855952
<b>Trillian Build Failed (tid-1599)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-896777004
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r563657612
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@weizhouapache With coreOS having reached EOL, we've enabled the systemVM template to be consumed by CKS for deploying K8s clusters. Hence docker is installed in the template. However, the service is only run on the cks nodes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906841842
<b>Trillian test result (tid-1782)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33172 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1782-xenserver-71.zip
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-908956101
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1071
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-910612047
<b>Trillian test result (tid-1887)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51080 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1887-kvm-centos7.zip
Smoke tests completed. 90 look OK, 3 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_deploy_vm_start_failure | `Error` | 71.90 | test_deploy_vm.py
test_deploy_vm_volume_creation_failure | `Error` | 127.15 | test_deploy_vm.py
test_vm_ha | `Error` | 60.84 | test_vm_ha.py
test_vm_sync | `Error` | 129.83 | test_vm_sync.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-907412440
<b>Trillian test result (tid-1793)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35867 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1793-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-924774744
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: [WIP] Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-717845057
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2292
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-721808713
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-812008368
<b>Trillian Build Failed (tid-329)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] alexandremattioli commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
alexandremattioli commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-936518914
@nvazquez all good with tests. I've used this for some customer demos and worked very well, retested this week and all good. LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] andrijapanicsb commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-935602414
Alex is reviewing this, so I will not. Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895146792
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-905569680
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1008
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-934073124
@rhtyd - @alexandremattioli has already provided their review - https://github.com/apache/cloudstack/pull/4329#issuecomment-933491250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903872749
<b>Trillian test result (tid-1745)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32714 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1745-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-725626585
<b>Trillian test result (tid-3152)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30520 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t3152-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Smoke tests completed. 84 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestKubernetesSupportedVersion>:setup | `Error` | 0.00 | test_kubernetes_supported_versions.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-723011279
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929838653
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1429
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
davidjumani commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-929077406
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-930782323
@blueorangutan test centos7 vmware-67u3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r565824268
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@weizhouapache I completely understand your concerns. However, the basic intent of moving to systemVM template was that we could leverage the already registered template, thus removing the need to register a new template for CKS. We've ensured that VRs and systemVMs will not be affected by these new packages installed. In case the concern is about the template size, it's increased by 500MB and have incorporated your PR https://github.com/apache/cloudstack/pull/3900 for growing / resizing the root disk partition during boot up to a predefined / provided disk size. We could also probably discuss this on the mailing list to get a feedback from a wider audience and incorporate and test all cases to ensure that there aren't any adverse effects of these changes.. Thanks @weizhouapache
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906203282
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-725211182
Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2362
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r694651365
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterStartWorker.java
##########
@@ -385,39 +385,45 @@ private void setupKubernetesClusterNetworkRules(Network network, List<UserVm> cl
network.getName(), kubernetesCluster.getName()));
}
+ createFirewallRules(publicIp, clusterVMIds);
+
+ // Load balancer rule fo API access for control node VMs
Review comment:
good method name; `LoadBalancerRuleForAPIAccessForControlNodeVMs`
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterStartWorker.java
##########
@@ -385,39 +385,45 @@ private void setupKubernetesClusterNetworkRules(Network network, List<UserVm> cl
network.getName(), kubernetesCluster.getName()));
}
+ createFirewallRules(publicIp, clusterVMIds);
+
+ // Load balancer rule fo API access for control node VMs
+ try {
+ provisionLoadBalancerRule(publicIp, network, owner, clusterVMIds, CLUSTER_API_PORT);
+ } catch (NetworkRuleConflictException | InsufficientAddressCapacityException e) {
+ throw new ManagementServerException(String.format("Failed to provision load balancer rule for API access for the Kubernetes cluster : %s", kubernetesCluster.getName()), e);
+ }
+
+ // Port forwarding rule fo SSH access on each node VM
Review comment:
love this one as well: `PortForwardingRuleForSSHAccessOnEachNodeVM`
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -453,6 +457,12 @@ public void allocate(final String vmInstanceName, final VirtualMachineTemplate t
final VirtualMachineProfileImpl vmProfile = new VirtualMachineProfileImpl(vmFinal, template, serviceOffering, null, null);
+ Long rootDiskSize = rootDiskOfferingInfo.getSize();
+ if (vm.getType().isUsedBySystem() && SystemVmRootDiskSize.value() != null && SystemVmRootDiskSize.value() > 0L) {
+ rootDiskSize = SystemVmRootDiskSize.value();
+ }
+ final Long rootDiskSizeFinal = rootDiskSize;
Review comment:
nice little utility method `getIntendedRootDiskSize(DiskOffering rootDiskOfferingInfo, VirtualMachine vm)` is in order here.
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterResourceModifierActionWorker.java
##########
@@ -548,13 +565,96 @@ protected String getKubernetesClusterNodeNamePrefix() {
return prefix;
}
- protected String getKubernetesClusterNodeAvailableName(final String hostName) {
- String name = hostName;
- int suffix = 1;
- while (vmInstanceDao.findVMByHostName(name) != null) {
- name = String.format("%s-%d", hostName, suffix);
- suffix++;
+ protected KubernetesClusterVO updateKubernetesClusterEntry(final Long cores, final Long memory,
+ final Long size, final Long serviceOfferingId, final Boolean autoscaleEnabled, final Long minSize, final Long maxSize) {
+ return Transaction.execute(new TransactionCallback<KubernetesClusterVO>() {
+ @Override
+ public KubernetesClusterVO doInTransaction(TransactionStatus status) {
+ KubernetesClusterVO updatedCluster = kubernetesClusterDao.createForUpdate(kubernetesCluster.getId());
+ if (cores != null) {
+ updatedCluster.setCores(cores);
+ }
+ if (memory != null) {
+ updatedCluster.setMemory(memory);
+ }
+ if (size != null) {
+ updatedCluster.setNodeCount(size);
+ }
+ if (serviceOfferingId != null) {
+ updatedCluster.setServiceOfferingId(serviceOfferingId);
+ }
+ if (autoscaleEnabled != null) {
+ updatedCluster.setAutoscalingEnabled(autoscaleEnabled.booleanValue());
+ }
+ updatedCluster.setMinSize(minSize);
+ updatedCluster.setMaxSize(maxSize);
+ return kubernetesClusterDao.persist(updatedCluster);
+ }
+ });
+ }
+
+ private KubernetesClusterVO updateKubernetesClusterEntry(final Boolean autoscaleEnabled, final Long minSize, final Long maxSize) throws CloudRuntimeException {
+ KubernetesClusterVO kubernetesClusterVO = updateKubernetesClusterEntry(null, null, null, null, autoscaleEnabled, minSize, maxSize);
+ if (kubernetesClusterVO == null) {
+ logTransitStateAndThrow(Level.ERROR, String.format("Scaling Kubernetes cluster %s failed, unable to update Kubernetes cluster",
+ kubernetesCluster.getName()), kubernetesCluster.getId(), KubernetesCluster.Event.OperationFailed);
+ }
+ return kubernetesClusterVO;
+ }
+
+ protected boolean autoscaleCluster(boolean enable, Long minSize, Long maxSize) {
Review comment:
could this be disected a bit more? at least the `if(enable)` and its `else` block in extra methods?
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,801 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String HASH_ALGORITHM = "MD5";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
Review comment:
rather throw an exception than return `null` imnsho
##########
File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
##########
@@ -2806,7 +2806,12 @@ public Snapshot allocSnapshot(Long volumeId, Long policyId, String snapshotName,
if (volume.getTemplateId() != null) {
VMTemplateVO template = _templateDao.findById(volume.getTemplateId());
- if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM) {
+ Long instanceId = volume.getInstanceId();
+ UserVmVO userVmVO = null;
+ if (instanceId != null) {
+ userVmVO = _userVmDao.findById(instanceId);
+ }
+ if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM && (userVmVO == null || !UserVmManager.CKS_NODE.equals(userVmVO.getUserVmType()))) {
Review comment:
this block is repeated ...
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
Review comment:
let's apply this
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterManagerImpl.java
##########
@@ -864,30 +807,92 @@ private void validateKubernetesClusterScaleParameters(ScaleKubernetesClusterCmd
final Long kubernetesClusterId = cmd.getId();
final Long serviceOfferingId = cmd.getServiceOfferingId();
final Long clusterSize = cmd.getClusterSize();
+ final List<Long> nodeIds = cmd.getNodeIds();
+ final Boolean isAutoscalingEnabled = cmd.isAutoscalingEnabled();
+ final Long minSize = cmd.getMinSize();
+ final Long maxSize = cmd.getMaxSize();
+
if (kubernetesClusterId == null || kubernetesClusterId < 1L) {
throw new InvalidParameterValueException("Invalid Kubernetes cluster ID");
}
+
KubernetesClusterVO kubernetesCluster = kubernetesClusterDao.findById(kubernetesClusterId);
if (kubernetesCluster == null || kubernetesCluster.getRemoved() != null) {
throw new InvalidParameterValueException("Invalid Kubernetes cluster ID");
}
+
final DataCenter zone = dataCenterDao.findById(kubernetesCluster.getZoneId());
if (zone == null) {
logAndThrow(Level.WARN, String.format("Unable to find zone for Kubernetes cluster : %s", kubernetesCluster.getName()));
}
+ if (serviceOfferingId == null && clusterSize == null && nodeIds == null && isAutoscalingEnabled == null) {
+ throw new InvalidParameterValueException(String.format("Kubernetes cluster %s cannot be scaled, either service offering or cluster size or nodeids to be removed or autoscaling must be passed", kubernetesCluster.getName()));
+ }
+
Account caller = CallContext.current().getCallingAccount();
accountManager.checkAccess(caller, SecurityChecker.AccessType.OperateEntry, false, kubernetesCluster);
- if (serviceOfferingId == null && clusterSize == null) {
- throw new InvalidParameterValueException(String.format("Kubernetes cluster : %s cannot be scaled, either a new service offering or a new cluster size must be passed", kubernetesCluster.getName()));
- }
-
final KubernetesSupportedVersion clusterVersion = kubernetesSupportedVersionDao.findById(kubernetesCluster.getKubernetesVersionId());
if (clusterVersion == null) {
throw new CloudRuntimeException(String.format("Invalid Kubernetes version associated with Kubernetes cluster : %s", kubernetesCluster.getName()));
}
+ if (!(kubernetesCluster.getState().equals(KubernetesCluster.State.Created) ||
+ kubernetesCluster.getState().equals(KubernetesCluster.State.Running) ||
+ kubernetesCluster.getState().equals(KubernetesCluster.State.Stopped))) {
Review comment:
and extract in a separate method
##########
File path: server/src/main/java/com/cloud/storage/snapshot/SnapshotManagerImpl.java
##########
@@ -834,7 +835,12 @@ public SnapshotPolicyVO createPolicy(CreateSnapshotPolicyCmd cmd, Account policy
if (volume.getTemplateId() != null) {
VMTemplateVO template = _templateDao.findById(volume.getTemplateId());
- if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM) {
+ Long instanceId = volume.getInstanceId();
+ UserVmVO userVmVO = null;
+ if (instanceId != null) {
+ userVmVO = _vmDao.findById(instanceId);
+ }
+ if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM && (userVmVO == null || !UserVmManager.CKS_NODE.equals(userVmVO.getUserVmType()))) {
Review comment:
this block seems to exist in `VolumeApiServiceImpl` as well (twice)
##########
File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
##########
@@ -2863,7 +2868,12 @@ public Snapshot allocSnapshotForVm(Long vmId, Long volumeId, String snapshotName
if (volume.getTemplateId() != null) {
VMTemplateVO template = _templateDao.findById(volume.getTemplateId());
- if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM) {
+ Long instanceId = volume.getInstanceId();
+ UserVmVO userVmVO = null;
+ if (instanceId != null) {
+ userVmVO = _userVmDao.findById(instanceId);
+ }
+ if (template != null && template.getTemplateType() == Storage.TemplateType.SYSTEM && (userVmVO == null || !UserVmManager.CKS_NODE.equals(userVmVO.getUserVmType()))) {
Review comment:
... here. can you extract?
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2722,6 +2750,46 @@ public ImageStore discoverImageStore(String name, String url, String providerNam
// populate template_store_ref table
_imageSrv.addSystemVMTemplatesToSecondary(store);
_imageSrv.handleTemplateSync(store);
+ if (providerName.equals(DataStoreProvider.NFS_IMAGE) && zoneId != null) {
Review comment:
Can you extract this block
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
+ private static final String relativeTemplatePath = "./engine/schema/dist/systemvm-templates/";
+ private static final String AbsolutetemplatesPath = "/usr/share/cloudstack-management/templates/";
+ private static final String templatesPath = fetchTemplatesPath();
+ private static final String metadataFileName = "metadata.ini";
+ private static final String metadataFile = templatesPath + metadataFileName;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
+ public static String CS_TINY_VERSION = "0";
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails() {
+ }
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> fileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> newTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static boolean validateIfSeeded(String url, String path) {
+ try {
+ mountStore(url);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = TEMPORARY_SECONDARY_STORE + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore();
+ }
+ }
+
+ private static String calculateChecksum(MessageDigest digest, File file) {
+ try {
+ FileInputStream fis = new FileInputStream(file);
+ byte[] byteArray = new byte[1024];
+ int bytesCount = 0;
+
+ while ((bytesCount = fis.read(byteArray)) != -1) {
+ digest.update(byteArray, 0, bytesCount);
+ }
+
+ fis.close();
+ byte[] bytes = digest.digest();
+
+ StringBuilder sb = new StringBuilder();
+ for (byte aByte : bytes) {
+ sb.append(Integer
+ .toString((aByte & 0xff) + 0x100, 16)
+ .substring(1));
+ }
+ return sb.toString();
+ } catch (IOException e) {
+ String errMsg = String.format("Failed to calculate Checksum of template file: %s ", file.getName());
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static long isTemplateAlreadyRegistered(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ long templateId = -1;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement("select id from `cloud`.`vm_template` where name = ? and removed is null order by id desc limit 1");
+ // Get systemvm template id for corresponding hypervisor
+ pstmt.setString(1, hypervisorAndTemplateName.second());
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ templateId = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "updateSystemVmTemplates: Exception caught while getting ids of SystemVM templates";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ } catch (SQLException e) {
+ String errorMessage = "Unable to upgrade the database";
+ LOGGER.error(errorMessage, e);
+ throw new CloudRuntimeException(errorMessage, e);
+ }
+ return templateId;
+ }
+
+ private static String fetchTemplatesPath() {
+ String filePath = relativeTemplatePath + metadataFileName;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ File metaFile = new File(filePath);
+ String templatePath = null;
+ if (metaFile.exists()) {
+ templatePath = relativeTemplatePath;
+ }
+ if (templatePath == null) {
+ filePath = AbsolutetemplatesPath + metadataFileName;
+ metaFile = new File(filePath);
+ templatePath = AbsolutetemplatesPath;
+ LOGGER.debug(String.format("Looking for file [ %s ] in the classpath.", filePath));
+ if (!metaFile.exists()) {
+ String errMsg = String.format("Unable to locate metadata file in your setup at %s", filePath.toString());
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+ }
+ return templatePath;
+ }
+
+ private static String getHypervisorName(String name) {
+ if (name.equals("xenserver")) {
+ return "xen";
+ }
+ if (name.equals("ovm3")) {
+ return "ovm";
+ }
+ return name;
+
+ }
+
+ private static Hypervisor.HypervisorType getHypervisorType(String hypervisor) {
+ if (hypervisor.equalsIgnoreCase("xen")) {
+ hypervisor = "xenserver";
+ } else if (hypervisor.equalsIgnoreCase("ovm")) {
+ hypervisor = "ovm3";
+ }
+ return Hypervisor.HypervisorType.getType(hypervisor);
+ }
+
+ private static List<Long> getEligibleZoneIds(Connection conn) {
+ List<Long> zones = new ArrayList<Long>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_ELIGIBLE_ZONES);
+ ResultSet rs = pstmt.executeQuery();
+ while (rs.next()) {
+ zones.add(rs.getLong(1));
+ }
+ } catch (SQLException e) {
+ String errMsg = "Failed to fetch eligible zones for SystemVM template registration";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return zones;
+ }
+
+ private static Pair<String, Long> getNfsStoreInZone(Connection conn, Long zoneId) {
+ String url = null;
+ Long storeId = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_IMAGE_STORE_PER_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ url = resultSet.getString(1);
+ storeId = resultSet.getLong(2);
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch NFS store in zone = %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ if (url == null || storeId == null) {
+ throw new CloudRuntimeException(String.format("Failed to get an NFS store in zone: %s", zoneId));
+ }
+ return new Pair<>(url, storeId);
+ }
+
+ public static void mountStore(String storeUrl) {
+ try {
+ if (storeUrl != null) {
+ URI uri = new URI(UriUtils.encodeURIComponent(storeUrl));
+ String host = uri.getHost();
+ String mountPath = uri.getPath();
+ Script.runSimpleBashScript("mkdir -p " + TEMPORARY_SECONDARY_STORE);
+ String mount = String.format(mountCommand, host + ":" + mountPath, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(mount);
+ }
+ } catch (Exception e) {
+ String msg = "NFS Store URL is not in the correct format";
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(msg, e);
+
+ }
+ }
+
+ private static List<String> fetchAllHypervisors(Connection conn, Long zoneId) {
+ List<String> hypervisorList = new ArrayList<>();
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(FETCH_DISTINCT_HYPERVISORS_IN_ZONE);
+ if(pstmt != null) {
+ pstmt.setLong(1, zoneId);
+ ResultSet resultSet = pstmt.executeQuery();
+ while (resultSet.next()) {
+ hypervisorList.add(resultSet.getString(1));
+ }
+ }
+ } catch (SQLException e) {
+ String errMsg = String.format("Failed to fetch distinct hypervisors in zone: %s for SystemVM template registration", zoneId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ return hypervisorList;
+ }
+
+ private static Long createTemplateObjectInDB(Connection conn, SystemVMTemplateDetails details) {
+ Long id = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_VM_TEMPLATE_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getUuid());
+ pstmt.setString(i++, details.getName());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.setString(i++, details.getChecksum());
+ pstmt.setString(i++, details.getName());
+ pstmt.setString(i++, details.getFormat().toString());
+ pstmt.setLong(i++, details.getGuestOsId());
+ pstmt.setString(i++, details.getHypervisorType().toString());
+ pstmt.setBoolean(i++, details.getHypervisorType() == Hypervisor.HypervisorType.VMware);
+ pstmt.executeUpdate();
+
+ pstmt = conn.prepareStatement("SELECT id FROM vm_template ORDER BY id DESC LIMIT 1");
+ try (ResultSet rs = pstmt.executeQuery()) {
+ if (rs.next()) {
+ id = rs.getLong(1);
+ }
+ } catch (final SQLException e) {
+ String errMsg = "Failed to fetch template id ";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create vm_template record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ return id;
+ }
+
+ private static void createTemplateStoreRefEntry(Connection conn, SystemVMTemplateDetails details) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(INSERT_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ int i = 1;
+ pstmt.setLong(i++, details.getStoreId());
+ pstmt.setLong(i++, details.getId());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setDate(i++, details.getCreated());
+ pstmt.setString(i++, details.getInstallPath());
+ pstmt.setString(i++, details.getUrl());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to create template_store_ref record for the systemVM template for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateDb(Connection conn, SystemVMTemplateDetails details) {
+ try {
+ int i = 1;
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_VM_TEMPLATE_ENTRY);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ i = 1;
+ pstmt = conn.prepareStatement(UPDATE_TEMPLATE_STORE_REF_TABLE);
+ if (pstmt != null) {
+ pstmt.setLong(i++, details.getSize());
+ pstmt.setLong(i++, details.getPhysicalSize());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setDate(i++, details.getUpdated());
+ pstmt.setLong(i++, details.getId());
+ pstmt.executeUpdate();
+ }
+ } catch (Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to update template_store_ref record for the systemVM template registered for hypervisor: %s", details.getHypervisorType().name()), e);
+ }
+ }
+
+ public static void updateSystemVMEntries(Connection conn, Long templateId, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ // update template ID of system Vms
+ try {
+ PreparedStatement update_templ_id_pstmt = conn
+ .prepareStatement("update `cloud`.`vm_instance` set vm_template_id = ? where type <> 'User' and hypervisor_type = ? and removed is NULL");
+ update_templ_id_pstmt.setLong(1, templateId);
+ update_templ_id_pstmt.setString(2, hypervisorAndTemplateName.first().toString());
+ update_templ_id_pstmt.executeUpdate();
+ } catch (SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates:Exception while setting template for %s to %s",hypervisorAndTemplateName.first().toString(), templateId);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void updateConfigurationParams(Connection conn, Map<String, String> configParams) {
+ String key = null;
+ String value = null;
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_CONFIGURATION_TABLE);
+ for (Map.Entry<String, String> config : configParams.entrySet()) {
+ key = config.getKey();
+ value = config.getValue();
+ pstmt.setString(1, value);
+ pstmt.setString(2, key);
+ pstmt.executeUpdate();
+ }
+
+ } catch (final SQLException e) {
+ String errMsg = String.format("updateSystemVmTemplates: Exception while setting %s to %s ", key, value);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ private static void readTemplateProperties(String path, SystemVMTemplateDetails details) {
+ File tmpFile = new File(path);
+ Long size = null;
+ Long physicalSize = 0L;
+ try (FileReader fr = new FileReader(tmpFile); BufferedReader brf = new BufferedReader(fr);) {
+ String line = null;
+ while ((line = brf.readLine()) != null) {
+ if (line.startsWith("size=")) {
+ physicalSize = Long.parseLong(line.split("=")[1]);
+ } else if (line.startsWith("virtualsize=")) {
+ size = Long.parseLong(line.split("=")[1]);
+ }
+ if (size == null) {
+ size = physicalSize;
+ }
+ }
+ } catch (IOException ex) {
+ LOGGER.warn("Failed to read from template.properties", ex);
+ }
+ details.setSize(size);
+ details.setPhysicalSize(physicalSize);
+ }
+
+ private static void updateTemplateTablesOnFailure(Connection conn, long templateId) {
+ try {
+ PreparedStatement pstmt = conn.prepareStatement(UPDATE_TEMPLATE_TABLE_ON_FAILURE);
+ if (pstmt != null) {
+ Date removedTime = new Date(DateUtil.currentGMTTime().getTime());
+ pstmt.setDate(1, removedTime);
+ pstmt.setLong(2, templateId);
+ pstmt.executeUpdate();
+ }
+
+ PreparedStatement pstmt1 = conn.prepareStatement(DELETE_TEMPLATE_REF_RECORD_ON_FAILURE);
+ if (pstmt1 != null) {
+ pstmt1.setLong(1, templateId);
+ pstmt1.executeUpdate();
+ }
+ } catch (Exception e) {
+ String errMsg = "updateSystemVmTemplates: Exception while updating vm_template and template_store_ref tables on failure";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void unmountStore() {
+ try {
+ LOGGER.info("Unmounting store");
+ String umountCmd = String.format(umountCommand, TEMPORARY_SECONDARY_STORE);
+ Script.runSimpleBashScript(umountCmd);
+ } catch (Exception e) {
+ String msg = String.format("Failed to unmount store mounted at %s", TEMPORARY_SECONDARY_STORE);
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(msg, e);
+ }
+ }
+ public static void registerTemplate(Connection conn, Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName, Pair<String, Long> storeUrlAndId) {
+ Long templateId = null;
+ try {
+ Hypervisor.HypervisorType hypervisor = hypervisorAndTemplateName.first();
+ final String templateName = UUID.randomUUID().toString();
+ Date created = new Date(DateUtil.currentGMTTime().getTime());
+ SystemVMTemplateDetails details = new SystemVMTemplateDetails(templateName, hypervisorAndTemplateName.second(), created,
+ newTemplateUrl.get(hypervisor), newTemplateChecksum.get(hypervisor), hypervisorImageFormat.get(hypervisor), hypervisorGuestOsMap.get(hypervisor), hypervisor, storeUrlAndId.second());
+ templateId = createTemplateObjectInDB(conn, details);
+ if (templateId == null) {
+ throw new CloudRuntimeException(String.format("Failed to register template for hypervisor: %s", hypervisor.name()));
+ }
+ details.setId(templateId);
+ String destTempFolderName = String.valueOf(templateId);
+ String destTempFolder = PARENT_TEMPLATE_FOLDER + PARTIAL_TEMPLATE_FOLDER + destTempFolderName;
+ details.setInstallPath(PARTIAL_TEMPLATE_FOLDER + destTempFolderName + File.separator + templateName + "." + hypervisorImageFormat.get(hypervisor).getFileExtension());
+ createTemplateStoreRefEntry(conn, details);
+ String storageScriptsDir = "scripts/storage/secondary";
+ String setupTmpltScript = Script.findScript(storageScriptsDir, "setup-sysvm-tmplt");
+ if (setupTmpltScript == null) {
+ throw new ConfigurationException("Unable to find the createtmplt.sh");
+ }
+ Script scr = new Script(setupTmpltScript, SCRIPT_TIMEOUT, LOGGER);
+ scr.add("-u", templateName);
+ scr.add("-f", templatesPath + fileNames.get(hypervisorAndTemplateName.first()));
+ scr.add("-h", hypervisorAndTemplateName.first().name().toLowerCase(Locale.ROOT));
+ scr.add("-d", destTempFolder);
+ String result = scr.execute();
+ if (result != null) {
+ String errMsg = String.format("failed to create template: %s ", result);
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+ details.setInstallPath(PARTIAL_TEMPLATE_FOLDER + destTempFolderName + File.separator + templateName + "." + hypervisorImageFormat.get(hypervisor).getFileExtension());
+ readTemplateProperties(destTempFolder + "/template.properties", details);
+ details.setUpdated(new Date(DateUtil.currentGMTTime().getTime()));
+ updateDb(conn, details);
+ Map<String, String> configParams = new HashMap<>();
+ configParams.put(SystemVmTemplateRegistration.routerTemplateConfigurationNames.get(hypervisorAndTemplateName.first()), hypervisorAndTemplateName.second());
+ configParams.put("minreq.sysvmtemplate.version", CS_MAJOR_VERSION + "." + CS_TINY_VERSION);
+ updateConfigurationParams(conn, configParams);
+ updateSystemVMEntries(conn, templateId, hypervisorAndTemplateName);
+ } catch (Exception e) {
+ String errMsg = String.format("Failed to register template for hypervisor: %s", hypervisorAndTemplateName.first());
+ LOGGER.error(errMsg, e);
+ if (templateId != null) {
+ updateTemplateTablesOnFailure(conn, templateId);
+ cleanupStore(templateId);
+ }
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public static void parseMetadataFile() {
+ try {
+ Ini ini = new Ini();
+ ini.load(new FileReader(metadataFile));
+ for (Hypervisor.HypervisorType hypervisorType : hypervisorList) {
+ String hypervisor = hypervisorType.name().toLowerCase(Locale.ROOT);
+ Ini.Section section = ini.get(hypervisor);
+ NewTemplateNameList.put(hypervisorType, section.get("templatename"));
+ fileNames.put(hypervisorType, section.get("filename"));
+ newTemplateChecksum.put(hypervisorType, section.get("checksum"));
+ newTemplateUrl.put(hypervisorType, section.get("downloadurl"));
+ }
+ } catch (Exception e) {
+ String errMsg = String.format("Failed to parse systemVM template metadata file: %s", metadataFile);
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ private static void cleanupStore(Long templateId) {
+ String destTempFolder = PARENT_TEMPLATE_FOLDER + PARTIAL_TEMPLATE_FOLDER + String.valueOf(templateId);
+ Script.runSimpleBashScript("rm -rf " + destTempFolder);
+ }
+
+ public static void registerTemplates(Connection conn, Set<Hypervisor.HypervisorType> hypervisorsInUse) {
+ GlobalLock lock = GlobalLock.getInternLock("UpgradeDatabase-Lock");
+ try {
+ LOGGER.info("Grabbing lock to register templates.");
+ if (!lock.lock(LOCK_WAIT_TIMEOUT)) {
+ throw new CloudRuntimeException("Unable to acquire lock to register SystemVM template.");
+ }
+ // Check if templates path exists
+ try {
+ Set<String> hypervisors = hypervisorsInUse.stream().map(Enum::name).
+ map(name -> name.toLowerCase(Locale.ROOT)).map(SystemVmTemplateRegistration::getHypervisorName).collect(Collectors.toSet());
+ List<String> templates = new ArrayList<>();
+ for (Hypervisor.HypervisorType hypervisorType : hypervisorsInUse) {
+ templates.add(fileNames.get(hypervisorType));
+ }
+
+ boolean templatesFound = true;
+ for (String hypervisor : hypervisors) {
+ String matchedTemplate = templates.stream().filter(x -> x.contains(hypervisor)).findAny().orElse(null);
+ if (matchedTemplate == null) {
+ templatesFound = false;
+ break;
+ }
+ MessageDigest mdigest = MessageDigest.getInstance(hashAlgorithm);
+ File tempFile = new File(templatesPath + matchedTemplate);
+ String templateChecksum = calculateChecksum(mdigest, tempFile);
+ if (!templateChecksum.equals(newTemplateChecksum.get(getHypervisorType(hypervisor)))) {
+ LOGGER.error(String.format("Checksum mismatch: %s != %s ", templateChecksum, newTemplateChecksum.get(getHypervisorType(hypervisor))));
+ templatesFound = false;
+ break;
+ }
+ }
+
+ if (!templatesFound) {
+ String errMsg = "SystemVm template not found. Cannot upgrade system Vms";
+ LOGGER.error(errMsg);
+ throw new CloudRuntimeException(errMsg);
+ }
+
+ // Perform Registration if templates not already registered
Review comment:
`PerformRegistration()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-901530835
<b>Trillian test result (tid-1701)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34468 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1701-kvm-centos7.zip
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695806267
##########
File path: plugins/integrations/kubernetes-service/src/main/resources/conf/k8s-control-node.yml
##########
@@ -17,10 +17,14 @@
# under the License.
---
-ssh_authorized_keys:
- {{ k8s.ssh.pub.key }}
-
-write-files:
+users:
+ - name: core
Review comment:
as mentioned above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911842718
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-911778189
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-766596849
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
shwstppr commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r693855982
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -453,6 +457,12 @@ public void allocate(final String vmInstanceName, final VirtualMachineTemplate t
final VirtualMachineProfileImpl vmProfile = new VirtualMachineProfileImpl(vmFinal, template, serviceOffering, null, null);
+ Long rootDiskSize = rootDiskOfferingInfo.getSize();
+ if (vm.getType().isUsedBySystem() && SystemVmRootDiskSize.value() != null && SystemVmRootDiskSize.value() > 0L) {
Review comment:
I think `SystemVmRootDiskSize.value()` will always give a non-null
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -405,6 +405,10 @@
static final ConfigKey<Boolean> HaVmRestartHostUp = new ConfigKey<Boolean>("Advanced", Boolean.class, "ha.vm.restart.hostup", "true",
"If an out-of-band stop of a VM is detected and its host is up, then power on the VM", true);
+ static final ConfigKey<Long> SystemVmRootDiskSize = new ConfigKey<Long>("Advanced",
+ Long.class, "systemvm.root.disk.size", "-1",
+ "root size (in GB) of systemvm and virtual routers", true);
Review comment:
```suggestion
"Size of root volume (in GB) of system VMs and virtual routers", true);
```
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
+ private static final String relativeTemplatePath = "./engine/schema/dist/systemvm-templates/";
+ private static final String AbsolutetemplatesPath = "/usr/share/cloudstack-management/templates/";
+ private static final String templatesPath = fetchTemplatesPath();
+ private static final String metadataFileName = "metadata.ini";
+ private static final String metadataFile = templatesPath + metadataFileName;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
Review comment:
Why different naming scheme for different variables with same access?
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
Review comment:
Not good for people with OCD :-D
```suggestion
private static final String hashAlgorithm = "MD5";
```
##########
File path: plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
##########
@@ -1667,6 +1667,7 @@ public boolean passCmdLine(final String vmName, final String cmdLine) throws Int
command.add("-n", vmName);
command.add("-c", cmdLine);
result = command.execute();
+
Review comment:
probably not needed
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java
##########
@@ -310,6 +333,7 @@ protected String getControlVmPrivateIp() {
}
protected void attachIsoKubernetesVMs(List<UserVm> clusterVMs, final KubernetesSupportedVersion kubernetesSupportedVersion) throws CloudRuntimeException {
+ //final long startTimeoutTime = System.currentTimeMillis() + KubernetesClusterService.KubernetesClusterStartTimeout.value() * 1000;
Review comment:
needed?
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,759 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.naming.ConfigurationException;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.net.URI;
+import java.security.MessageDigest;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String mountCommand = "sudo mount -t nfs %s %s";
+ private static final String umountCommand = "sudo umount %s";
+ private static final String hashAlgorithm = "MD5";
+ private static final String relativeTemplatePath = "./engine/schema/dist/systemvm-templates/";
+ private static final String AbsolutetemplatesPath = "/usr/share/cloudstack-management/templates/";
+ private static final String templatesPath = fetchTemplatesPath();
+ private static final String metadataFileName = "metadata.ini";
+ private static final String metadataFile = templatesPath + metadataFileName;
+ private static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARENT_TEMPLATE_FOLDER = TEMPORARY_SECONDARY_STORE;
+ private static final String PARTIAL_TEMPLATE_FOLDER = "/template/tmpl/1/";
+ private static final String FETCH_DISTINCT_ELIGIBLE_ZONES = "SELECT DISTINCT(data_center_id) FROM `cloud`.`image_store` WHERE protocol = \"nfs\" AND role = \"Image\" AND removed is null";
+ private static final String FETCH_DISTINCT_HYPERVISORS_IN_ZONE = "SELECT DISTINCT(hypervisor_type) FROM `cloud`.`cluster` where removed is null AND data_center_id=?";
+ private static final String FETCH_IMAGE_STORE_PER_ZONE = "SELECT url,id FROM `cloud`.`image_store` WHERE data_center_id=? AND role = \"Image\" AND image_provider_name = \"NFS\" AND removed IS NULL LIMIT 1";
+ private static final String INSERT_VM_TEMPLATE_TABLE = "INSERT INTO `cloud`.`vm_template` (uuid, unique_name, name, public, featured, created, type, hvm, bits, account_id, url, checksum, enable_password, display_text, format, guest_os_id, cross_zones, hypervisor_type, state, deploy_as_is)" +
+ "VALUES (?, ?, ?, 0, 0, ?, 'SYSTEM', 0, 64, 1, ?, ?, 0, ?, ?, ?, 1, ?, 'Inactive', ?)";
+ private static final String INSERT_TEMPLATE_STORE_REF_TABLE = "INSERT INTO `cloud`.`template_store_ref` (store_id, template_id, created, last_updated, job_id, download_pct, download_state, error_str, local_path, install_path, url, state, destroyed, is_copy," +
+ " update_count, ref_cnt, store_role) VALUES (?, ?, ?, ?, NULL, 0, 'NOT_DOWNLOADED', NULL, NULL, ?, ?, 'Allocated', 0, 0, 0, 0, 'Image')";
+ private static final String UPDATE_TEMPLATE_STORE_REF_TABLE = "UPDATE `cloud`.`template_store_ref` SET download_pct=100, download_state='DOWNLOADED', " +
+ "state='Ready', size=?, physical_size=?, last_updated=?, updated=? where template_id=?";
+ private static final String UPDATE_VM_TEMPLATE_ENTRY = "UPDATE `cloud`.`vm_template` set size = ?, state = 'Active' where id = ?";
+ private static final String UPDATE_CONFIGURATION_TABLE = "UPDATE `cloud`.`configuration` SET value = ? WHERE name = ?";
+ private static final String UPDATE_TEMPLATE_TABLE_ON_FAILURE = "UPDATE `cloud`.`vm_template` set removed = ?, state = 'Inactive' where id = ?";
+ private static final String DELETE_TEMPLATE_REF_RECORD_ON_FAILURE = "DELETE from `cloud`.`template_store_ref` where template_id = ?";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ public static String CS_MAJOR_VERSION = "4.16";
Review comment:
We will update this default value for every version in future?
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterVO.java
##########
@@ -333,6 +369,16 @@ public KubernetesClusterVO(String name, String description, long zoneId, long ku
this.checkForGc = false;
}
+ public KubernetesClusterVO(String name, String description, long zoneId, long kubernetesVersionId, long serviceOfferingId, long templateId,
Review comment:
Is this used anywhere? Can't find its usage in my IDE
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterManagerImpl.java
##########
@@ -864,30 +807,92 @@ private void validateKubernetesClusterScaleParameters(ScaleKubernetesClusterCmd
final Long kubernetesClusterId = cmd.getId();
final Long serviceOfferingId = cmd.getServiceOfferingId();
final Long clusterSize = cmd.getClusterSize();
+ final List<Long> nodeIds = cmd.getNodeIds();
+ final Boolean isAutoscalingEnabled = cmd.isAutoscalingEnabled();
+ final Long minSize = cmd.getMinSize();
+ final Long maxSize = cmd.getMaxSize();
+
if (kubernetesClusterId == null || kubernetesClusterId < 1L) {
throw new InvalidParameterValueException("Invalid Kubernetes cluster ID");
}
+
KubernetesClusterVO kubernetesCluster = kubernetesClusterDao.findById(kubernetesClusterId);
if (kubernetesCluster == null || kubernetesCluster.getRemoved() != null) {
throw new InvalidParameterValueException("Invalid Kubernetes cluster ID");
}
+
final DataCenter zone = dataCenterDao.findById(kubernetesCluster.getZoneId());
if (zone == null) {
logAndThrow(Level.WARN, String.format("Unable to find zone for Kubernetes cluster : %s", kubernetesCluster.getName()));
}
+ if (serviceOfferingId == null && clusterSize == null && nodeIds == null && isAutoscalingEnabled == null) {
+ throw new InvalidParameterValueException(String.format("Kubernetes cluster %s cannot be scaled, either service offering or cluster size or nodeids to be removed or autoscaling must be passed", kubernetesCluster.getName()));
+ }
+
Account caller = CallContext.current().getCallingAccount();
accountManager.checkAccess(caller, SecurityChecker.AccessType.OperateEntry, false, kubernetesCluster);
- if (serviceOfferingId == null && clusterSize == null) {
- throw new InvalidParameterValueException(String.format("Kubernetes cluster : %s cannot be scaled, either a new service offering or a new cluster size must be passed", kubernetesCluster.getName()));
- }
-
final KubernetesSupportedVersion clusterVersion = kubernetesSupportedVersionDao.findById(kubernetesCluster.getKubernetesVersionId());
if (clusterVersion == null) {
throw new CloudRuntimeException(String.format("Invalid Kubernetes version associated with Kubernetes cluster : %s", kubernetesCluster.getName()));
}
+ if (!(kubernetesCluster.getState().equals(KubernetesCluster.State.Created) ||
+ kubernetesCluster.getState().equals(KubernetesCluster.State.Running) ||
+ kubernetesCluster.getState().equals(KubernetesCluster.State.Stopped))) {
Review comment:
Use list `'contains`?
##########
File path: api/src/main/java/com/cloud/vm/UserVmService.java
##########
@@ -379,7 +379,7 @@ UserVm createAdvancedVirtualMachine(DataCenter zone, ServiceOffering serviceOffe
String hostName, String displayName, Long diskOfferingId, Long diskSize, String group, HypervisorType hypervisor, HTTPMethod httpmethod, String userData,
String sshKeyPair, Map<Long, IpAddresses> requestedIps, IpAddresses defaultIps, Boolean displayVm, String keyboard, List<Long> affinityGroupIdList,
Map<String, String> customParameters, String customId, Map<String, Map<Integer, String>> dhcpOptionMap, Map<Long, DiskOffering> dataDiskTemplateToDiskOfferingMap,
- Map<String, String> templateOvfPropertiesMap, boolean dynamicScalingEnabled)
+ Map<String, String> templateOvfPropertiesMap, boolean dynamicScalingEnabled, String type)
Review comment:
Can `type` be an enum value?
##########
File path: plugins/integrations/kubernetes-service/src/main/java/org/apache/cloudstack/api/command/user/kubernetes/cluster/CreateKubernetesClusterCmd.java
##########
@@ -55,6 +57,7 @@
public class CreateKubernetesClusterCmd extends BaseAsyncCreateCmd {
public static final Logger LOGGER = Logger.getLogger(CreateKubernetesClusterCmd.class.getName());
public static final String APINAME = "createKubernetesCluster";
+ private static final Long DEFAULT_NODE_ROOT_DISK_SIZE = 8L;
Review comment:
Is this some tested value?
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/KubernetesClusterManagerImpl.java
##########
@@ -261,60 +269,6 @@ private void logAndThrow(final Level logLevel, final String message, final Excep
logTransitStateAndThrow(logLevel, message, null, null, ex);
}
- private boolean isKubernetesServiceTemplateConfigured(DataCenter zone) {
- // Check Kubernetes VM template for zone
- boolean isHyperVAvailable = false;
- boolean isKVMAvailable = false;
- boolean isVMwareAvailable = false;
- boolean isXenserverAvailable = false;
- List<ClusterVO> clusters = clusterDao.listByZoneId(zone.getId());
- for (ClusterVO clusterVO : clusters) {
- if (Hypervisor.HypervisorType.Hyperv.equals(clusterVO.getHypervisorType())) {
- isHyperVAvailable = true;
- }
- if (Hypervisor.HypervisorType.KVM.equals(clusterVO.getHypervisorType())) {
- isKVMAvailable = true;
- }
- if (Hypervisor.HypervisorType.VMware.equals(clusterVO.getHypervisorType())) {
- isVMwareAvailable = true;
- }
- if (Hypervisor.HypervisorType.XenServer.equals(clusterVO.getHypervisorType())) {
- isXenserverAvailable = true;
- }
- }
- List<Pair<String, String>> templatePairs = new ArrayList<>();
- if (isHyperVAvailable) {
- templatePairs.add(new Pair<>(KubernetesClusterHyperVTemplateName.key(), KubernetesClusterHyperVTemplateName.value()));
- }
- if (isKVMAvailable) {
Review comment:
@Pearl1594 a follow up question on this - I've a running k8s cluster. I upgrade ACS to 4.16. I scale-up my k8s cluster. Now there will be mixed node VMs (with coreos and systemvm templates). Now when I try to upgrade k8s version on my k8s cluster will there be an issue? As user on these VMs may be different `core` and `root`.
##########
File path: engine/schema/src/main/java/org/apache/cloudstack/storage/datastore/db/ImageStoreVO.java
##########
@@ -135,6 +135,10 @@ public Long getDataCenterId() {
return this.dcId;
}
+ public Long getDcId() {
+ return this.dcId;
+ }
+
Review comment:
Using `getDataCenterId` won't work?
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java
##########
@@ -203,11 +218,19 @@ protected void logTransitStateDetachIsoAndThrow(final Level logLevel, final Stri
throw new CloudRuntimeException(message, e);
}
+ protected void deleteTemplateLaunchPermission() {
+ if (clusterTemplate != null && owner != null) {
+ LOGGER.info("Revoking launch permission for systemVM template");
+ launchPermissionDao.removePermissions(clusterTemplate.getId(), Collections.singletonList(owner.getId()));
+ }
+ }
+
protected void logTransitStateAndThrow(final Level logLevel, final String message, final Long kubernetesClusterId, final KubernetesCluster.Event event, final Exception e) throws CloudRuntimeException {
logMessage(logLevel, message, e);
if (kubernetesClusterId != null && event != null) {
stateTransitTo(kubernetesClusterId, event);
}
+ deleteTemplateLaunchPermission();
Review comment:
Is it possible to call it when needed? Not sure but as I can understand this might be called everytime there is an issue the k8s cluster such as upgrade/start (from stopped state) as well. Revoking permission log can be confusing in those cases. Though not a major issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-804904022
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r695808853
##########
File path: server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
##########
@@ -4179,6 +4179,7 @@ public UserVmVO doInTransaction(TransactionStatus status) throws InsufficientCap
}
}
+ vm.setUserVmType(type);
Review comment:
cks nodes are at the end of the day UserVMs - hence, CKSNode userVM type is to further categorize the user vm
##########
File path: server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
##########
@@ -4603,12 +4604,54 @@ private UserVm startVirtualMachine(long vmId, Long podId, Long clusterId, Long h
return vm;
}
+ private void addUserVMCmdlineArgs(Long vmId, VirtualMachineProfile profile, DeployDestination dest, StringBuilder buf) {
+ UserVmVO k8sVM = _vmDao.findById(vmId);
+ buf.append(" template=domP");
+ buf.append(" name=").append(profile.getHostName());
+ buf.append(" type=").append(k8sVM.getUserVmType());
+ for (NicProfile nic : profile.getNics()) {
+ int deviceId = nic.getDeviceId();
+ if (nic.getIPv4Address() == null) {
+ buf.append(" eth").append(deviceId).append("ip=").append("0.0.0.0");
Review comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-927674818
ping @davidjumani @Pearl1594 can you resolve conflicts and kick tests again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-914909763
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697271548
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,778 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.ClusterDaoImpl;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.DataCenterDaoImpl;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.DataStoreRole;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateStorageResourceAssoc;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateDaoImpl;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.upgrade.dao.BasicTemplateDataStoreDaoImpl;
+import com.cloud.user.Account;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.vm.dao.VMInstanceDaoImpl;
+import org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDao;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreVO;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreVO;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.inject.Inject;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.Date;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
+ private static final Logger LOGGER = Logger.getLogger(SystemVmTemplateRegistration.class);
+ private static final String MOUNT_COMMAND = "sudo mount -t nfs %s %s";
+ private static final String UMOUNT_COMMAND = "sudo umount %s";
+ private static final String RELATIVE_TEMPLATE_PATH = "./engine/schema/dist/systemvm-templates/";
+ private static final String ABSOLUTE_TEMPLATE_PATH = "/usr/share/cloudstack-management/templates/systemvm/";
+ private static final String TEMPLATES_PATH = fetchTemplatesPath();
+ private static final String METADATA_FILE_NAME = "metadata.ini";
+ private static final String METADATA_FILE = TEMPLATES_PATH + METADATA_FILE_NAME;
+ public static final String TEMPORARY_SECONDARY_STORE = "/tmp/tmpSecStorage";
+ private static final String PARTIAL_TEMPLATE_FOLDER = String.format("/template/tmpl/%d/", Account.ACCOUNT_ID_SYSTEM);
+ private static final String FETCH_REGISTERED_TEMPLATE_INSTALL_PATH = "SELECT install_path FROM `cloud`.`template_store_ref` where template_id = ? LIMIT 1";
+ private static final String storageScriptsDir = "scripts/storage/secondary";
+ private static final Integer SCRIPT_TIMEOUT = 1800000;
+ private static final Integer LOCK_WAIT_TIMEOUT = 1200;
+ private static final Integer TOKEN_LENGTH = 10;
+
+ public static String CS_MAJOR_VERSION = null;
+ public static String CS_TINY_VERSION = null;
+
+ @Inject
+ DataCenterDao dataCenterDao;
+ @Inject
+ VMTemplateDao vmTemplateDao;
+ @Inject
+ TemplateDataStoreDao templateDataStoreDao;
+ @Inject
+ VMInstanceDao vmInstanceDao;
+ @Inject
+ ImageStoreDao imageStoreDao;
+ @Inject
+ ClusterDao clusterDao;
+ @Inject
+ ConfigurationDao configurationDao;
+
+ public SystemVmTemplateRegistration() {
+ dataCenterDao = new DataCenterDaoImpl();
+ vmTemplateDao = new VMTemplateDaoImpl();
+ templateDataStoreDao = new BasicTemplateDataStoreDaoImpl();
+ vmInstanceDao = new VMInstanceDaoImpl();
+ imageStoreDao = new ImageStoreDaoImpl();
+ clusterDao = new ClusterDaoImpl();
+ configurationDao = new ConfigurationDaoImpl();
+ }
+
+ private static class SystemVMTemplateDetails {
+ Long id;
+ String uuid;
+ String name;
+ String uniqueName;
+ Date created;
+ String url;
+ String checksum;
+ ImageFormat format;
+ Integer guestOsId;
+ Hypervisor.HypervisorType hypervisorType;
+ Long storeId;
+ Long size;
+ Long physicalSize;
+ String installPath;
+ boolean deployAsIs;
+ Date updated;
+
+ SystemVMTemplateDetails(String uuid, String name, Date created, String url, String checksum,
+ ImageFormat format, Integer guestOsId, Hypervisor.HypervisorType hypervisorType,
+ Long storeId) {
+ this.uuid = uuid;
+ this.name = name;
+ this.created = created;
+ this.url = url;
+ this.checksum = checksum;
+ this.format = format;
+ this.guestOsId = guestOsId;
+ this.hypervisorType = hypervisorType;
+ this.storeId = storeId;
+ }
+
+ public void setId(Long id) {
+ this.id = id;
+ }
+
+ public Long getId() {
+ return id;
+ }
+
+ public String getUuid() {
+ return uuid;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public Date getCreated() {
+ return created;
+ }
+
+ public String getUrl() {
+ return url;
+ }
+
+ public String getChecksum() {
+ return checksum;
+ }
+
+ public ImageFormat getFormat() {
+ return format;
+ }
+
+ public Integer getGuestOsId() {
+ return guestOsId;
+ }
+
+ public Hypervisor.HypervisorType getHypervisorType() {
+ return hypervisorType;
+ }
+
+ public Long getStoreId() {
+ return storeId;
+ }
+
+ public Long getSize() {
+ return size;
+ }
+
+ public void setSize(Long size) {
+ this.size = size;
+ }
+
+ public Long getPhysicalSize() {
+ return physicalSize;
+ }
+
+ public void setPhysicalSize(Long physicalSize) {
+ this.physicalSize = physicalSize;
+ }
+
+ public String getInstallPath() {
+ return installPath;
+ }
+
+ public void setInstallPath(String installPath) {
+ this.installPath = installPath;
+ }
+
+ public String getUniqueName() {
+ return uniqueName;
+ }
+
+ public void setUniqueName(String uniqueName) {
+ this.uniqueName = uniqueName;
+ }
+
+ public boolean isDeployAsIs() {
+ return deployAsIs;
+ }
+
+ public void setDeployAsIs(boolean deployAsIs) {
+ this.deployAsIs = deployAsIs;
+ }
+
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+ }
+
+ public static final List<Hypervisor.HypervisorType> hypervisorList = Arrays.asList(Hypervisor.HypervisorType.KVM,
+ Hypervisor.HypervisorType.VMware,
+ Hypervisor.HypervisorType.XenServer,
+ Hypervisor.HypervisorType.Hyperv,
+ Hypervisor.HypervisorType.LXC,
+ Hypervisor.HypervisorType.Ovm3
+ );
+
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateNameList = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> FileNames = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateUrl = new HashMap<Hypervisor.HypervisorType, String>();
+ public static final Map<Hypervisor.HypervisorType, String> NewTemplateChecksum = new HashMap<Hypervisor.HypervisorType, String>();
+
+ public static final Map<Hypervisor.HypervisorType, String> routerTemplateConfigurationNames = new HashMap<Hypervisor.HypervisorType, String>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, "router.template.kvm");
+ put(Hypervisor.HypervisorType.VMware, "router.template.vmware");
+ put(Hypervisor.HypervisorType.XenServer, "router.template.xenserver");
+ put(Hypervisor.HypervisorType.Hyperv, "router.template.hyperv");
+ put(Hypervisor.HypervisorType.LXC, "router.template.lxc");
+ put(Hypervisor.HypervisorType.Ovm3, "router.template.ovm3");
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, Integer> hypervisorGuestOsMap = new HashMap<Hypervisor.HypervisorType, Integer>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, 15);
+ put(Hypervisor.HypervisorType.XenServer, 99);
+ put(Hypervisor.HypervisorType.VMware, 99);
+ put(Hypervisor.HypervisorType.Hyperv, 15);
+ put(Hypervisor.HypervisorType.LXC, 15);
+ put(Hypervisor.HypervisorType.Ovm3, 183);
+ }
+ };
+
+ public static final Map<Hypervisor.HypervisorType, ImageFormat> hypervisorImageFormat = new HashMap<Hypervisor.HypervisorType, ImageFormat>() {
+ {
+ put(Hypervisor.HypervisorType.KVM, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.XenServer, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.VMware, ImageFormat.OVA);
+ put(Hypervisor.HypervisorType.Hyperv, ImageFormat.VHD);
+ put(Hypervisor.HypervisorType.LXC, ImageFormat.QCOW2);
+ put(Hypervisor.HypervisorType.Ovm3, ImageFormat.RAW);
+ }
+ };
+
+ public static String generateToken(int length) {
+ String charset = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+ Random rand = new Random(System.currentTimeMillis());
+ StringBuffer sb = new StringBuffer();
+ for (int i = 0; i < length; i++) {
+ int pos = rand.nextInt(charset.length());
+ sb.append(charset.charAt(pos));
+ }
+ return sb.toString();
+ }
+
+ public static boolean validateIfSeeded(String url, String path) {
+ String filePath = TEMPORARY_SECONDARY_STORE + generateToken(TOKEN_LENGTH);
+ try {
+ mountStore(url, filePath);
+ int lastIdx = path.lastIndexOf(File.separator);
+ String partialDirPath = path.substring(0, lastIdx);
+ String templatePath = filePath + File.separator + partialDirPath;
+ File templateProps = new File(templatePath + "/template.properties");
+ if (templateProps.exists()) {
+ LOGGER.info("SystemVM template already seeded, skipping registration");
+ return true;
+ }
+ LOGGER.info("SystemVM template not seeded");
+ return false;
+ } catch (Exception e) {
+ LOGGER.error("Failed to verify if the template is seeded", e);
+ throw new CloudRuntimeException("Failed to verify if the template is seeded", e);
+ } finally {
+ unmountStore(filePath);
+ }
+ }
+
+ private String calculateChecksum(File file) {
+ try (InputStream is = Files.newInputStream(Paths.get(file.getPath()))) {
+ return DigestUtils.md5Hex(is);
+ } catch (IOException e) {
+ String errMsg = "Failed to calculate template checksum";
+ LOGGER.error(errMsg, e);
+ throw new CloudRuntimeException(errMsg, e);
+ }
+ }
+
+ public long getRegisteredTemplateId(Pair<Hypervisor.HypervisorType, String> hypervisorAndTemplateName) {
+ VMTemplateVO vmTemplate = vmTemplateDao.findLatestTemplateByName(hypervisorAndTemplateName.second());
+ long templateId = -1;
Review comment:
Nit - Magic no. check can be avoided if primitive fields are not used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-927740682
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1407
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-927702937
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r709849274
##########
File path: engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java
##########
@@ -0,0 +1,849 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.upgrade;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.dc.dao.ClusterDaoImpl;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.dc.dao.DataCenterDaoImpl;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.DataStoreRole;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.ImageFormat;
+import com.cloud.storage.VMTemplateStorageResourceAssoc;
+import com.cloud.storage.VMTemplateVO;
+import com.cloud.storage.dao.VMTemplateDao;
+import com.cloud.storage.dao.VMTemplateDaoImpl;
+import com.cloud.template.VirtualMachineTemplate;
+import com.cloud.upgrade.dao.BasicTemplateDataStoreDaoImpl;
+import com.cloud.user.Account;
+import com.cloud.utils.DateUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.UriUtils;
+import com.cloud.utils.db.GlobalLock;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.Script;
+import com.cloud.vm.dao.VMInstanceDao;
+import com.cloud.vm.dao.VMInstanceDaoImpl;
+import org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDao;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreDaoImpl;
+import org.apache.cloudstack.storage.datastore.db.ImageStoreVO;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreDao;
+import org.apache.cloudstack.storage.datastore.db.TemplateDataStoreVO;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.log4j.Logger;
+import org.ini4j.Ini;
+
+import javax.inject.Inject;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.Date;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+public class SystemVmTemplateRegistration {
Review comment:
```suggestion
public class SystemVmTemplate {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r517205707
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41400to41500.sql
##########
@@ -833,3 +833,17 @@ INSERT INTO `cloud`.`guest_os_hypervisor` (uuid,hypervisor_type, hypervisor_vers
-- Fix OS category for Guest OS 'Other PV Virtio-SCSI (64-bit)'
UPDATE `cloud`.`guest_os` SET category_id = 7 WHERE id = 275 AND display_name = 'Other PV Virtio-SCSI (64-bit)';
+
+-- TODO : Move to 4.16
Review comment:
Note - we need to move it to a 4.15->4.16 path in future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-914971566
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-908957425
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-920664398
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-805396454
<b>Trillian test result (tid-237)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37846 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t237-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 83 look OK, 3 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_10_traceroute_in_vr | `Failure` | 60.84 | test_diagnostics.py
test_01_basic_lifecycle_kubernetes_cluster | `Failure` | 52.68 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 60.58 | test_kubernetes_clusters.py
test_03_invalid_upgrade_kubernetes_cluster | `Failure` | 47.16 | test_kubernetes_clusters.py
test_04_scale_kubernetes_cluster | `Failure` | 54.37 | test_kubernetes_clusters.py
test_05_autoscale_kubernetes_cluster | `Failure` | 60.54 | test_kubernetes_clusters.py
test_06_deploy_kubernetes_ha_cluster | `Failure` | 54.30 | test_kubernetes_clusters.py
test_07_upgrade_kubernetes_ha_cluster | `Failure` | 48.12 | test_kubernetes_clusters.py
test_08_delete_kubernetes_ha_cluster | `Failure` | 50.18 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 268.89 | test_kubernetes_clusters.py
ContextSuite context=TestVAppsVM>:setup | `Error` | 43.38 | test_vm_life_cycle.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-893481372
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-894053888
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-910708474
<b>Trillian test result (tid-1886)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55192 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1886-xenserver-71.zip
Smoke tests completed. 90 look OK, 3 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_deploy_vm_start_failure | `Error` | 156.08 | test_deploy_vm.py
test_deploy_vm_volume_creation_failure | `Error` | 179.42 | test_deploy_vm.py
test_vm_ha | `Error` | 85.21 | test_vm_ha.py
test_vm_sync | `Error` | 166.79 | test_vm_sync.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-921013545
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903663046
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895098975
@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-778168653
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r563719624
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@Pearl1594
we could have different template for cks. debian9 is one of an options for cks template. if cloudstack users want to have centos8 or ubuntu 2004 as cks template,we could build them as well.
we can create separated folder in tools/appliance/, like cks-template-debian10, cks-template-centos8, etc
if we use same template on VR and cks, and we need to make more changes in the template, it will impact the systemvm and vrs as well. cloudstack systemvm template is quite stable now. do not touch it if possible.
in addition, some packages in systemvm template are not needed for cks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-897299766
<b>Trillian test result (tid-1598)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44933 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4329-t1598-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 84 look OK, 5 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_add_primary_storage_disabled_host | `Error` | 0.58 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.11 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.19 | test_primary_storage.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Failure` | 316.51 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Failure` | 415.96 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 114.15 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Failure` | 169.33 | test_internal_lb.py
test_02_list_snapshots_with_removed_data_store | `Error` | 1.20 | test_snapshots.py
test_01_secure_vm_migration | `Error` | 197.51 | test_vm_life_cycle.py
test_02_unsecure_vm_migration | `Error` | 274.30 | test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration | `Error` | 146.99 | test_vm_life_cycle.py
test_08_migrate_vm | `Error` | 43.71 | test_vm_life_cycle.py
test_hostha_kvm_host_degraded | `Error` | 708.93 | test_hostha_kvm.py
test_hostha_kvm_host_fencing | `Error` | 675.09 | test_hostha_kvm.py
test_hostha_kvm_host_recovering | `Error` | 727.74 | test_hostha_kvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
rhtyd commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r697291233
##########
File path: ui/src/components/view/ResourceView.vue
##########
@@ -135,27 +135,8 @@ export default {
this.$emit('onTabChange', key)
},
showTab (tab) {
- if ('networkServiceFilter' in tab) {
- if (this.resource && this.resource.virtualmachineid && !this.resource.vpcid && tab.name !== 'firewall') {
- return false
- }
- if (this.resource && this.resource.virtualmachineid && this.resource.vpcid) {
- return false
- }
- // dont display any option for source NAT IP of VPC
- if (this.resource && this.resource.vpcid && !this.resource.issourcenat && tab.name !== 'firewall') {
- return true
- }
- // display LB and PF options for isolated networks if static nat is disabled
- if (this.resource && !this.resource.vpcid) {
- if (!this.resource.isstaticnat) {
- return true
- } else if (tab.name === 'firewall') {
- return true
- }
- }
- return this.networkService && this.networkService.service &&
- tab.networkServiceFilter(this.networkService.service)
+ if (this.networkService && this.networkService.service && tab.networkServiceFilter) {
Review comment:
@davidjumani can you check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-900991404
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
shwstppr commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r693882850
##########
File path: plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java
##########
@@ -203,11 +218,19 @@ protected void logTransitStateDetachIsoAndThrow(final Level logLevel, final Stri
throw new CloudRuntimeException(message, e);
}
+ protected void deleteTemplateLaunchPermission() {
+ if (clusterTemplate != null && owner != null) {
+ LOGGER.info("Revoking launch permission for systemVM template");
+ launchPermissionDao.removePermissions(clusterTemplate.getId(), Collections.singletonList(owner.getId()));
+ }
+ }
+
protected void logTransitStateAndThrow(final Level logLevel, final String message, final Long kubernetesClusterId, final KubernetesCluster.Event event, final Exception e) throws CloudRuntimeException {
logMessage(logLevel, message, e);
if (kubernetesClusterId != null && event != null) {
stateTransitTo(kubernetesClusterId, event);
}
+ deleteTemplateLaunchPermission();
Review comment:
Is it possible to call it only when needed? Not sure but as I can understand this might be called everytime there is an issue the k8s cluster such as upgrade/start (from stopped state) as well. Revoking permission log can be confusing in those cases. Though not a major issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#discussion_r563556732
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@davidjumani
we do not need docker installed in systemvm template.
it would be better to create separated folder in tools/appliance/
##########
File path: tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh
##########
@@ -70,20 +76,33 @@ function install_packages() {
radvd \
sharutils genisoimage aria2 \
strongswan libcharon-extra-plugins libstrongswan-extra-plugins strongswan-charon strongswan-starter \
- virt-what open-vm-tools qemu-guest-agent hyperv-daemons
+ virt-what open-vm-tools qemu-guest-agent hyperv-daemons cloud-guest-utils \
+ apt-transport-https ca-certificates curl gnupg gnupg-agent software-properties-common cloud-init
apt-get -y autoremove --purge
apt-get clean
apt-get autoclean
+ apt_clean
${apt_get} install links
+ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
+ apt-key fingerprint 0EBFCD88
+
#32 bit architecture support for vhd-util: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install libuuid1:i386 libc6:i386
+
+ add-apt-repository \
+ "deb [arch=amd64] https://download.docker.com/linux/debian \
+ $(lsb_release -cs) \
+ stable"
+ apt-get update
+ ${apt_get} install docker-ce docker-ce-cli containerd.io
Review comment:
@Pearl1594
we could have different template for cks. debian9 is one of an options for cks template. if cloudstack users want to have centos8 or ubuntu 2004 as cks template,we could build them as well.
we can create separated folder in tools/appliance/, like cks-template-debian10, cks-template-centos8, etc
if we use same template on VR and cks, and we need to make more changes in the template, it will impact the systemvm and vrs as well. cloudstack systemvm template is quite stable now. do not touch it if possible.
in addition, some packages in systemvm template are not needed for cks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906480441
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-909865265
@Pearl1594 a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-895874642
@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-906981057
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] Pearl1594 commented on pull request #4329: Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements
Posted by GitBox <gi...@apache.org>.
Pearl1594 commented on pull request #4329:
URL: https://github.com/apache/cloudstack/pull/4329#issuecomment-903706164
@shwstppr wrt your comment - https://github.com/apache/cloudstack/pull/4329#discussion_r693886064 - the new systemvm template when used for deploying CKS clusters will create a user named `core` so the upgrade should work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org