You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@vcl.apache.org by Jonathan Casco <jc...@fiu.edu> on 2018/02/14 16:56:00 UTC
Question on xCAT implementation in VCL
Hello,
I am reading through the documentation for xCAT in VCL but was a little confused on the purpose xCAT has here.
Is xCAT being used for provisioning VM hosts like VMware to then get managed by a separate VCL server or does VCL use xCAT to create servers that would be used for VCL guests?
Thank you,
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
Re: Question on xCAT implementation in VCL
Posted by Jonathan Casco <jc...@fiu.edu>.
Thank you for the comprehensive response Andy!
I will be trying this out and report back later on how it works. This is a huge help!
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
From: Andy Kurth <an...@ncsu.edu>
Reply-To: "user@vcl.apache.org" <us...@vcl.apache.org>
Date: Thursday, February 15, 2018 at 1:02 PM
To: "user@vcl.apache.org" <us...@vcl.apache.org>
Subject: Re: Question on xCAT implementation in VCL
You've probably already done this, but run:
copycds VMware-VMvisor-Installer-201701001-4887370.x86_64.iso
This should have created:
/install/esxi6.5/x86_64/
You're correct that recent xCAT doesn't seem to create an osimage automatically by running copycds. To create an osimage manually, first create a stanza file:
vi /root/esxi6.5-x86_64-install-hypervisor.stanza
Add this:
>>> BEGIN
# <xCAT data object stanza file>
esxi6.5-x86_64-install-hypervisor:
objtype=osimage
imagetype=linux
osarch=x86_64
osname=Linux
osvers=esxi6.5
otherpkgdir=/install/post/otherpkgs/esxi6.5/x86_64
profile=hypervisor
provmethod=install
template=/opt/xcat/share/xcat/install/esxi/hypervisor.tmpl
<<< END
Create the osimage from the stanza file:
cat /root/esxi6.5-x86_64-install-hypervisor.stanza | mkdef -z
View the osimage settings:
lsdef -t osimage esxi6.5-x86_64-install-hypervisor
If you want to change any the osimage settings, do so like this:
chdef -t osimage esxi6.5-x86_64-install-hypervisor template=/opt/xcat/share/xcat/install/esxi/<MY CUSTOM>.tmpl
The template file used will be the stock one. Make sure it exists:
/opt/xcat/share/xcat/install/esxi/hypervisor.tmpl
You need to make sure all of the xCAT node/osimage/etc properties used in the template are defined. If not, you'll get an error when you run the nodeset command later on.
For example, the stock template sets the ESXi root password via this line in hypervisor.tmpl:
rootpw --iscrypted #CRYPT:passwd:key=vmware,username=root:password#
I ran a quick test installing a VM with ESXi 6.5 via xCAT and it seems the password behavior changed with 6.5. I had to use the following to set an encrypted password in xCAT. (Plain text in xCAT's passwd table used to work) Without the openssh part, the installation failed with an error related to a bad "crypted password":
chtab key=vmware passwd.username=root passwd.password=`openssl passwd -1 '<MY $TRONG PASSWORD>'`
You'll need to add a node in xCAT, set up the network table, get DHCP working, and a few other things. These are pretty much stock xCAT procedures so I won't get into them. Reply if you need help though.
Set the node to use the osimage:
nodeset <NODE> osimage=esxi6.5-x86_64-install-hypervisor
Again, if you get errors it probably means one or more of the xCAT settings are missing for the node or network.
Before trying to install, verify node's PXE files look correct:
/tftpboot/xcat/xnba/nodes/<NODE>
/tftpboot/xcat/xnba/nodes/<NODE>.pxelinux
/tftpboot/xcat/xnba/nodes/<NODE>.uefi
The main file that will need tweaking is the template file generated for the node by nodeset (derived from /opt/xcat/share/xcat/install/esxi/hypervisor.tmpl):
/install/autoinst/<NODE>
If you get the node to install and want to work on tweaking the config, what I usually do is edit the node's template file directly and then restart/PXE boot/reinstall it. I don't rerun nodeset in between or else the node's template file will be overwritten with the osimage template settings.
Once I have the configuration correct, I'll rework the osimage template using the appropriate xCAT variable substitutions.
Below is an old custom template file used for ESXi 6.0. It certainly needs some tweaking but may give you some pointers on how to configure things. The commented out lines at the beginning are used for debugging problems. Even though commented out, the xCAT variables get substituted when you run nodeset. I'll put in lines for most of the variables used later on to make it easy to see if a value is missing or incorrect. After ESXi is installed, we'll run some custom scripts stored on a NAS datastore which configure the firewall and do some other tweaks not stored in the template.
>>> BEGIN
# node name: '#TABLE:nodetype:$NODE:node#'
# node private IP address: '#TABLE:hosts:$NODE:ip#'
# node private network: '#TABLE:networks:netname=private:net#'
# node private subnet mask: '#TABLE:networks:netname=private:mask#'
# node private MAC address: '#TABLE:mac:$NODE:mac#'
# nameservers: '#TABLE:site:key=nameservers:value#'
# domain: '#TABLE:site:key=domain:value#'
# management node: '#TABLE:site:key=master:value#'
# TFTP server: '#TABLE:noderes:$NODE:tftpserver#'
# default gateway: '#COMMAND: route -n |grep '^0\.0\.0\.0' |awk '{print $2}' #'
# Install NIC: 'vmnic#COMMAND: echo -n #TABLE:noderes:$NODE:installnic# | grep -o "[0-9]"#'
# Primary NIC: 'vmnic#COMMAND: echo -n #TABLE:noderes:$NODE:primarynic# | grep -o "[0-9]"#'
vmaccepteula
rootpw --iscrypted #CRYPT:passwd:key=vmware,username=root:password#
clearpart --firstdisk=USB-IBM,usb,mptsas,mpt2sas,local --overwritevmfs
install --firstdisk=USB-IBM,usb,mptsas,mpt2sas,local --overwritevmfs
serialnum --esx=#TABLE:prodkey:$NODE,product=esxi6.free:key#
network --bootproto=dhcp --device=vmnic#COMMAND: echo -n #TABLE:noderes:$NODE:installnic# | grep -o "[0-9]"# --addvmportgroup=0
reboot
# =================================================================================================
%pre --interpreter=busybox
# =================================================================================================
%post --interpreter=busybox --ignorefailure=true
localcli network firewall unload
sleep 5
echo -e "<xcatrequest>\n<command>nextdestiny</command>\n</xcatrequest>" | /bin/openssl s_client -quiet -connect #TABLE:site:key=master:value#:3001 2>&1 | tee /tmp/foo.log
# =================================================================================================
%firstboot --interpreter=busybox
#esxcfg-advcfg -s 1 /Net/FollowHardwareMac
#sleep 60
INSTALL_NIC=vmnic$(echo -n #TABLE:noderes:$NODE:installnic# | grep -o "[0-9]")
PRIMARY_NIC=vmnic$(echo -n #TABLE:noderes:$NODE:primarynic# | grep -o "[0-9]")
INSTALL_MAC=$(esxcli network nic list | grep $INSTALL_NIC | grep -E -o '([0-9a-f]{2}:){5}[0-9a-f]{2}')
PRIMARY_MAC=$(esxcli network nic list | grep $PRIMARY_NIC | grep -E -o '([0-9a-f]{2}:){5}[0-9a-f]{2}')
PRIVATE_IP=#TABLE:hosts:$NODE:ip#
PRIVATE_MASK=#TABLE:networks:netname=private:mask#
echo "INSTALL_NIC: ${INSTALL_NIC}"
echo "INSTALL_MAC: ${INSTALL_MAC}"
echo "PRIMARY_NIC: ${PRIMARY_NIC}"
echo "PRIMARY_MAC: ${PRIMARY_MAC}"
#----------------------------
#esxcli network vswitch standard remove --vswitch-name=vSwitchUSB0
esxcli network ip interface remove --interface-name vmk0
esxcli network vswitch standard portgroup remove --portgroup-name "Management Network" --vswitch-name vSwitch0
esxcli network vswitch standard add --vswitch-name vSwitch1
esxcli network vswitch standard uplink add --uplink-name ${PRIMARY_NIC} --vswitch-name vSwitch1
esxcli network vswitch standard portgroup add --portgroup-name VMkernel-Public --vswitch-name vSwitch1
esxcli network ip interface add --interface-name vmk2 --portgroup-name VMkernel-Public --mac-address=${PRIMARY_MAC}
esxcli network ip interface ipv4 set --interface-name vmk2 --type dhcp
esxcli network ip interface tag add -i vmk2 -t Management
esxcli network vswitch standard portgroup add --portgroup-name Public --vswitch-name vSwitch1
esxcli network vswitch standard portgroup add --portgroup-name VMkernel-Private --vswitch-name vSwitch0
esxcli network ip interface add --interface-name vmk0 --portgroup-name VMkernel-Private --mac-address=${INSTALL_MAC}
esxcli network ip interface ipv4 set --interface-name vmk0 --ipv4 ${PRIVATE_IP} --netmask ${PRIVATE_MASK} --type static
esxcli network ip interface tag remove --interface-name vmk0 --tagname Management
esxcli network vswitch standard portgroup add --portgroup-name Private --vswitch-name vSwitch0
esxcfg-route esxcfg-route -a default #COMMAND: route -n |grep '^0\.0\.0\.0' |awk '{print $2}' #
esxcli network ip interface ipv4 set --interface-name vmk2 --peer-dns true --type=dhcp
esxcli system hostname set --host=#TABLE:nodetype:$NODE:node#
esxcli system hostname set --fqdn= #TABLE:nodetype:$NODE:node#.#TABLE:site:key=domain:value#
#-----------------------------------------------------------------------------------------------------------
# SSH
esxcli network firewall ruleset set --ruleset-id sshServer --enabled yes
chkconfig SSH on
echo "#COMMAND: cat /etc/vcl/vcl.key.pub#" > /etc/ssh/keys-root/authorized_keys
#-----------------------------------------------------------------------------------------------------------
# STORAGE
LOCAL_DATASTORE_NAME="local_#TABLE:nodetype:$NODE:node#"
REPOSITORY_NAME="repository"
# Rename the local datastore
vim-cmd hostsvc/datastore/rename datastore1 ${LOCAL_DATASTORE_NAME}
vim-cmd hostsvc/datastore/refresh ${LOCAL_DATASTORE_NAME}
vim-cmd hostsvc/storage/refresh
# Mount repository
esxcfg-nas --add --host <NAS IP> --share /repository ${REPOSITORY_NAME}
#-----------------------------------------------------------------------------------------------------------
# Add hostname to terminal prompt
echo "PS1='#TABLE:nodetype:$NODE:node#:\w \$ '" >> /etc/profile.local
#-----------------------------------------------------------------------------------------------------------
# FIREWALL
/vmfs/volumes/${REPOSITORY_NAME}/_scripts/esxi_firewall.sh > /var/log/esxi_firewall.log
#-----------------------------------------------------------------------------------------------------------
# TWEAKS
/vmfs/volumes/${REPOSITORY_NAME}/_scripts/esxi_tweaks.sh > /var/log/esxi_tweaks.log
#-----------------------------------------------------------------------------------------------------------
# SAVE LOGS
# Copy %firstboot script logs to persisted datastore
mkdir -p /vmfs/volumes/${LOCAL_DATASTORE_NAME}/install_logs
cp /var/log/* /vmfs/volumes/${LOCAL_DATASTORE_NAME}/install_logs/
#-----------------------------------------------------------------------------------------------------------
# Backup ESXi configuration to persist changes
/sbin/auto-backup.sh
reboot
<<<END
On Wed, Feb 14, 2018 at 2:25 PM, Jonathan Casco <jc...@fiu.edu>> wrote:
Thanks for that information Andy.
My current hangup is this part of the process in your list:
* Add the computer to xCAT and get things configured so xCAT can install and properly configure it (this is no easy feat)
I was thinking (as you mentioned) that xCAT would make a VMware host which could then be used by VCL, likely with the help of a kickstart file.
However xCAT does not seem to be too happy when loading ESXi 6 as it does not create an entry in the osimage table for it. It seems like this is a feature which was abandoned in xCAT several versions ago as it was not requested often. https://github.com/xcat2/xcat-core/issues/4108<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xcat2_xcat-2Dcore_issues_4108&d=DwMFaQ&c=lhMMI368wojMYNABHh1gQQ&r=0Mz5KaUjFlgb7ODSsLhATg&m=fi8Y17kZ-dX6IpkeLBgX96yFIr0tuKRhqP50_x38gGg&s=S3BcK5Rj8zDw3KQEdtzw0scrWZKcKzdoYRPk0sb3Tao&e=>
Would you be willing to give some detail on what version of ESXi you used in the past and how you got the xCAT provisioning for ESXi accomplished?
Thank you again,
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
From: Andy Kurth <an...@ncsu.edu>>
Reply-To: "user@vcl.apache.org<ma...@vcl.apache.org>" <us...@vcl.apache.org>>
Date: Wednesday, February 14, 2018 at 2:10 PM
To: "user@vcl.apache.org<ma...@vcl.apache.org>" <us...@vcl.apache.org>>
Subject: Re: Question on xCAT implementation in VCL
xCAT is only used within VCL to deploy bare-metal machines. The VM deployment functionality of xCAT isn't used at all. VCL can deploy bare metal machines for user reservations. It's pretty rare nowadays to deploy end user reservations on full bare-metal machines, but it's still useful.
For larger installations, you could set things up so that xCAT deploys and configures the bare-metal VM hosts (VMware and/or KVM). Here at NCSU, we have some rather complex Kickstart templates for each that set up ESXi, KVM bridging, etc.
In theory, the flow would go like this:
* Add the bare-metal computer to VCL, set the provisioning to xCAT
* Add the computer to xCAT and get things configured so xCAT can install and properly configure it (this is no easy feat)
* Add an image to the VCL database matching the xCAT template name
* In VCL, change the computer's state by selecting "convert to vmhostinuse"
* VCL instructs xCAT to automatically reload the computer using the xCAT template name
* When done, the computer is in the "vmhostinuse" state and you can assign VMs to it
In practice, I gave up on having VCL initiate the xCAT deployment long ago. There are too many things that can break. It's easier to initiate the xCAT reinstall manually, verify, then add the computer to VCL with the provisioning module set to none so that VCL never tries to reinstall it.
-Andy
On Wed, Feb 14, 2018 at 11:56 AM, Jonathan Casco <jc...@fiu.edu>> wrote:
Hello,
I am reading through the documentation for xCAT in VCL but was a little confused on the purpose xCAT has here.
Is xCAT being used for provisioning VM hosts like VMware to then get managed by a separate VCL server or does VCL use xCAT to create servers that would be used for VCL guests?
Thank you,
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
--
Andy Kurth
Research Storage Specialist
NC State University
Office of Information Technology
P: 919-513-4090
311A Hillsborough<https://urldefense.proofpoint.com/v2/url?u=https-3A__maps.google.com_-3Fq-3D311A-2BHillsborough-26entry-3Dgmail-26source-3Dg&d=DwMFaQ&c=lhMMI368wojMYNABHh1gQQ&r=0Mz5KaUjFlgb7ODSsLhATg&m=fi8Y17kZ-dX6IpkeLBgX96yFIr0tuKRhqP50_x38gGg&s=Urjk8BrLFibfwZwVAez_eQKEeW5_LVR81mfk09lKllo&e=> Building
Campus Box 7109
Raleigh, NC 27695
Error! Filename not specified.
--
Andy Kurth
Research Storage Specialist
NC State University
Office of Information Technology
P: 919-513-4090
311A Hillsborough Building
Campus Box 7109
Raleigh, NC 27695
[Image removed by sender.]
Re: Question on xCAT implementation in VCL
Posted by Andy Kurth <an...@ncsu.edu>.
You've probably already done this, but run:
copycds VMware-VMvisor-Installer-201701001-4887370.x86_64.iso
This should have created:
/install/esxi6.5/x86_64/
You're correct that recent xCAT doesn't seem to create an osimage
automatically by running copycds. To create an osimage manually, first
create a stanza file:
vi /root/esxi6.5-x86_64-install-hypervisor.stanza
Add this:
>>> BEGIN
# <xCAT data object stanza file>
esxi6.5-x86_64-install-hypervisor:
objtype=osimage
imagetype=linux
osarch=x86_64
osname=Linux
osvers=esxi6.5
otherpkgdir=/install/post/otherpkgs/esxi6.5/x86_64
profile=hypervisor
provmethod=install
template=/opt/xcat/share/xcat/install/esxi/hypervisor.tmpl
<<< END
Create the osimage from the stanza file:
cat /root/esxi6.5-x86_64-install-hypervisor.stanza | mkdef -z
View the osimage settings:
lsdef -t osimage esxi6.5-x86_64-install-hypervisor
If you want to change any the osimage settings, do so like this:
chdef -t osimage esxi6.5-x86_64-install-hypervisor
template=/opt/xcat/share/xcat/install/esxi/<MY
CUSTOM>.tmpl
The template file used will be the stock one. Make sure it exists:
/opt/xcat/share/xcat/install/esxi/hypervisor.tmpl
You need to make sure all of the xCAT node/osimage/etc properties used in
the template are defined. If not, you'll get an error when you run the
nodeset command later on.
For example, the stock template sets the ESXi root password via this line
in hypervisor.tmpl:
rootpw --iscrypted #CRYPT:passwd:key=vmware,username=root:password#
I ran a quick test installing a VM with ESXi 6.5 via xCAT and it seems the
password behavior changed with 6.5. I had to use the following to set an
encrypted password in xCAT. (Plain text in xCAT's passwd table used to
work) Without the openssh part, the installation failed with an error
related to a bad "crypted password":
chtab key=vmware passwd.username=root passwd.password=`openssl passwd -1
'<MY $TRONG PASSWORD>'`
You'll need to add a node in xCAT, set up the network table, get DHCP
working, and a few other things. These are pretty much stock xCAT
procedures so I won't get into them. Reply if you need help though.
Set the node to use the osimage:
nodeset <NODE> osimage=esxi6.5-x86_64-install-hypervisor
Again, if you get errors it probably means one or more of the xCAT settings
are missing for the node or network.
Before trying to install, verify node's PXE files look correct:
/tftpboot/xcat/xnba/nodes/<NODE>
/tftpboot/xcat/xnba/nodes/<NODE>.pxelinux
/tftpboot/xcat/xnba/nodes/<NODE>.uefi
The main file that will need tweaking is the template file generated for
the node by nodeset (derived from
/opt/xcat/share/xcat/install/esxi/hypervisor.tmpl):
/install/autoinst/<NODE>
If you get the node to install and want to work on tweaking the config,
what I usually do is edit the node's template file directly and then
restart/PXE boot/reinstall it. I don't rerun nodeset in between or else
the node's template file will be overwritten with the osimage template
settings.
Once I have the configuration correct, I'll rework the osimage template
using the appropriate xCAT variable substitutions.
Below is an old custom template file used for ESXi 6.0. It certainly needs
some tweaking but may give you some pointers on how to configure things.
The commented out lines at the beginning are used for debugging problems.
Even though commented out, the xCAT variables get substituted when you run
nodeset. I'll put in lines for most of the variables used later on to make
it easy to see if a value is missing or incorrect. After ESXi is
installed, we'll run some custom scripts stored on a NAS datastore which
configure the firewall and do some other tweaks not stored in the template.
>>> BEGIN
# node name: '#TABLE:nodetype:$NODE:node#'
# node private IP address: '#TABLE:hosts:$NODE:ip#'
# node private network: '#TABLE:networks:netname=private:net#'
# node private subnet mask: '#TABLE:networks:netname=private:mask#'
# node private MAC address: '#TABLE:mac:$NODE:mac#'
# nameservers: '#TABLE:site:key=nameservers:value#'
# domain: '#TABLE:site:key=domain:value#'
# management node: '#TABLE:site:key=master:value#'
# TFTP server: '#TABLE:noderes:$NODE:tftpserver#'
# default gateway: '#COMMAND: route -n |grep '^0\.0\.0\.0' |awk '{print
$2}' #'
# Install NIC: 'vmnic#COMMAND: echo -n #TABLE:noderes:$NODE:installnic# |
grep -o "[0-9]"#'
# Primary NIC: 'vmnic#COMMAND: echo -n #TABLE:noderes:$NODE:primarynic# |
grep -o "[0-9]"#'
vmaccepteula
rootpw --iscrypted #CRYPT:passwd:key=vmware,username=root:password#
clearpart --firstdisk=USB-IBM,usb,mptsas,mpt2sas,local --overwritevmfs
install --firstdisk=USB-IBM,usb,mptsas,mpt2sas,local --overwritevmfs
serialnum --esx=#TABLE:prodkey:$NODE,product=esxi6.free:key#
network --bootproto=dhcp --device=vmnic#COMMAND: echo -n
#TABLE:noderes:$NODE:installnic# | grep -o "[0-9]"# --addvmportgroup=0
reboot
#
=================================================================================================
%pre --interpreter=busybox
#
=================================================================================================
%post --interpreter=busybox --ignorefailure=true
localcli network firewall unload
sleep 5
echo -e "<xcatrequest>\n<command>nextdestiny</command>\n</xcatrequest>" |
/bin/openssl s_client -quiet -connect #TABLE:site:key=master:value#:3001
2>&1 | tee /tmp/foo.log
#
=================================================================================================
%firstboot --interpreter=busybox
#esxcfg-advcfg -s 1 /Net/FollowHardwareMac
#sleep 60
INSTALL_NIC=vmnic$(echo -n #TABLE:noderes:$NODE:installnic# | grep -o
"[0-9]")
PRIMARY_NIC=vmnic$(echo -n #TABLE:noderes:$NODE:primarynic# | grep -o
"[0-9]")
INSTALL_MAC=$(esxcli network nic list | grep $INSTALL_NIC | grep -E -o
'([0-9a-f]{2}:){5}[0-9a-f]{2}')
PRIMARY_MAC=$(esxcli network nic list | grep $PRIMARY_NIC | grep -E -o
'([0-9a-f]{2}:){5}[0-9a-f]{2}')
PRIVATE_IP=#TABLE:hosts:$NODE:ip#
PRIVATE_MASK=#TABLE:networks:netname=private:mask#
echo "INSTALL_NIC: ${INSTALL_NIC}"
echo "INSTALL_MAC: ${INSTALL_MAC}"
echo "PRIMARY_NIC: ${PRIMARY_NIC}"
echo "PRIMARY_MAC: ${PRIMARY_MAC}"
#----------------------------
#esxcli network vswitch standard remove --vswitch-name=vSwitchUSB0
esxcli network ip interface remove --interface-name vmk0
esxcli network vswitch standard portgroup remove --portgroup-name
"Management Network" --vswitch-name vSwitch0
esxcli network vswitch standard add --vswitch-name vSwitch1
esxcli network vswitch standard uplink add --uplink-name ${PRIMARY_NIC}
--vswitch-name vSwitch1
esxcli network vswitch standard portgroup add --portgroup-name
VMkernel-Public --vswitch-name vSwitch1
esxcli network ip interface add --interface-name vmk2 --portgroup-name
VMkernel-Public --mac-address=${PRIMARY_MAC}
esxcli network ip interface ipv4 set --interface-name vmk2 --type dhcp
esxcli network ip interface tag add -i vmk2 -t Management
esxcli network vswitch standard portgroup add --portgroup-name Public
--vswitch-name vSwitch1
esxcli network vswitch standard portgroup add --portgroup-name
VMkernel-Private --vswitch-name vSwitch0
esxcli network ip interface add --interface-name vmk0 --portgroup-name
VMkernel-Private --mac-address=${INSTALL_MAC}
esxcli network ip interface ipv4 set --interface-name vmk0 --ipv4
${PRIVATE_IP} --netmask ${PRIVATE_MASK} --type static
esxcli network ip interface tag remove --interface-name vmk0 --tagname
Management
esxcli network vswitch standard portgroup add --portgroup-name Private
--vswitch-name vSwitch0
esxcfg-route esxcfg-route -a default #COMMAND: route -n |grep '^0\.0\.0\.0'
|awk '{print $2}' #
esxcli network ip interface ipv4 set --interface-name vmk2 --peer-dns true
--type=dhcp
esxcli system hostname set --host=#TABLE:nodetype:$NODE:node#
esxcli system hostname set --fqdn=
#TABLE:nodetype:$NODE:node#.#TABLE:site:key=domain:value#
#-----------------------------------------------------------------------------------------------------------
# SSH
esxcli network firewall ruleset set --ruleset-id sshServer --enabled yes
chkconfig SSH on
echo "#COMMAND: cat /etc/vcl/vcl.key.pub#" >
/etc/ssh/keys-root/authorized_keys
#-----------------------------------------------------------------------------------------------------------
# STORAGE
LOCAL_DATASTORE_NAME="local_#TABLE:nodetype:$NODE:node#"
REPOSITORY_NAME="repository"
# Rename the local datastore
vim-cmd hostsvc/datastore/rename datastore1 ${LOCAL_DATASTORE_NAME}
vim-cmd hostsvc/datastore/refresh ${LOCAL_DATASTORE_NAME}
vim-cmd hostsvc/storage/refresh
# Mount repository
esxcfg-nas --add --host <NAS IP> --share /repository ${REPOSITORY_NAME}
#-----------------------------------------------------------------------------------------------------------
# Add hostname to terminal prompt
echo "PS1='#TABLE:nodetype:$NODE:node#:\w \$ '" >> /etc/profile.local
#-----------------------------------------------------------------------------------------------------------
# FIREWALL
/vmfs/volumes/${REPOSITORY_NAME}/_scripts/esxi_firewall.sh >
/var/log/esxi_firewall.log
#-----------------------------------------------------------------------------------------------------------
# TWEAKS
/vmfs/volumes/${REPOSITORY_NAME}/_scripts/esxi_tweaks.sh >
/var/log/esxi_tweaks.log
#-----------------------------------------------------------------------------------------------------------
# SAVE LOGS
# Copy %firstboot script logs to persisted datastore
mkdir -p /vmfs/volumes/${LOCAL_DATASTORE_NAME}/install_logs
cp /var/log/* /vmfs/volumes/${LOCAL_DATASTORE_NAME}/install_logs/
#-----------------------------------------------------------------------------------------------------------
# Backup ESXi configuration to persist changes
/sbin/auto-backup.sh
reboot
<<<END
On Wed, Feb 14, 2018 at 2:25 PM, Jonathan Casco <jc...@fiu.edu> wrote:
> Thanks for that information Andy.
>
>
>
> My current hangup is this part of the process in your list:
>
> * Add the computer to xCAT and get things configured so xCAT can install
> and properly configure it (this is no easy feat)
>
>
>
> I was thinking (as you mentioned) that xCAT would make a VMware host which
> could then be used by VCL, likely with the help of a kickstart file.
>
> However xCAT does not seem to be too happy when loading ESXi 6 as it does
> not create an entry in the osimage table for it. It seems like this is a
> feature which was abandoned in xCAT several versions ago as it was not
> requested often. https://github.com/xcat2/xcat-core/issues/4108
>
>
>
> Would you be willing to give some detail on what version of ESXi you used
> in the past and how you got the xCAT provisioning for ESXi accomplished?
>
>
>
> Thank you again,
>
> -----------------------
>
> Jonathan Casco
>
> HPC System Administrator
>
> Florida International University
>
> *From: *Andy Kurth <an...@ncsu.edu>
> *Reply-To: *"user@vcl.apache.org" <us...@vcl.apache.org>
> *Date: *Wednesday, February 14, 2018 at 2:10 PM
> *To: *"user@vcl.apache.org" <us...@vcl.apache.org>
> *Subject: *Re: Question on xCAT implementation in VCL
>
>
>
> xCAT is only used within VCL to deploy bare-metal machines. The VM
> deployment functionality of xCAT isn't used at all. VCL can deploy bare
> metal machines for user reservations. It's pretty rare nowadays to
> deploy end user reservations on full bare-metal machines, but it's still
> useful.
>
>
>
> For larger installations, you could set things up so that xCAT deploys and
> configures the bare-metal VM hosts (VMware and/or KVM). Here at NCSU, we
> have some rather complex Kickstart templates for each that set up ESXi, KVM
> bridging, etc.
>
>
>
> In theory, the flow would go like this:
>
> * Add the bare-metal computer to VCL, set the provisioning to xCAT
>
> * Add the computer to xCAT and get things configured so xCAT can install
> and properly configure it (this is no easy feat)
>
> * Add an image to the VCL database matching the xCAT template name
>
> * In VCL, change the computer's state by selecting "convert to vmhostinuse"
>
> * VCL instructs xCAT to automatically reload the computer using the xCAT
> template name
>
> * When done, the computer is in the "vmhostinuse" state and you can assign
> VMs to it
>
>
>
> In practice, I gave up on having VCL initiate the xCAT deployment long
> ago. There are too many things that can break. It's easier to initiate
> the xCAT reinstall manually, verify, then add the computer to VCL with the
> provisioning module set to none so that VCL never tries to reinstall it.
>
>
>
> -Andy
>
>
>
>
>
>
>
> On Wed, Feb 14, 2018 at 11:56 AM, Jonathan Casco <jc...@fiu.edu> wrote:
>
> Hello,
>
>
>
> I am reading through the documentation for xCAT in VCL but was a little
> confused on the purpose xCAT has here.
>
>
>
> Is xCAT being used for provisioning VM hosts like VMware to then get
> managed by a separate VCL server or does VCL use xCAT to create servers
> that would be used for VCL guests?
>
>
>
> Thank you,
>
> -----------------------
>
> Jonathan Casco
>
> HPC System Administrator
>
> Florida International University
>
>
>
>
>
>
>
> --
>
> *Andy Kurth*
>
> Research Storage Specialist
>
> NC State University
>
> Office of Information Technology
>
>
>
> P: 919-513-4090
>
> 311A Hillsborough
> <https://maps.google.com/?q=311A+Hillsborough&entry=gmail&source=g>
> Building
>
> Campus Box 7109
>
> Raleigh, NC 27695
>
> [image: Image removed by sender.]
>
--
*Andy Kurth*
Research Storage Specialist
NC State University
Office of Information Technology
P: 919-513-4090
311A Hillsborough Building
Campus Box 7109
Raleigh, NC 27695
Re: Question on xCAT implementation in VCL
Posted by Jonathan Casco <jc...@fiu.edu>.
Thanks for that information Andy.
My current hangup is this part of the process in your list:
* Add the computer to xCAT and get things configured so xCAT can install and properly configure it (this is no easy feat)
I was thinking (as you mentioned) that xCAT would make a VMware host which could then be used by VCL, likely with the help of a kickstart file.
However xCAT does not seem to be too happy when loading ESXi 6 as it does not create an entry in the osimage table for it. It seems like this is a feature which was abandoned in xCAT several versions ago as it was not requested often. https://github.com/xcat2/xcat-core/issues/4108
Would you be willing to give some detail on what version of ESXi you used in the past and how you got the xCAT provisioning for ESXi accomplished?
Thank you again,
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
From: Andy Kurth <an...@ncsu.edu>
Reply-To: "user@vcl.apache.org" <us...@vcl.apache.org>
Date: Wednesday, February 14, 2018 at 2:10 PM
To: "user@vcl.apache.org" <us...@vcl.apache.org>
Subject: Re: Question on xCAT implementation in VCL
xCAT is only used within VCL to deploy bare-metal machines. The VM deployment functionality of xCAT isn't used at all. VCL can deploy bare metal machines for user reservations. It's pretty rare nowadays to deploy end user reservations on full bare-metal machines, but it's still useful.
For larger installations, you could set things up so that xCAT deploys and configures the bare-metal VM hosts (VMware and/or KVM). Here at NCSU, we have some rather complex Kickstart templates for each that set up ESXi, KVM bridging, etc.
In theory, the flow would go like this:
* Add the bare-metal computer to VCL, set the provisioning to xCAT
* Add the computer to xCAT and get things configured so xCAT can install and properly configure it (this is no easy feat)
* Add an image to the VCL database matching the xCAT template name
* In VCL, change the computer's state by selecting "convert to vmhostinuse"
* VCL instructs xCAT to automatically reload the computer using the xCAT template name
* When done, the computer is in the "vmhostinuse" state and you can assign VMs to it
In practice, I gave up on having VCL initiate the xCAT deployment long ago. There are too many things that can break. It's easier to initiate the xCAT reinstall manually, verify, then add the computer to VCL with the provisioning module set to none so that VCL never tries to reinstall it.
-Andy
On Wed, Feb 14, 2018 at 11:56 AM, Jonathan Casco <jc...@fiu.edu>> wrote:
Hello,
I am reading through the documentation for xCAT in VCL but was a little confused on the purpose xCAT has here.
Is xCAT being used for provisioning VM hosts like VMware to then get managed by a separate VCL server or does VCL use xCAT to create servers that would be used for VCL guests?
Thank you,
-----------------------
Jonathan Casco
HPC System Administrator
Florida International University
--
Andy Kurth
Research Storage Specialist
NC State University
Office of Information Technology
P: 919-513-4090
311A Hillsborough Building
Campus Box 7109
Raleigh, NC 27695
[Image removed by sender.]
Re: Question on xCAT implementation in VCL
Posted by Andy Kurth <an...@ncsu.edu>.
xCAT is only used within VCL to deploy bare-metal machines. The VM
deployment functionality of xCAT isn't used at all. VCL can deploy bare
metal machines for user reservations. It's pretty rare nowadays to deploy
end user reservations on full bare-metal machines, but it's still useful.
For larger installations, you could set things up so that xCAT deploys and
configures the bare-metal VM hosts (VMware and/or KVM). Here at NCSU, we
have some rather complex Kickstart templates for each that set up ESXi, KVM
bridging, etc.
In theory, the flow would go like this:
* Add the bare-metal computer to VCL, set the provisioning to xCAT
* Add the computer to xCAT and get things configured so xCAT can install
and properly configure it (this is no easy feat)
* Add an image to the VCL database matching the xCAT template name
* In VCL, change the computer's state by selecting "convert to vmhostinuse"
* VCL instructs xCAT to automatically reload the computer using the xCAT
template name
* When done, the computer is in the "vmhostinuse" state and you can assign
VMs to it
In practice, I gave up on having VCL initiate the xCAT deployment long
ago. There are too many things that can break. It's easier to initiate
the xCAT reinstall manually, verify, then add the computer to VCL with the
provisioning module set to none so that VCL never tries to reinstall it.
-Andy
On Wed, Feb 14, 2018 at 11:56 AM, Jonathan Casco <jc...@fiu.edu> wrote:
> Hello,
>
>
>
> I am reading through the documentation for xCAT in VCL but was a little
> confused on the purpose xCAT has here.
>
>
>
> Is xCAT being used for provisioning VM hosts like VMware to then get
> managed by a separate VCL server or does VCL use xCAT to create servers
> that would be used for VCL guests?
>
>
>
> Thank you,
>
> -----------------------
>
> Jonathan Casco
>
> HPC System Administrator
>
> Florida International University
>
>
>
--
*Andy Kurth*
Research Storage Specialist
NC State University
Office of Information Technology
P: 919-513-4090
311A Hillsborough Building
Campus Box 7109
Raleigh, NC 27695