You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by bh...@apache.org on 2017/10/25 05:19:57 UTC
[cloudstack-docs] branch master updated: CLOUDSTACK-10103:
Documentation for Cloudian Connector
This is an automated email from the ASF dual-hosted git repository.
bhaisaab pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cloudstack-docs.git
The following commit(s) were added to refs/heads/master by this push:
new f86e77e CLOUDSTACK-10103: Documentation for Cloudian Connector
f86e77e is described below
commit f86e77e3c03bf7af634acb3b94e8606547e3b641
Author: Rohit Yadav <ro...@shapeblue.com>
AuthorDate: Fri Oct 6 16:06:33 2017 +0530
CLOUDSTACK-10103: Documentation for Cloudian Connector
Signed-off-by: Rohit Yadav <ro...@shapeblue.com>
---
rtd/source/_static/images/cloudian-s3_ss_cache.png | Bin 0 -> 69503 bytes
.../_static/images/cloudian-s3_ss_config.png | Bin 0 -> 45092 bytes
.../_static/images/cloudian-ss_globalopt.png | Bin 0 -> 71861 bytes
rtd/source/_static/images/cloudian-tab.png | Bin 0 -> 6122 bytes
rtd/source/conf.py | 6 +-
rtd/source/index.rst | 13 +
rtd/source/integration/cloudian-connector.rst | 473 +++++++++++++++++++++
7 files changed, 489 insertions(+), 3 deletions(-)
diff --git a/rtd/source/_static/images/cloudian-s3_ss_cache.png b/rtd/source/_static/images/cloudian-s3_ss_cache.png
new file mode 100644
index 0000000..5d97186
Binary files /dev/null and b/rtd/source/_static/images/cloudian-s3_ss_cache.png differ
diff --git a/rtd/source/_static/images/cloudian-s3_ss_config.png b/rtd/source/_static/images/cloudian-s3_ss_config.png
new file mode 100644
index 0000000..3ad273a
Binary files /dev/null and b/rtd/source/_static/images/cloudian-s3_ss_config.png differ
diff --git a/rtd/source/_static/images/cloudian-ss_globalopt.png b/rtd/source/_static/images/cloudian-ss_globalopt.png
new file mode 100644
index 0000000..d1b9272
Binary files /dev/null and b/rtd/source/_static/images/cloudian-ss_globalopt.png differ
diff --git a/rtd/source/_static/images/cloudian-tab.png b/rtd/source/_static/images/cloudian-tab.png
new file mode 100644
index 0000000..57383e8
Binary files /dev/null and b/rtd/source/_static/images/cloudian-tab.png differ
diff --git a/rtd/source/conf.py b/rtd/source/conf.py
index f99e2ee..1fc6e29 100644
--- a/rtd/source/conf.py
+++ b/rtd/source/conf.py
@@ -63,16 +63,16 @@ master_doc = 'index'
# General information about the project.
project = u'Apache CloudStack'
-copyright = u'2016, Apache CloudStack'
+copyright = u'2017, Apache CloudStack'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
-version = '4.8'
+version = '4.11'
# The full version, including alpha/beta/rc tags.
-release = '4.8.0'
+release = '4.11.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/rtd/source/index.rst b/rtd/source/index.rst
index 62437c8..706a90f 100644
--- a/rtd/source/index.rst
+++ b/rtd/source/index.rst
@@ -61,6 +61,18 @@ Below you will find very specific documentation on advanced networking_ which
you can skip if you are just getting started. Developers will also find below
a short developers_ guide.
+
+.. _integrations:
+
+Integration Guides
+------------------
+
+.. toctree::
+ :maxdepth: 2
+
+ integration/cloudian-connector.rst
+
+
.. _networking:
@@ -77,6 +89,7 @@ Advanced Networking Guides
networking/ipv6
networking/autoscale_without_netscaler.rst
+
.. _developers:
diff --git a/rtd/source/integration/cloudian-connector.rst b/rtd/source/integration/cloudian-connector.rst
new file mode 100644
index 0000000..58bb170
--- /dev/null
+++ b/rtd/source/integration/cloudian-connector.rst
@@ -0,0 +1,473 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information#
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+The Cloudian Connector Plugin
+=============================
+
+Introduction to the Cloudian Connector Plugin
+---------------------------------------------
+
+The Cloudian (HyperStore) Connector is a native CloudStack plugin that allow
+integration between Apache CloudStack and Cloudian Management Console (CMC). The
+Connector integrates Cloudian S3 Storage into the CloudStack Management GUI and
+allows administrators to easily give their CloudStack users access to and manage
+their own S3 storage areas.
+
+Compatibilty
+~~~~~~~~~~~~
+
+The following table shows the compatiblity of Cloudian Connector with CloudStack.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------+-------------------------+
+| Connector Version | CloudStack version | Cloudian Compatibility |
++=====================+======================+=========================+
+| 4.9_6.2-1 | 4.9 | Version 6.2 and onwards |
++---------------------+----------------------+-------------------------+
+| 4.11+ | 4.11+ | Version 6.2 and onwards |
++---------------------+----------------------+-------------------------+
+
+Table: Support Matrix
+
+.. note::
+ Starting CloudStack 4.11, the Connector will be part of the CloudStack
+ release and will not need to be externally installed.
+
+Connector Overview
+------------------
+
+Single-Sign-On Integration
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The connector plugin adds a Cloudian Storage button to the CloudStack UI. This
+button is available for all users on the bottom left of the menu.
+
+.. figure:: /_static/images/cloudian-tab.png
+ :align: center
+ :alt: a screenshot of an enabled Cloudian Connector
+
+When a user clicks this button, a new window or tab (depending on the web
+browser preferences) is opened for the HyperStore CMC GUI. The CloudStack user
+is automatically logged in to CMC as the correctly mapped HyperStore user using
+Single-Sign-On (SSO).
+
+With the connector enabled, when the user clicks ‘Log Out’ in the CloudStack UI
+this first logs the user out of CloudStack and then redirects the page to log
+out any logged-in Cloudian user out of CMC GUI and finally redirects the page to
+the CloudStack login page.
+
+Single-Sign-On is a technique where CloudStack and HyperStore are configured to
+trust each other. This is achieved by configuring both HyperStore and the
+CloudStack connector with the same SSO Shared Key. The CloudStack connector
+creates a special login URL for CMC which it signs using this shared key. Upon
+receiving the special signed login URL, CMC validates the request by comparing
+the signature to its own copy of the shared key and the user is automatically
+logged in.
+
+User Mapping and Provisioning/De-provisioning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+CloudStack domains are mapped to Cloudian Groups. CloudStack accounts within
+those domains are mapped to Cloudian users. The Cloudian user and group are
+created on demand if it doesn’t already exist when the CloudStack user accesses
+CMC through the Cloudian Storage button. When accounts and domains are created
+or removed in CloudStack, they automatically create or remove users or groups in
+CMC.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------------+
+| CloudStack Entity | Equivalent Cloudian Entity |
++=====================+============================+
+| Account | User |
++---------------------+----------------------------+
+| Domain | Group |
++---------------------+----------------------------+
+
+Table: Mapping Between Cloudian and CloudStack
+
+.. note::
+ Adding groups or users directly through Cloudian does not add
+ corresponding CloudStack Domains or Accounts. The integration is driven
+ completely from the CloudStack side.
+
+Special Admin User Mapping
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The special CloudStack admin is are mapped to a special HyperStore Admin user
+account which defaults to the user id admin. As the admin user on HyperStore is
+configurable, there is a configuration option to control this mapping. This
+mapping dictates which HyperStore user is automatically logged in using SSO when
+the CloudStack admin user clicks "Cloudian Storage".
+
+.. note::
+ The Cloudian Admin user default is called admin. Older versions of
+ Cloudian used to use admin@cloudian.com.
+
+DNS Resolution Requirements
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The CloudStack Management Server will need to be access the Cloudian admin
+service. The Cloudian admin service is commonly run on the same nodes as your
+Cloudian S3 servers. The admin service is used to provision and deprovision
+Cloudian users and groups automatically by CloudStack.
+
+Additionally, your CloudStack users will need to be able to resolve your CMC
+server hostname on their desktops so that they can access CMC.
+
+Example domain names that should resolve:
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------+------------------------------+
+| Resolvable Name | Required By | Description |
++=====================+======================+==============================+
+| mgmt.abc-cloud.com | User's browser | CloudStack Management Server |
++---------------------+----------------------+------------------------------+
+| cmc.abc-cloud.com | User's browser | Cloudian CMC |
++---------------------+----------------------+------------------------------+
+| admin.abc-cloud.com | Management Server | Cloudian Admin Server |
++---------------------+----------------------+------------------------------+
+
+Table: DNS Name Resolution Example
+
+
+Configuring the Cloudian Connector
+----------------------------------
+
+Prerequisites
+~~~~~~~~~~~~~
+
+Cloudian ships with SSO disabled by default. You will need to enable it on each
+CMC server. Additionally, you will need to choose a unique SSO shared key that
+you will also configure in the CloudStack connector further below.
+
+Edit Puppet config to enable SSO on all CMC servers:
+
+ ::
+
+ # vi /etc/cloudian-[version]-puppet/modules/cmc/templates/mts-ui.properties.erb
+ sso.enabled=true
+ sso.shared.key=YourSecretKeyHere
+
+
+.. note::
+ Once configured in Puppet, you should roll out out to each CMC server and
+ restart CMC services. Please refer to the HyperStore documentation for how to
+ do this.
+
+Connector Configuration
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The main way to configure, enable and disable the connector is using the
+CloudStack global setting. The global settings provide an easy way to configure
+the connector and synchronize setting across multiple management server(s). The
+following global setting can be accessed and changed using the CloudStack UI:
+
+.. cssclass:: table-striped table-bordered table-hover
+
++------------------------------+------------------------------------------------+
+| Global Setting | Description |
++==============================+================================================+
+| cloudian.connector.enabled | Setting to enable/disable the plugin |
++------------------------------+------------------------------------------------+
+| cloudian.admin.host | The Cloudian admin server host |
++------------------------------+------------------------------------------------+
+| cloudian.admin.port | The Cloudian admin server port, usually 19443 |
+| | (https) or 18081 (http) |
++------------------------------+------------------------------------------------+
+| cloudian.admin.protocol | The Cloudian admin server protocol, http/https |
++------------------------------+------------------------------------------------+
+| cloudian.validate.ssl | Whether to validate SSL certificate of Cloudian|
+| | admin service while making API calls |
++------------------------------+------------------------------------------------+
+| cloudian.admin.user | Basic auth user name for Cloudian admin server |
++------------------------------+------------------------------------------------+
+| cloudian.admin.password | Basic auth password for Cloudian admin server |
++------------------------------+------------------------------------------------+
+| cloudian.api.request.timeout | The admin API request timeout in seconds |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.admin.user | The user id of the CMC admin that maps to |
+| | CloudStack admin user |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.host | The Cloudian Management Console hostname |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.port | The Cloudian Management Console port |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.protocol | The Cloudian Management Console protocol |
++------------------------------+------------------------------------------------+
+| cloudian.sso.key | The shared secret as configured in Cloudian CMC|
++------------------------------+------------------------------------------------+
+
+Table: Cloudian Connector Global Settings
+
+.. note::
+ Change in only ‘cloudian.connector.enabled’ setting requires restarting of
+ all the CloudStack management server(s), rest of the setting can be changed
+ dynamically without requiring to restart the CloudStack management server(s).
+
+Enabling the Cloudian Connector
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Cloudian Connector comes disabled by default, enabling the connector is the
+last step. You should have already configured the Cloudian Connector global
+settings. To enable the connector, ensure that the global setting
+"cloudian.connector.enabled" is set to true. Finally, restart each of the
+management server(s) to reload and enable the connector.
+
+For example, here is how you can restart the CloudStack management server
+installed on CentOS7:
+
+ ::
+
+ # systemctl restart cloudstack-management
+
+
+Troubleshooting
+~~~~~~~~~~~~~~~~
+
+Most of the trouble you may run into will be configuration related.
+
+There are a few things which can go wrong for SSO. Here are the most common
+problems and things to check:
+
+- Does the global settings cloudian.cmc.admin.user point to the correct Cloudian
+ (admin) user?
+
+- Is SSO configured and enabled on Cloudian HyperStore CMC?
+
+- Check for errors in the CMC log file.
+
+- Are both CloudStack and HyperStore CMC configured with the same cloudian.sso.key?
+
+- Check the /var/log/cloudstack/management/management-server.log file and
+ search for errors relating to SSO.
+
+- Try access the CMC host directly from the problem users host using the
+ configured cloudian.cmc.host, cloudian.cmc.port and cloudian.cmc.protocol
+ configured in the CloudStack global settings.
+
+- If you log out of the management server and log in again, does the Cloudian
+ Storage button work?
+
+
+Adding/Deleting Domains or Accounts fails: These operations use the Cloudian
+Admin Server. It's likely that something has changed with the connection or the
+admin server is down. Check list:
+
+- Is the admin server alive and listening?
+
+- Try access the admin server host directly from the problem users host using
+ the configured cloudian.admin.host, cloudian.admin.port and
+ cloudian.admin.protocol configured in the CloudStack global settings. Check the
+ configured auth settings cloudian.admin.user and cloudian.admin.password.
+
+- If you’re experiencing timeout issues, trying changing the API timeout value
+ defined in cloudian.api.request.timeout global setting.
+
+- Look for errors in the admin log file /var/log/cloudian/cloudian-admin.log.
+
+
+------------
+
+
+Cloudian as CloudStack Secondary Storage
+----------------------------------------
+
+This section is a supplementary guide for CloudStack and describes how to
+configure CloudStack to use Cloudian HyperStore as Secondary Storage. Please
+also review CloudStack’s documentation (Getting Started Guide) for configuring
+and using S3 as Secondary Storage.
+
+CloudStack, as of version 4.2.1, can utilize Cloudian HyperStore as S3 Secondary
+Storage out of the box. There is no need for any modifications or to install any
+connectors. Secondary Storage is used to store ISOs, Templates, Snapshots and
+Volumes.
+
+.. note::
+ CloudStack still requires an NFS Secondary Storage Staging Server with is
+ mentioned in the requirements below.
+
+Requirements:
+
+- CloudStack 4.5+ (installed/configured and running)
+- Cloudian HyperStore 5.0 or greater (installed/configured and running)
+
+NFS Secondary Storage Staging Server Requirement
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The use of S3 as Secondary Storage for CloudStack also requires an NFS server.
+The NFS server is required because the various hypervisors cannot yet talk
+directly to S3 but instead talk through the standard file system API. As such,
+CloudStack requires an NFS staging server which the Hypervisors use to read and
+write data from/to. The NFS storage requirements for the staging server are
+small however as space is only required while objects are staged (moving)
+between the S3 server and the VMs.
+
+
+DNS Name Resolution Requirement
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All CloudStack Management Servers, system VMs and customer VMs (if required)
+must be able to resolve your S3 bucket names. Usually, if you already have
+Cloudian installed and running in your environment, this is already working.
+At a minimum the following names should resolve to the correct IP addresses
+using the DNS server that your Management Server and System VMs are using.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+------------------------------+
+| Example Name | DNS Name Types |
++=====================+==============================+
+| s3.mycloud.com | Cloudian S3 Endpoint |
++---------------------+------------------------------+
+| sec.s3.mycloud.com | Bucket for Secondary Storage |
++---------------------+------------------------------+
+| s3-admin.mycloud.com| Cloudian Admin Server |
++---------------------+------------------------------+
+
+Table: Example Domain Names that should Resolve on CloudStack Servers
+
+Adding Cloudian as CloudStack Secondary Storage
+-----------------------------------------------
+
+Setup a Cloudian User and Bucket for Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+S3 Secondary Storage stores the CloudStack templates, snapshots etc in a
+dedicated S3 Bucket. To properly configure CloudStack you will need to know the
+S3 Bucket name and how to access your S3 Server (the S3 endpoint, access key and
+secret key).
+
+Below, we will create a dedicated Cloudian user and a dedicated bucket which we
+will assign for use as Secondary Storage.
+
+Create a dedicated user/group:
+
+- Login to the Cloudian Management Console (CMC) as the Cloudian admin user.
+
+- Create a new group called cloudstack. Any group name is ok.
+
+- Create a new user called cloudstack in the cloudstack group. Any user name is ok.
+
+
+Create a dedicated bucket:
+
+- Login to CMC as the cloudstack user created above.
+
+- Create a bucket called secondary. Any bucket name will do.
+
+- On the top menu bar on the right hand side, use the drop down menu under your
+ user name to select Security Credentials and copy and paste your Access and
+ Secret Keys to a note for later use. CloudStack will need these when you attach
+ Cloudian as Secondary Storage in a later step below.
+
+Open Up Access to your S3 Network from Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If your S3 server is on a different network to your Secondary Storage VM, you
+will need to open up access to the S3 network. This also allows users to
+download templates from their S3 object store areas.
+
+.. figure:: /_static/images/cloudian-ss_globalopt.png
+ :align: center
+ :alt: a screenshot of changing global settings
+
+.. note::
+ Editing the Global Settings requires you to restart the management server(s).
+
+Add an NFS Secondary Storage Staging Server
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As mentioned previously, S3 Secondary Storage currently requires the use of an
+NFS Secondary Staging Server. Add NFS Secondary Storage Staging Server:
+
+- Login to CloudStack Management Server as the admin user.
+
+- Navigate to Infrastructure → Secondary Storage.
+
+- Click Select View and select Secondary Staging Store.
+
+- Click Add Secondary Staging Store.
+
+- Configure the zone, server and path for your desired secondary staging store.
+ For example nfs.mycloud.com and /export/staging.
+
+.. figure:: /_static/images/cloudian-s3_ss_cache.png
+ :align: center
+ :alt: a screenshot of adding secondary staging store
+
+Attach Cloudian as Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+CloudStack supports using either S3 or NFS as Secondary Storage but not both.
+The below instructions assume you are not using Secondary Storage on NFS and
+that you can delete it to add the S3 storage.
+
+.. note::
+ Already using NFS for Secondary Storage with CloudStack? You need to migrate
+ your Secondary Storage. Refer to CloudStack’s instructions for migrating
+ existing NFS Secondary Storage to an S3 object storage. CloudStack 4.5 onwards
+ supports migrating data via special commands which are described in the Getting
+ Started Guide in a section titled Upgrading from NFS to Object Storage.
+
+Adding S3 Secondary Storage:
+
+- Login to CloudStack Management Server as the admin user.
+
+- Navigate to Infrastructure → Secondary Storage.
+
+- If it exists, select and delete any existing NFS Secondary Storage server
+ setting. NOTE: Do not do this if you want to migrate existing NFS secondary
+ storage to S3. Instead, see warning above.
+
+- Click the Add Secondary Storage button. This will open up a pop-up form which
+ you can fill out similarly to below.
+
+.. figure:: /_static/images/cloudian-s3_ss_config.png
+ :align: center
+ :alt: a screenshot of configuring S3 secondary storage
+
+.. note::
+ CloudStack doesn’t currently allow you to re-edit the S3 configuration so take
+ time to double check what you enter. If you make a mistake the only options
+ currently are either a) delete and recreate the storage or b) directly edit the
+ entry in the database.
+
+When you have finished adding Cloudian as Secondary Storage in the previous
+steps, CloudStack will populate the new secondary storage with the system and
+default templates. This can take some time do download as the templates are
+quite big.
+
+.. note::
+ You can check if the system template and the default template have properly
+ downloaded to the new secondary storage by navigating to Templates, selecting a
+ template, clicking on the Zones tab and checking its Status is Ready 100%
+ Downloaded.
+
+.. note::
+ Should you continue to have problems, sometimes it is necessary to restart
+ the Secondary Storage VM. You can do this by navigating to Infrastructure,
+ System VMs, selecting and rebooting the Secondary Storage VM.
+
+CloudStack should now ready to use Cloudian HyperStore for S3 Secondary Storage.
+
+Revision History
+----------------
+
+- Fri Oct 6 2017 Rohit Yadav rohit.yadav@shapeblue.com Documentation
+ created for 4.11.0 version of the Cloudian Connector Plugin
--
To stop receiving notification emails like this one, please contact
['"commits@cloudstack.apache.org" <co...@cloudstack.apache.org>'].