You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by se...@apache.org on 2013/07/23 17:29:42 UTC
git commit: updated refs/heads/master to 0648406
Updated Branches:
refs/heads/master d2ce17ea7 -> 06484061f
add midterm review
Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/06484061
Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/06484061
Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/06484061
Branch: refs/heads/master
Commit: 06484061f39f27dd3249572016e8a9adbf95851a
Parents: d2ce17e
Author: Ian Duffy <ia...@ianduffy.ie>
Authored: Tue Jul 23 14:13:39 2013 +0100
Committer: Sebastien Goasguen <ru...@gmail.com>
Committed: Tue Jul 23 11:29:31 2013 -0400
----------------------------------------------------------------------
docs/en-US/gsoc-midsummer-ian.xml | 319 ++++++++++++++++++-
docs/en-US/gsoc-midsummer.xml | 4 +-
.../images/add-ldap-configuration-failure.png | Bin 0 -> 27841 bytes
docs/en-US/images/add-ldap-configuration.png | Bin 0 -> 27393 bytes
.../delete-ldap-configuration-failure.png | Bin 0 -> 29262 bytes
docs/en-US/images/delete-ldap.png | Bin 0 -> 27832 bytes
docs/en-US/images/jenkins-pipeline.png | Bin 0 -> 29136 bytes
docs/en-US/images/ldap-account-addition.png | Bin 0 -> 69225 bytes
docs/en-US/images/ldap-configuration.png | Bin 0 -> 33360 bytes
docs/en-US/images/ldap-global-settings.png | Bin 0 -> 31145 bytes
docs/en-US/images/ldap-list-users.png | Bin 0 -> 56047 bytes
docs/en-US/images/list-ldap-configuration.png | Bin 0 -> 30977 bytes
12 files changed, 317 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/gsoc-midsummer-ian.xml
----------------------------------------------------------------------
diff --git a/docs/en-US/gsoc-midsummer-ian.xml b/docs/en-US/gsoc-midsummer-ian.xml
index 348418f..c62cdc3 100644
--- a/docs/en-US/gsoc-midsummer-ian.xml
+++ b/docs/en-US/gsoc-midsummer-ian.xml
@@ -11,9 +11,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -23,6 +23,317 @@
-->
<section id="gsoc-midsummer-ian">
- <title>Mid-Summer Progress Updates</title>
- <para>This section describes ...</para>
+ <title>Mid-Summer Progress Updates for Ian Duffy - "Ldap User Provisioning"</title>
+ <para>This section describes my progress with the project titled "LDAP User Provisioning".</para>
+ <section id="introduction">
+ <title>Introduction</title>
+ <para>
+ Progress on my project is moving along smoothly. The Cloudstack community along with my mentor Abhi have been very accomodating. Since the community bonding period communication has been consistent and the expectations have been clear. Sebastien, head mentor has given us great guidance. I have enjoyed their teaching style. I found it was a nice gradual build up starting with creating a simple document update patch to eventually submitting a new Cloudstack Plugin.
+ </para>
+ <para>
+ I am pleased with my progress on the project to date. I feel as if the goals set out in my proposal are very doable and that they should be achieved.
+ </para>
+ </section>
+ <section id="jenkins">
+ <title>Continuous Integration with Jenkins</title>
+ <para>
+ In order to try deliver working solutions of good quality I felt it would be a good idea to implement a continuous integration environment using Jenkins. The idea of this would be to automatically build and test my code. This was welcomed and aided by community members greatly.
+ </para>
+ <para>
+ The pipeline for this is as follows:
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/jenkins-pipeline.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>jenkins-pipeline.png: Screenshot of the build pipeline.</phrase>
+ </textobject>
+ </mediaobject>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Acquire Code Base - This pulls down the latest Cloudstack codebase and builds it executing all unit tests.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Static Analysis - This runs tests on my code to ensure quality and good practice. This is being achieved with sonar source.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Integration Tests - This deploys the Cloudstack database. Brings up the Cloudstack Manager with jetty and their simulator. All checkin/integration tests are ran and then the jetty server is shutdown.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Package(Only exists on my local Jenkins) - The codebase is packaged up into an RPM and placed onto a local yum repo. If the time allows this will be used for future automated acceptance testing.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ If your are interested in this I have created a screencast on youtube which walks through it: <ulink url="http://www.youtube.com/watch?v=8k9IS3wMRok"><citetitle>Continuous testing environment</citetitle></ulink>
+ </para>
+ </section>
+ <section id="ldap-plugin-implementation">
+ <title>Ldap Plugin implementation</title>
+ <para>
+ At the start of the coding stage I began to review the current LDAP implementation. This includes:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ The user authenticator - This enables LDAP users to login to Cloudstack once the user exists within the internal Cloudstack database.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAPConfig -This allows for adding LDAP configuration. This is detailed over here: <ulink url="https://cloudstack.apache.org/docs/api/apidocs-4.1/root_admin/ldapConfig.html"><citetitle>ldapConfig API reference</citetitle></ulink> This did not allow multiple configurations.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAPRemove - This allows for removing the LDAP configuration
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ UI features. Global settings -> LDAP configuration allowed for the addition of a single LDAP server using the LDAPConfig command and the removal of an LDAP server using the LDAPRemove command.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ After reviewing this code and implementation for some time I realised that it wasn't the most maintainable code. I realised I could extend it if required. But it would involve creating more unmaintainable code and it would be messy. This goes against my own principles of developing quality. This made me make the steep but justified decision to completely redo the LDAP implementation within Cloudstack. By doing this I did expanded the scope of the project.
+ </para>
+ <para>
+ I began to research the most appropriate way of structuring this. I started of by redoing the implementation. This meant creating the following classes(Excluding DAOs):
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ LdapManager: Manages all LDAP connections.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LdapConfiguration: Supplies all configuration from within the Cloudstack database or defaults where required.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LdapUserManager: Handles any interaction with LDAP user information.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LdapUtils: Supplies static helpers, e.g. escape search queries, get attributes from search queries.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LdapContextFactory: Manages the creation of contexts.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LdapAuthenticator: Supplies an authenticator to Cloudstack using the LdapManager.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ From this I had a solid foundation for creating API commands to allow the user to interact with an LDAP server. I went on to create the following commands:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ LdapAddConfiguration - This allows for adding multiple LDAP configurations. Each configuration is just seen as a hostname and port.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/add-ldap-configuration.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>add-ldap-configuration.png: Screenshot of API response.</phrase>
+ </textobject>
+ </mediaobject>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/add-ldap-configuration-failure.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>add-ldap-configuration-failure.png: Screenshot of API response.</phrase>
+ </textobject>
+ </mediaobject>
+ </listitem>
+ <listitem>
+ <para>
+ LdapDeleteConfiguration - This allows for the deletion of an LDAP configuration based on its hostname.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/delete-ldap-configuration.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>delete-ldap-configuration.png: Screenshot of API response.</phrase>
+ </textobject>
+ </mediaobject>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/delete-ldap-configuration-failure.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>delete-ldap-configuration-failure.png: Screenshot of API response.</phrase>
+ </textobject>
+ </mediaobject>
+ </listitem>
+ <listitem>
+ <para>
+ LdapListConfiguration - This lists all of the LDAP configurations that exist within the database.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/list-ldap-configuration.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>list-ldap-configuration.png: Screenshot of the build pipeline.</phrase>
+ </textobject>
+ </mediaobject>
+ </listitem>
+ <listitem>
+ <para>
+ LdapListAllUsers - This lists all the users within LDAP.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/ldap-list-users.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>ldap-list-users.png: Screenshot of the build pipeline.</phrase>
+ </textobject>
+ </mediaobject>
+ </listitem>
+ </itemizedlist>
+ <para>
+ Along with this global configuration options were added, this includes:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ LDAP basedn: This allows the user to set the basedn for their LDAP configuration
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP bind password: This allows the user to set the password to use for binding to LDAP for creating the system context. If this is left blank along with bind principal then anonymous binding is used.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP bind principal: This allows the user to set the principle to use for binding with LDAP for creating the system context. If this is left blank along with the bind password then anonymous binding is used.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP email attribute: This sets out the attribute to use for getting the users email address. Within both OpenLDAP and ActiveDirectory this is mail. For this reason this is set to mail by default.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP realname attribute: This sets out the attribute to use for getting the users realname. Within both OpenLDAP and ActiveDiretory this is cn. For this reason this is set to cn by default.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP username attribute: This sets out the attribute to use for getting the users username. Within OpenLDAP this is uid and within ActiveDirectory this is samAccountName. In order to comply with posix standards this is set as uid by default.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ LDAP user object: This sets out the object type of user accounts within LDAP. Within OpenLDAP this is inetOrgPerson and within ActiveDirectory this is user. Again, in order to comply with posix standards this is set as inetOrgperson by default.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ With this implementation I believe it allows for a much more extendable and flexible approach. The whole implementation is abstracted from the Cloudstack codebase using the "plugin" model. This allows all of the LDAP features to be contained within one place. Along with this the implementation supplies a good foundation. A side affect of redoing the implementation allowed me to add support for multiple LDAP servers. This means failover is support, so for example, if you have a standard ActiveDirectory with primary and secondary domain controller. Both can be added to Cloudstack which will allow it to failover to either one assume one of them is down.
+ </para>
+ <para>
+ The API changes required me to update the UI interface within Cloudstack. With the improved API implementation this was easier. The Global Settings -> Ldap Configuration page has support for multiple LDAP servers however it only requires a hostname and port. All "global" ldap settings are set within the global settings page.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/ldap-global-settings.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>ldap-global-settings.png: Screenshot the LDAP related settings within global settings.</phrase>
+ </textobject>
+ </mediaobject>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/ldap-configuration.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>ldap-configuration.png: Screenshot of the LDAP configuration page.</phrase>
+ </textobject>
+ </mediaobject>
+ </section>
+ <section id="accounts-ui">
+ <title>Add accounts UI</title>
+ <para>
+ Extending the UI to allow for easy provisioning of LDAP users is currently a work in progress. At the moment I have a 'working' implementation, see below screenshot. I am in need of assistance with it and am waiting on a review to be looked at.
+ </para>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="./images/ldap-account-addition.png"/>
+ </imageobject>
+ <textobject>
+ <phrase>ldap-account-addition.png: Screenshot of add user screen when LDAP is enabled.</phrase>
+ </textobject>
+ </mediaobject>
+ </section>
+ <section id="testing">
+ <title>Testing</title>
+ <para>
+ Unit tests have 92% code coverage within the LDAP Plugin. The unit tests were wrote in groovy using the spock framework. This allowed for a BDD style of of testing.
+ </para>
+ <para>
+ Integration tests have been wrote in python using the marvin test framework for Cloudstack. This test configures a LDAP server and attempts to login as an LDAP user. The plugin comes with an embedded LDAP server for testing purposes.
+ </para>
+ <para>Execute integration tests:</para>
+ <programlisting>nosetests --with-marvin --marvin-config=setup/dev/local.cfg test/integration/component/test_ldap.py --loa</programlisting>
+ <para>Start embedded LDAP server:</para>
+ <programlisting>mvn -pl :cloud-plugin-user-authenticator-ldap ldap:run</programlisting>
+ </section>
+ <section id="conclusion">
+ <title>Conclusion</title>
+ <para>
+ I am very pleased with the learning outcomes of this project so far. I have been exposed to many things that my college's computer science curriculum does not cover. This includes:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>Usage of source control management tools(git) and dealing with code collaboration</para>
+ </listitem>
+ <listitem>
+ <para>Usage of a dependency manager and build tool(maven)</para>
+ </listitem>
+ <listitem>
+ <para>Usage of continous testing environments(jenkins)</para>
+ </listitem>
+ <listitem>
+ <para>Usage of an IDE(eclipse)</para>
+ </listitem>
+ <listitem>
+ <para>Exposure to testing, both unit and integration tests</para>
+ </listitem>
+ <listitem>
+ <para>Exposure to a functional programming language(groovy)</para>
+ </listitem>
+ <listitem>
+ <para>Exposure to web development libraries(jQuery)</para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ The experience gained from this project is invalueable and it is great that the Google Summer Of Code program exist.
+ </para>
+ </section>
</section>
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/gsoc-midsummer.xml
----------------------------------------------------------------------
diff --git a/docs/en-US/gsoc-midsummer.xml b/docs/en-US/gsoc-midsummer.xml
index ffb031a..74ca62a 100644
--- a/docs/en-US/gsoc-midsummer.xml
+++ b/docs/en-US/gsoc-midsummer.xml
@@ -11,9 +11,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/add-ldap-configuration-failure.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/add-ldap-configuration-failure.png b/docs/en-US/images/add-ldap-configuration-failure.png
new file mode 100644
index 0000000..4da295a
Binary files /dev/null and b/docs/en-US/images/add-ldap-configuration-failure.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/add-ldap-configuration.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/add-ldap-configuration.png b/docs/en-US/images/add-ldap-configuration.png
new file mode 100644
index 0000000..e041eb2
Binary files /dev/null and b/docs/en-US/images/add-ldap-configuration.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/delete-ldap-configuration-failure.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/delete-ldap-configuration-failure.png b/docs/en-US/images/delete-ldap-configuration-failure.png
new file mode 100644
index 0000000..b5b50e0
Binary files /dev/null and b/docs/en-US/images/delete-ldap-configuration-failure.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/delete-ldap.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/delete-ldap.png b/docs/en-US/images/delete-ldap.png
new file mode 100644
index 0000000..c97bb4c
Binary files /dev/null and b/docs/en-US/images/delete-ldap.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/jenkins-pipeline.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/jenkins-pipeline.png b/docs/en-US/images/jenkins-pipeline.png
new file mode 100644
index 0000000..0788c26
Binary files /dev/null and b/docs/en-US/images/jenkins-pipeline.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/ldap-account-addition.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/ldap-account-addition.png b/docs/en-US/images/ldap-account-addition.png
new file mode 100644
index 0000000..0c8573f
Binary files /dev/null and b/docs/en-US/images/ldap-account-addition.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/ldap-configuration.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/ldap-configuration.png b/docs/en-US/images/ldap-configuration.png
new file mode 100644
index 0000000..c840e59
Binary files /dev/null and b/docs/en-US/images/ldap-configuration.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/ldap-global-settings.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/ldap-global-settings.png b/docs/en-US/images/ldap-global-settings.png
new file mode 100644
index 0000000..0567de8
Binary files /dev/null and b/docs/en-US/images/ldap-global-settings.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/ldap-list-users.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/ldap-list-users.png b/docs/en-US/images/ldap-list-users.png
new file mode 100644
index 0000000..aedcb06
Binary files /dev/null and b/docs/en-US/images/ldap-list-users.png differ
http://git-wip-us.apache.org/repos/asf/cloudstack/blob/06484061/docs/en-US/images/list-ldap-configuration.png
----------------------------------------------------------------------
diff --git a/docs/en-US/images/list-ldap-configuration.png b/docs/en-US/images/list-ldap-configuration.png
new file mode 100644
index 0000000..6d75674
Binary files /dev/null and b/docs/en-US/images/list-ldap-configuration.png differ