You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@whimsical.apache.org by cu...@apache.org on 2017/04/07 22:15:02 UTC
[whimsy] branch master updated: Improve docs with contents, FAQ,
links
This is an automated email from the ASF dual-hosted git repository.
curcuru pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git
The following commit(s) were added to refs/heads/master by this push:
new 361d836 Improve docs with contents, FAQ, links
361d836 is described below
commit 361d83640a0eab7f92454ed1afbdbd88a313cfa6
Author: Shane Curcuru <as...@shanecurcuru.org>
AuthorDate: Fri Apr 7 18:14:56 2017 -0400
Improve docs with contents, FAQ, links
---
CONFIGURE.md | 4 +++-
DEPLOYMENT.md | 34 +++++++++++++++++++++-------
DEVELOPMENT.md | 70 ++++++++++++++++++++++++++++++++++++++++++----------------
3 files changed, 80 insertions(+), 28 deletions(-)
diff --git a/CONFIGURE.md b/CONFIGURE.md
index 22aa61e..f6e4511 100644
--- a/CONFIGURE.md
+++ b/CONFIGURE.md
@@ -30,10 +30,12 @@ Whimsy can be run on a client or in a local container for development use.
* **Tool-specific configurations** can be found in config/ directory
-## Production whimsy
+## Production Server Configuration
See [DEPLOYMENT.md](DEPLOYMENT.md) for full details. We use Puppet to
provision the production VM with the basic dependencies as well as the
Whimsy code. A number of configuration steps ensure the production instance
has access to LDAP, SVN repositories (some read/write), local mail
sending and receiving/subscriptions.
+
+**Log files** are LDAP secured to Members in: https://whimsy.apache.org/members/log/
diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md
index b6adaf3..9e48d74 100644
--- a/DEPLOYMENT.md
+++ b/DEPLOYMENT.md
@@ -1,10 +1,19 @@
-Deployment
+Deploying Production Whimsy.apache.org
==========
-The contents of this repository are automatically deployed to the https://whimsy.apache.org/ VM every 30 minutes.
+The contents of this repository are automatically deployed to the production
+https://whimsy.apache.org/ VM every 30 minutes - so be sure to test
+your changes before pushing to master.
-This VM runs Ubuntu 14.04 and is managed by Puppet. The puppet definition is
-contained in the following files:<a name="puppetnode"></a>
+Production Configuration
+==========
+
+The Whimsy VM runs Ubuntu 14.04 and is fully managed by Puppet using
+the normal methods Apache infra uses for managing servers. Note however
+that management of Whimsy is a PMC responsibility.
+
+<a name="puppetnode"></a>
+The **puppet definition** is contained in the following files:
* https://github.com/apache/infrastructure-puppet/blob/deployment/data/nodes/whimsy-vm3.apache.org.yaml
@@ -14,8 +23,11 @@ contained in the following files:<a name="puppetnode"></a>
* https://github.com/apache/infrastructure-puppet/blob/deployment/modules/whimsy_server/manifests/procmail.pp
-Instructions for local testing of deployment changes:
+Before pushing any changes here, understand the Apache Infra puppet workflow and test:
+ * https://cwiki.apache.org/confluence/display/INFRA/Git+workflow+for+infrastructure-puppet+repo
+ To understand the high-level workflow for puppet changes.
+
* https://github.com/apache/infrastructure-puppet-kitchen#readme
* addition to [Make modules useable](https://github.com/apache/infrastructure-puppet-kitchen#make-modules-useable) step:
@@ -24,11 +36,17 @@ Instructions for local testing of deployment changes:
echo "class zmanda_asf::client (){}" > zmanda_asf/manifests/client.pp
* https://github.com/apache/infrastructure-puppet/blob/deployment/modules/vhosts_whimsy/README.md
+ This details the changes to default puppet we use for Whimsy.
-Workflow:
+Configuration Locations
+----
+Application developers may need to know where different things are configured:
+
+- Much of the server config is in the puppet definition whimsy-vm3.apache.org.yaml
+- SVN updaters are in [repository.yml](repository.yml)
+- Public JSON generation comes from tools and controlled by whimsy_server/manifests/cronjobs.pp
+-
- * https://cwiki.apache.org/confluence/display/INFRA/Git+workflow+for+infrastructure-puppet+repo
-
Manual Steps
------------
diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md
index 867b1bb..350142a 100644
--- a/DEVELOPMENT.md
+++ b/DEVELOPMENT.md
@@ -4,8 +4,8 @@ Preface
Whimsy is a set of independent tools and a common library which typically will
need to access various ASF SVN directories and/or LDAP. To do development and
testing, you will need access to a machine on which you are willing to install
-libraries which do things like access LDAP, XML parsing, composing mail and
-the like for full functionality.
+libraries which do things like access LDAP, XML parsing, ASF Subversion repos,
+composing mail and the like for full functionality.
While some tools may work on Microsoft Windows, many don't currently.
Alternatives include a Docker image, a custom Vagrant VM, and a Kitchen/Puppet
@@ -14,6 +14,16 @@ of using an image or a VM is isolation. The primary disadvantage is that
you will need to install your SVN credentials there and arrange to either
duplicate or mount needed SVN directories.
+Contents :books:
+-------
+[Architecture](#architecture-overview)
+[Setup Whimsy Locally](#setup-whimsy-locally)
+[Running Whimsy Applications](#running-whimsy-applications)
+[Advanced Configuration](#advanced-configuration)
+[Server Configuration](DEPLOYMENT.md)
+[Further Reading](#further-reading)
+[How To / FAQ](#how-to-faq)
+
Architecture Overview
========
@@ -50,7 +60,7 @@ Setup Whimsy Locally
This section is for those desiring to run a whimsy tool on their own machine.
[See below for deploying](#advanced-configuration) in a Docker container or a Vagrant VM.
-1. Setup ruby 1.9.3 or higher. Verify with `ruby -v`.
+1. **Setup ruby 1.9.3 or higher.** Verify with `ruby -v`.
If you use a system provided version of Ruby, you may need to prefix
certain commands (like gem install) with `sudo`. Alternatives to using
the system provided version include using a Ruby version manager like
@@ -66,32 +76,31 @@ This section is for those desiring to run a whimsy tool on their own machine.
3. [Ruby Version Manager](https://rvm.io/)
-2. Install the `whimsy-asf` and `bundler` gems:
+2. **Install ruby gems:** `whimsy-asf` and `bundler`:
`gem install whimsy-asf bundler`
-3. SVN checkout ASF repositories into (or linked to from)
+3. **SVN checkout ASF repositories** into (or linked to from)
`/srv/svn`
svn co --depth=files https://svn.apache.org/repos/private/foundation
You can specify an alternate location for these directories by placing
- a [configuration file](CONFIGURE.md) named `.whimsy` in your home directory. The format
- for this file is YAML, and an example (be sure to include the dashed
- lines):
+ a YAML [configuration file](CONFIGURE.md) named `.whimsy` in your home
+ directory An minimal example (be sure to include the dashed lines!):
:svn:
- /home/rubys/svn/foundation
- /home/rubys/svn/committers
-4. Configure LDAP servers and certificates:
+4. **Configure LDAP** servers and certificates:
- 1. The model code determines what host and port to connect to by parsing
+ 1. The model code determines what host and port to connect to by parsing
either `/etc/ldap/ldap.conf` or `/etc/openldap/ldap.conf` for a line that
looks like the following:
`uri ldaps://ldap1-lw-us.apache.org:636`
- 2. A `TLS_CACERT` can be obtained via either of the following commands:
+ 2. A `TLS_CACERT` can be obtained via either of the following commands:
`ruby -r whimsy/asf -e "puts ASF::LDAP.cert"`<br/>
`openssl s_client -connect ldap1-lw-us.apache.org:636 </dev/null`
@@ -109,20 +118,21 @@ This section is for those desiring to run a whimsy tool on their own machine.
Note: the certificate is needed because the ASF LDAP hosts use a
self-signed certificate.
- **Simple way to configure LDAP is**:
+ Simple way to configure LDAP is:
sudo ruby -r whimsy/asf -e "ASF::LDAP.configure"
- These above command can also be used to update your configuration as
- the ASF changes LDAP servers.
+ These commands can also be used to update your configuration as
+ the ASF changes LDAP servers; they are cached in your `.whimsy`.
-5. Verify that the configuration is correct by running:
+5. **Verify your configuration** by running:
`ruby examples/board.rb`
+ It should print out an HTML page with current board members.
See comments in that file for running the script as a standalone server.
-6. Configure sending of mail (optional):
+6. **Configure mail sending** :mailbox_with_mail: (_optional_):
Configuration of outbound mail delivery is done through the `.whimsy`
file. Three examples are provided below, followed by links to where
@@ -153,7 +163,7 @@ This section is for those desiring to run a whimsy tool on their own machine.
[testmailer](http://www.rubydoc.info/github/mikel/mail/Mail/TestMailer), and
[filedelivery](http://www.rubydoc.info/github/mikel/mail/Mail/FileDelivery)
-Running Scripts/Applications
+Running Whimsy Applications
============================
If there is a `Gemfile` in the directory containing the script or application
@@ -186,7 +196,7 @@ command `bundle install`.
password. Should your ASF availid differ from your local user id,
set the `USER` environment variable prior to executing this command.
-Advanced configuration
+Advanced Configuration
======================
Setting things up so that the **entire** whimsy website is available as
@@ -221,7 +231,29 @@ a virtual host, complete with authentication:
start on dbus SIGNAL=SessionNew
exec /srv/whimsy/tools/toucher
-More details about the live Whimsy instance are in [DEPLOYMENT.md](DEPLOYMENT.md)
+More details about the production Whimsy instance are in [DEPLOYMENT.md](DEPLOYMENT.md)
+
+How To / FAQ
+============
+
+### How To: Create A New Whimsy CGI
+
+The simplest way to create a new standalone tool is copy an existing .cgi.
+Important things to check:
+
+- chmod 755 is likely needed
+- Double-check paths to the lib/asf files (which you will be using a lot)
+- Test locally first; in production logs are in [/members/log](https://whimsy.apache.org/members/log/)
+
+### How To: Use New SVN Directories
+
+Some SVN repos/files are checked out via cron jobs regularly for
+caching and read only access. Some applications checkout needed files
+just when running into temp dirs (typically to modify them and commit
+changes). If you have trouble using the existing [ASF::SVN classes](lib/whimsy/asf/svn.rb)
+class to access files from Subversion on the server, then check:
+
+- Default SVN checkouts: [repository.yml](repository.yml)
Further Reading
===============
--
To stop receiving notification emails like this one, please contact
['"commits@whimsical.apache.org" <co...@whimsical.apache.org>'].