You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by be...@apache.org on 2014/02/13 19:12:42 UTC
[44/57] [abbrv] couchdb commit: updated
refs/heads/1994-merge-rcouch-multi-repo to b19d055
remove couch_plugins
Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/20a68fec
Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/20a68fec
Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/20a68fec
Branch: refs/heads/1994-merge-rcouch-multi-repo
Commit: 20a68fec4d762ccb20862edf6d6b85fbd25344ee
Parents: ae6eae3
Author: Benoit Chesneau <bc...@gmail.com>
Authored: Thu Feb 13 16:38:39 2014 +0100
Committer: Benoit Chesneau <bc...@gmail.com>
Committed: Thu Feb 13 16:38:39 2014 +0100
----------------------------------------------------------------------
apps/couch_plugins/README.md | 159 -----------
apps/couch_plugins/src/couch_plugins.app.src | 23 --
apps/couch_plugins/src/couch_plugins.erl | 300 --------------------
apps/couch_plugins/src/couch_plugins_httpd.erl | 65 -----
4 files changed, 547 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/couchdb/blob/20a68fec/apps/couch_plugins/README.md
----------------------------------------------------------------------
diff --git a/apps/couch_plugins/README.md b/apps/couch_plugins/README.md
deleted file mode 100644
index b00a080..0000000
--- a/apps/couch_plugins/README.md
+++ /dev/null
@@ -1,159 +0,0 @@
-Heya,
-
-I couldn’t help myself thinking about plugin stuff and ended up
-whipping up a proof of concept.
-
-Here’s a <1 minute demo video:
-
- https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.mov
-
-Alternative encoding:
-
- https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.m4v)
-
-
-In my head the whole plugin idea is a very wide area, but I was so
-intrigued by the idea of getting something running with a click on a
-button in Futon. So I looked for a minimally viable plugin system.
-
-
-## Design principles
-
-It took me a day to put this all together and this was only possible
-because I took a lot of shortcuts. I believe they are all viable for a
-first iteration of a plugins system:
-
-1. Install with one click on a button in Futon (or HTTP call)
-2. Only pure Erlang plugins are allowed.
-3. The plugin author must provide a binary package for each Erlang (and,
- later, each CouchDB version).
-4. Complete trust-based system. You trust me to not do any nasty things
- when you click on the install button. No crypto, no nothing. Only
- people who can commit to Futon can release new versions of plugins.
-5. Minimal user-friendlyness: won’t install plugins that don’t match
- the current Erlang version, gives semi-sensible error messages
- (wrapped in a HTTP 500 response :)
-6. Require a pretty strict format for binary releases.
-
-
-## Roadmap
-
-Here’s a list of things this first iterations does and doesn’t do:
-
-- Pure Erlang plugins only. No C-dependencies, no JavaScript, no nothing.
-- No C-dependencies.
-- Install a plugin via Futon (or HTTP call). Admin only.
-- A hardcoded list of plugins in Futon.
-- Loads a pre-packaged, pre-compiled .tar.gz file from a URL.
-- Only installs if Erlang version matches.
-- No security checking of binaries.
-- No identity checking of binaries.
-- Register installed plugins in the config system.
-- Make sure plugins start with the next restart of CouchDB.
-- Uninstall a plugin via Futon (or HTTP call). Admin only.
-- Show when a particular plugin is installed.
-- Only installs if CouchDB version matches.
-- Serve static web assets (for Futon/Fauxton) from `/_plugins/<name>/`.
-
-I hope you agree we can ship this with a few warnings so people can get a
-hang of it.
-
-
-A roadmap, progress and issues can be found here:
-
-https://issues.apache.org/jira/issues/?jql=component+%3D+Plugins+AND+project+%3D+COUCHDB+AND+resolution+%3D+Unresolved+ORDER+BY+priority+DESC
-
-
-
-## How it works
-
-This plugin system lives in `src/couch_plugins` and is a tiny CouchDB
-module.
-
-It exposes one new API endpoint `/_plugins` that an admin user can
-POST to.
-
-The additional Futon page lives at `/_utils/plugins.html` it is
-hardcoded.
-
-Futon (or you) post an object to `/_plugins` with four properties:
-
- {
- "name": "geocouch", // name of the plugin, must be unique
- "url": "http://people.apache.org/~jan", // “base URL” for plugin releases (see below)
- "version": "couchdb1.2.x_v0.3.0-11-g4ea0bea", // whatever version internal to the plugin
- "checksums": {
- "R15B03": "ZetgdHj2bY2w37buulWVf3USOZs=" // base64’d sha hash over the binary
- }
- }
-
-`couch_plugins` then attempts to download a .tar.gz from this
-location:
-
- http://people.apache.org/~jan/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03.tar.gz
-
-It should be obvious how the URL is constructed from the POST data.
-(This url is live, feel free to play around with this tarball).
-
-Next it calculates the sha hash for the downloaded .tar.gz file and
-matches it against the correct version in the `checksums` parameter.
-
-If that succeeds, we unpack the .tar.gz file (currently in `/tmp`,
-need to find a better place for this) and adds the extracted directory
-to the Erlang code path
-(`code:add_path("/tmp/couchdb_plugins/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03/ebin")`)
-and loads the included application (`application:load(geocouch)`).
-
-Then it looks into the `./priv/default.d` directory that lives next to
-`ebin/` in the plugin directory for configuration `.ini` files and loads them.
-On next startup these configuration files are loaded after global defaults,
-and before any local configuration.
-
-If that all goes to plan, we report success back to the HTTP caller.
-
-That’s it! :)
-
-It’s deceptively simple, probably does a few things very wrong and
-leaves a few things open (see above).
-
-One open question I’d like an answer for is finding a good location to
-unpack & install the plugin files that isn’t `tmp`. If the answer is
-different for a pre-BigCouch/rcouch-merge and post-BigCouch/rcouch-
-merge world, I’d love to know :)
-
-
-## Code
-
-The main branch for this is 1867-feature-plugins:
-
- ASF: https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/1867-feature-plugins
- GitHub: https://github.com/janl/couchdb/compare/apache:master...1867-feature-plugins
-
-I created a branch on GeoCouch that adds a few lines to its `Makefile`
-that shows how a binary package is built:
-
- https://github.com/janl/geocouch/compare/couchbase:couchdb1.3.x...couchdb1.3.x-plugins
-
-
-## Build
-
-Build CouchDB as usual:
-
- ./bootstrap
- ./configure
- make
- make dev
- ./utils/run
-
-* * *
-
-I hope you like this :) Please comment and improve heavily!
-
-Let me know if you have any questions :)
-
-If you have any criticism, please phrase it in a way that we can use
-to improve this, thanks!
-
-Best,
-Jan
---
http://git-wip-us.apache.org/repos/asf/couchdb/blob/20a68fec/apps/couch_plugins/src/couch_plugins.app.src
----------------------------------------------------------------------
diff --git a/apps/couch_plugins/src/couch_plugins.app.src b/apps/couch_plugins/src/couch_plugins.app.src
deleted file mode 100644
index 9e69b84..0000000
--- a/apps/couch_plugins/src/couch_plugins.app.src
+++ /dev/null
@@ -1,23 +0,0 @@
-% Licensed under the Apache License, Version 2.0 (the "License"); you may not
-% use this file except in compliance with the License. You may obtain a copy of
-% the License at
-%
-% http://www.apache.org/licenses/LICENSE-2.0
-%
-% Unless required by applicable law or agreed to in writing, software
-% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-% License for the specific language governing permissions and limitations under
-% the License.
-{application, couch_plugins,
- [
- {description, "A CouchDB Plugin Installer"},
- {vsn, "0.1.0"},
- {registered, []},
- {applications, [
- kernel,
- stdlib
- ]},
- {mod, { couch_plugins_app, []}},
- {env, []}
- ]}.
http://git-wip-us.apache.org/repos/asf/couchdb/blob/20a68fec/apps/couch_plugins/src/couch_plugins.erl
----------------------------------------------------------------------
diff --git a/apps/couch_plugins/src/couch_plugins.erl b/apps/couch_plugins/src/couch_plugins.erl
deleted file mode 100644
index dcbd2d3..0000000
--- a/apps/couch_plugins/src/couch_plugins.erl
+++ /dev/null
@@ -1,300 +0,0 @@
-% Licensed under the Apache License, Version 2.0 (the "License"); you may not
-% use this file except in compliance with the License. You may obtain a copy of
-% the License at
-%
-% http://www.apache.org/licenses/LICENSE-2.0
-%
-% Unless required by applicable law or agreed to in writing, software
-% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-% License for the specific language governing permissions and limitations under
-% the License.
--module(couch_plugins).
--include_lib("couch/include/couch_db.hrl").
--export([install/1, uninstall/1]).
-
-% couch_plugins:install({"geocouch", "http://127.0.0.1:8000", "1.0.0", [{"R15B03", "+XOJP6GSzmuO2qKdnjO+mWckXVs="}]}).
-% couch_plugins:install({"geocouch", "http://people.apache.org/~jan/", "couchdb1.2.x_v0.3.0-11-gd83ba22", [{"R15B03", "ZetgdHj2bY2w37buulWVf3USOZs="}]}).
-
-plugin_dir() ->
- couch_config:get("couchdb", "plugin_dir").
-
-log(T) ->
- ?LOG_DEBUG("[couch_plugins] ~p ~n", [T]).
-
-%% "geocouch", "http://localhost:8000/dist", "1.0.0"
--type plugin() :: {string(), string(), string(), list()}.
--spec install(plugin()) -> ok | {error, string()}.
-install({Name, _BaseUrl, Version, Checksums}=Plugin) ->
- log("Installing " ++ Name),
-
- {ok, LocalFilename} = download(Plugin),
- log("downloaded to " ++ LocalFilename),
-
- ok = verify_checksum(LocalFilename, Checksums),
- log("checksum verified"),
-
- ok = untargz(LocalFilename),
- log("extraction done"),
-
- ok = add_code_path(Name, Version),
- log("added code path"),
-
- ok = register_plugin(Name, Version),
- log("registered plugin"),
-
- load_config(Name, Version),
- log("loaded config"),
-
- ok.
-
-% Idempotent uninstall, if you uninstall a non-existant
-% plugin, you get an `ok`.
--spec uninstall(plugin()) -> ok | {error, string()}.
-uninstall({Name, _BaseUrl, Version, _Checksums}) ->
- % unload config
- ok = unload_config(Name, Version),
- log("config unloaded"),
-
- % delete files
- ok = delete_files(Name, Version),
- log("files deleted"),
-
- % delete code path
- ok = del_code_path(Name, Version),
- log("deleted code path"),
-
- % unregister plugin
- ok = unregister_plugin(Name),
- log("unregistered plugin"),
-
- % done
- ok.
-
-%% * * *
-
-
-%% Plugin Registration
-%% On uninstall:
-%% - add plugins/name = version to config
-%% On uninstall:
-%% - remove plugins/name from config
-
--spec register_plugin(string(), string()) -> ok.
-register_plugin(Name, Version) ->
- couch_config:set("plugins", Name, Version).
-
--spec unregister_plugin(string()) -> ok.
-unregister_plugin(Name) ->
- couch_config:delete("plugins", Name).
-
-%% * * *
-
-
-%% Load Config
-%% Parses <plugindir>/priv/default.d/<pluginname.ini> and applies
-%% the contents to the config system, or removes them on uninstall
-
--spec load_config(string(), string()) -> ok.
-load_config(Name, Version) ->
- loop_config(Name, Version, fun set_config/1).
-
--spec unload_config(string(), string()) -> ok.
-unload_config(Name, Version) ->
- loop_config(Name, Version, fun delete_config/1).
-
--spec loop_config(string(), string(), function()) -> ok.
-loop_config(Name, Version, Fun) ->
- lists:foreach(fun(File) -> load_config_file(File, Fun) end,
- filelib:wildcard(file_names(Name, Version))).
-
--spec load_config_file(string(), function()) -> ok.
-load_config_file(File, Fun) ->
- {ok, Config} = couch_config:parse_ini_file(File),
- lists:foreach(Fun, Config).
-
--spec set_config({{string(), string()}, string()}) -> ok.
-set_config({{Section, Key}, Value}) ->
- ok = couch_config:set(Section, Key, Value).
-
--spec delete_config({{string(), string()}, _Value}) -> ok.
-delete_config({{Section, Key}, _Value}) ->
- ok = couch_config:delete(Section, Key).
-
--spec file_names(string(), string()) -> string().
-file_names(Name, Version) ->
- filename:join(
- [plugin_dir(), get_file_slug(Name, Version),
- "priv", "default.d", "*.ini"]).
-
-%% * * *
-
-
-%% Code Path Management
-%% The Erlang code path is where the Erlang runtime looks for `.beam`
-%% files to load on, say, `application:load()`. Since plugin directories
-%% are created on demand and named after CouchDB and Erlang versions,
-%% we manage the Erlang code path semi-automatically here.
-
--spec add_code_path(string(), string()) -> ok | {error, bad_directory}.
-add_code_path(Name, Version) ->
- PluginPath = plugin_dir() ++ "/" ++ get_file_slug(Name, Version) ++ "/ebin",
- case code:add_path(PluginPath) of
- true -> ok;
- Else ->
- ?LOG_ERROR("Failed to add PluginPath: '~s'", [PluginPath]),
- Else
- end.
-
--spec del_code_path(string(), string()) -> ok | {error, atom()}.
-del_code_path(Name, Version) ->
- PluginPath = plugin_dir() ++ "/" ++ get_file_slug(Name, Version) ++ "/ebin",
- case code:del_path(PluginPath) of
- true -> ok;
- _Else ->
- ?LOG_DEBUG("Failed to delete PluginPath: '~s', ignoring", [PluginPath]),
- ok
- end.
-
-%% * * *
-
-
--spec untargz(string()) -> {ok, string()} | {error, string()}.
-untargz(Filename) ->
- % read .gz file
- {ok, GzData} = file:read_file(Filename),
- % gunzip
- log("unzipped"),
- TarData = zlib:gunzip(GzData),
- ok = filelib:ensure_dir(plugin_dir()),
- % untar
- erl_tar:extract({binary, TarData}, [{cwd, plugin_dir()}, keep_old_files]).
-
--spec delete_files(string(), string()) -> ok | {error, atom()}.
-delete_files(Name, Version) ->
- PluginPath = plugin_dir() ++ "/" ++ get_file_slug(Name, Version),
- mochitemp:rmtempdir(PluginPath).
-
-
-% downloads a pluygin .tar.gz into a local plugins directory
--spec download(string()) -> ok | {error, string()}.
-download({Name, _BaseUrl, Version, _Checksums}=Plugin) ->
- TargetFile = "/tmp/" ++ get_filename(Name, Version),
- case file_exists(TargetFile) of
- %% wipe and redownload
- true -> file:delete(TargetFile);
- _Else -> ok
- end,
- Url = get_url(Plugin),
- HTTPOptions = [
- {connect_timeout, 30*1000}, % 30 seconds
- {timeout, 30*1000} % 30 seconds
- ],
- % todo: windows
- Options = [
- {stream, TargetFile}, % /tmp/something
- {body_format, binary},
- {full_result, false}
- ],
- % todo: reduce to just httpc:request()
- case httpc:request(get, {Url, []}, HTTPOptions, Options) of
- {ok, _Result} ->
- log("downloading " ++ Url),
- {ok, TargetFile};
- Error -> Error
- end.
-
--spec verify_checksum(string(), list()) -> ok | {error, string()}.
-verify_checksum(Filename, Checksums) ->
-
- CouchDBVersion = couchdb_version(),
- case proplists:get_value(CouchDBVersion, Checksums) of
- undefined ->
- ?LOG_ERROR("[couch_plugins] Can't find checksum for CouchDB Version '~s'", [CouchDBVersion]),
- {error, no_couchdb_checksum};
- OTPChecksum ->
- OTPRelease = erlang:system_info(otp_release),
- case proplists:get_value(OTPRelease, OTPChecksum) of
- undefined ->
- ?LOG_ERROR("[couch_plugins] Can't find checksum for Erlang Version '~s'", [OTPRelease]),
- {error, no_erlang_checksum};
- Checksum ->
- do_verify_checksum(Filename, Checksum)
- end
- end.
-
--spec do_verify_checksum(string(), string()) -> ok | {error, string()}.
-do_verify_checksum(Filename, Checksum) ->
- ?LOG_DEBUG("Checking Filename: ~s", [Filename]),
- case file:read_file(Filename) of
- {ok, Data} ->
- ComputedChecksum = binary_to_list(base64:encode(crypto:sha(Data))),
- case ComputedChecksum of
- Checksum -> ok;
- _Else ->
- ?LOG_ERROR("Checksum mismatch. Wanted: '~p'. Got '~p'", [Checksum, ComputedChecksum]),
- {error, checksum_mismatch}
- end;
- Error -> Error
- end.
-
-
-%% utils
-
--spec get_url(plugin()) -> string().
-get_url({Name, BaseUrl, Version, _Checksums}) ->
- BaseUrl ++ "/" ++ get_filename(Name, Version).
-
--spec get_filename(string(), string()) -> string().
-get_filename(Name, Version) ->
- get_file_slug(Name, Version) ++ ".tar.gz".
-
--spec get_file_slug(string(), string()) -> string().
-get_file_slug(Name, Version) ->
- % OtpRelease does not include patch levels like the -1 in R15B03-1
- OTPRelease = erlang:system_info(otp_release),
- CouchDBVersion = couchdb_version(),
- string:join([Name, Version, OTPRelease, CouchDBVersion], "-").
-
--spec file_exists(string()) -> boolean().
-file_exists(Filename) ->
- does_file_exist(file:read_file_info(Filename)).
--spec does_file_exist(term()) -> boolean().
-does_file_exist({error, enoent}) -> false;
-does_file_exist(_Else) -> true.
-
-couchdb_version() ->
- couch_server:get_version(short).
-
-% installing a plugin:
-% - POST /_plugins -d {plugin-def}
-% - get plugin definition
-% - get download URL (matching erlang version)
-% - download archive
-% - match checksum
-% - untar-gz archive into a plugins dir
-% - code:add_path(“geocouch-{geocouch_version}-{erlang_version}/ebin”)
-% - [cp geocouch-{geocouch_version}-{erlang_version}/etc/ ]
-% - application:start(geocouch)
-% - register plugin in plugin registry
-
-% Plugin registry impl:
-% - _plugins database
-% - pro: known db ops
-% - con: no need for replication, needs to be system db etc.
-% - _config/plugins namespace in config
-% - pro: lightweight, fits rarely-changing nature better
-% - con: potentially not flexible enough
-
-
-
-% /geocouch
-% /geocouch/dist/
-% /geocouch/dist/geocouch-{geocouch_version}-{erlang_version}.tar.gz
-
-% tar.gz includes:
-% geocouch-{geocouch_version}-{erlang_version}/
-% geocouch-{geocouch_version}-{erlang_version}/ebin
-% [geocouch-{geocouch_version}-{erlang_version}/config/config.erlt]
-% [geocouch-{geocouch_version}-{erlang_version}/share/]
-
http://git-wip-us.apache.org/repos/asf/couchdb/blob/20a68fec/apps/couch_plugins/src/couch_plugins_httpd.erl
----------------------------------------------------------------------
diff --git a/apps/couch_plugins/src/couch_plugins_httpd.erl b/apps/couch_plugins/src/couch_plugins_httpd.erl
deleted file mode 100644
index 4dabbb4..0000000
--- a/apps/couch_plugins/src/couch_plugins_httpd.erl
+++ /dev/null
@@ -1,65 +0,0 @@
-% Licensed under the Apache License, Version 2.0 (the "License"); you may not
-% use this file except in compliance with the License. You may obtain a copy of
-% the License at
-%
-% http://www.apache.org/licenses/LICENSE-2.0
-%
-% Unless required by applicable law or agreed to in writing, software
-% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-% License for the specific language governing permissions and limitations under
-% the License.
--module(couch_plugins_httpd).
-
--export([handle_req/1]).
-
--include_lib("couch/include/couch_db.hrl").
-
-handle_req(#httpd{method='POST'}=Req) ->
- ok = couch_httpd:verify_is_server_admin(Req),
- couch_httpd:validate_ctype(Req, "application/json"),
-
- {PluginSpec} = couch_httpd:json_body_obj(Req),
- Url = binary_to_list(couch_util:get_value(<<"url">>, PluginSpec)),
- Name = binary_to_list(couch_util:get_value(<<"name">>, PluginSpec)),
- Version = binary_to_list(couch_util:get_value(<<"version">>, PluginSpec)),
- Delete = couch_util:get_value(<<"delete">>, PluginSpec),
- {Checksums0} = couch_util:get_value(<<"checksums">>, PluginSpec),
- Checksums = parse_checksums(Checksums0),
-
- Plugin = {Name, Url, Version, Checksums},
- case do_install(Delete, Plugin) of
- ok ->
- couch_httpd:send_json(Req, 202, {[{ok, true}]});
- Error ->
- ?LOG_DEBUG("Plugin Spec: ~p", [PluginSpec]),
- couch_httpd:send_error(Req, {bad_request, Error})
- end;
-% handles /_plugins/<pluginname>/<file>
-% serves <plugin_dir>/<pluginname>-<pluginversion>-<otpversion>-<couchdbversion>/<file>
-handle_req(#httpd{method='GET',path_parts=[_, Name0 | Path0]}=Req) ->
- Name = ?b2l(Name0),
- Path = lists:map(fun binary_to_list/1, Path0),
- OTPRelease = erlang:system_info(otp_release),
- PluginVersion = couch_config:get("plugins", Name),
- CouchDBVersion = couch_server:get_version(short),
- FullName = string:join([Name, PluginVersion, OTPRelease, CouchDBVersion], "-"),
- FullPath = filename:join([FullName, "priv", "www", string:join(Path, "/")]) ++ "/",
- ?LOG_DEBUG("Serving ~p from ~p", [FullPath, plugin_dir()]),
- couch_httpd:serve_file(Req, FullPath, plugin_dir());
-handle_req(Req) ->
- couch_httpd:send_method_not_allowed(Req, "POST").
-
-plugin_dir() ->
- couch_config:get("couchdb", "plugin_dir").
-do_install(false, Plugin) ->
- couch_plugins:install(Plugin);
-do_install(true, Plugin) ->
- couch_plugins:uninstall(Plugin).
-
-parse_checksums(Checksums) ->
- lists:map(fun({K, {V}}) ->
- {binary_to_list(K), parse_checksums(V)};
- ({K, V}) ->
- {binary_to_list(K), binary_to_list(V)}
- end, Checksums).