You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ct...@apache.org on 2020/03/02 21:46:12 UTC
[lucene-solr] branch branch_8x updated: Ref Guide: Add license
header, typos in Package Manager docs
This is an automated email from the ASF dual-hosted git repository.
ctargett pushed a commit to branch branch_8x
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
The following commit(s) were added to refs/heads/branch_8x by this push:
new 902c601 Ref Guide: Add license header, typos in Package Manager docs
902c601 is described below
commit 902c6016d0efb18d8848ae410ce359a7eabaa8b6
Author: Cassandra Targett <ca...@lucidworks.com>
AuthorDate: Mon Mar 2 12:05:42 2020 -0600
Ref Guide: Add license header, typos in Package Manager docs
---
.../src/package-manager-internals.adoc | 156 +++++++++++++--------
solr/solr-ref-guide/src/package-manager.adoc | 1 -
2 files changed, 94 insertions(+), 63 deletions(-)
diff --git a/solr/solr-ref-guide/src/package-manager-internals.adoc b/solr/solr-ref-guide/src/package-manager-internals.adoc
index 467691b..1c73f26 100644
--- a/solr/solr-ref-guide/src/package-manager-internals.adoc
+++ b/solr/solr-ref-guide/src/package-manager-internals.adoc
@@ -1,11 +1,29 @@
= Package Manager Internals
+:page-tocclass: right
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
The package manager (CLI) internally uses various Solr APIs to install, deploy and update packages. This document contains an overview of those APIs.
== Salient Features
* Zero disruption deployment (hot deployment): Should be possible to install and update packages without node restarts or core reloads, and hence deployments should be quick and without failed requests or stale caches.
* Easy packaging:
-** Standard plugin concepts, e.g. query parser, search component, request handler, URP etc., should be supported without any special code/packaging changes.
+** Standard plugin concepts, e.g., query parser, search component, request handler, URP etc., should be supported without any special code/packaging changes.
** Artifacts (jars containing custom plugins) that users have already deployed (and are using in production) should be compatible, without needing to recompile or re-package, for greater adoption.
** Should support single jar packages as well as multiple jar packages.
** Use familiar / standard naming
@@ -44,22 +62,22 @@ $ bin/solr package add-key my_key.der
Package store is a distributed file store which can store arbitrary files in the file system.
* This is a fully replicated file system based repository.
-* It lives at <solr.home>/filestore on each Solr node
-* Every entry is a file + metadata. The metadata is named .<filename>.json
-* The metadata file contains the sha256, signatures of the file
-* Users can’t create files starting with period (.)
-* It is agnostic of content type of files. You may store jars as well as other files.
+* It lives at <solr.home>/filestore on each Solr node.
+* Every entry is a file + metadata. The metadata is named .<filename>.json.
+* The metadata file contains the sha256, signatures of the file.
+* Users can’t create files starting with period (.).
+* It is agnostic of content type of files. You may store jars as well as other files..
=== How Does the Package Store Work?
-When a file is uploaded to the PackageStore ,
+When a file is uploaded to the PackageStore, the following is true:
-* It’s saved to the local file system
+* It’s saved to the local file system.
* It’s saved along with the metadata. The metadata file stores the sha512 and signatures of the jar files too.
-* Every live node in the cluster is asked to download it as well
+* Every live node in the cluster is asked to download it as well.
=== Package Store APIs
-The end points are
+The end points are:
* `PUT /api/cluster/files/{full/path/to/file}` in each node
* `GET /api/node/files/{full/path/to/file}` to download the file
@@ -70,30 +88,34 @@ The end points are
Use the following steps to upload a jar signed with your public key:
-1) If you don't have a jar file with plugins, download a sample from github
-
+. If you don't have a jar file with plugins, download a sample from github:
++
[source, bash]
----
$ curl -o runtimelibs.jar -LO https://github.com/apache/lucene-solr/blob/master/solr/core/src/test-files/runtimecode/runtimelibs.jar.bin?raw=true
-
----
-2) Sign the jar with your private key
+
+. Sign the jar with your private key:
++
[source, bash]
----
$ openssl dgst -sha1 -sign my_key.pem runtimelibs.jar | openssl enc -base64 | sed 's/+/%2B/g' | tr -d \\n | sed
----
-3) Upload your jar with signature. (replace the `sig` param with the output you got from the previous command)
+. Upload your jar with signature, replacing the `sig` parameter with the output from the previous command:
++
[source, bash]
----
$ curl --data-binary @runtimelibs.jar -X PUT http://localhost:7574/api/cluster/files/mypkg/1.0/myplugins.jar?sig=<signature-of-jar>
----
-4) Verify your jar upload
+. Verify your jar upload:
++
[source, bash]
----
$ curl http://localhost:7574/api/node/files/mypkg/1.0?omitHeader=true
----
++
[source, json]
----
{
@@ -104,22 +126,21 @@ $ curl http://localhost:7574/api/node/files/mypkg/1.0?omitHeader=true
"sig":["elNjhmWIOgTgbAzeZ+OcwR42N7vqL6Ig9eAqn4YoP2thT7FJuhiaZuCPivjMkD682EBo9gveSCTyXIsZKjOCbQ=="]}]}}
----
-
== Packages
A Package have the following attributes:
* A unique name
* One or more versions with the following attributes:
-** `version` : The version string
-** `files` : An array of files from the package store
+** `version`: The version string
+** `files`: An array of files from the package store
-For every package / version in the packages definition, there is a unique `SolrResourceLoader` instance. This is a child of the `CoreContainer` resource loader.
+For every package/version in the packages definition, there is a unique `SolrResourceLoader` instance. This is a child of the `CoreContainer` resource loader.
=== packages.json
-The package configurations live in a file called /packages.json in ZooKeeper. At any given moment we can have multiple versions of a given package in the package configuration. The system will always use the latest version . Versions are sorted by their numeric value and the biggest is the latest.
+The package configurations live in a file called `packages.json` in ZooKeeper. At any given moment we can have multiple versions of a given package in the package configuration. The system will always use the latest version. Versions are sorted by their numeric value and the highest is the latest.
-*example:*
+For example:
[source,json]
----
@@ -142,41 +163,44 @@ The package configurations live in a file called /packages.json in ZooKeeper. At
* `GET /api/cluster/package` Get the list of packages
* `POST /api/cluster/package` edit packages
** `add` command: add a version of a package
-** `delete` command : delete a version of a package
+** `delete` command: delete a version of a package
=== How to Upgrade?
-Use the add command to add a version that is bigger than the current version
-=== How to Downgrade ?
-Use the delete command to delete a version that is the highest and the next highest version will be chosen
+Use the `add` command to add a version that is higher than the current version.
+
+=== How to Downgrade?
+
+Use the `delete` command to delete the highest version and choose the next highest version.
=== Using Multiple Versions in Parallel
-We use the `params.json` in the collection config to store a version of a package it uses. by default it is the $LATEST.
+
+We use `params.json` in the collection config to store a version of a package it uses. By default it is the `$LATEST`.
[source, json]
----
{"params":{
"PKG_VERSIONS": {
- "mypkg": "0.1",
- "pkg2" : "$LATEST",
+ "mypkg": "0.1", <1>
+ "pkg2" : "$LATEST", <2>
}}}
----
-* for `mypkg` use the version `0.1` irrespective of whether there is a newer version is available or not
-* for `pkg2` use the latest. This is optional. The default is ‘$LATEST’
+<1> For `mypkg`, use the version `0.1` irrespective of whether there is a newer version available or not.
+<2> For `pkg2`, use the latest. This is optional. The default is `$LATEST`.
-=== The Workflow
-* A new version of a package is added
-* The package loader loads the classes and notifies every plugin holder of the availability of the new version
-* It checks if it is supposed to use a specific version, Ignore the update
-* If not, reload the plugin
+=== Workflow
+* A new version of a package is added.
+* The package loader loads the classes and notifies every plugin holder of the availability of the new version.
+* It checks if it is supposed to use a specific version, Ignore the update.
+* If not, reload the plugin.
-=== Using Them in Plugins
-Any class name can be prefixed with the packagename e.g : `mypkg:fully.qualified.ClassName` and Solr would use the latest version of the package to load the classes from. The plugins loaded from packages cannot depend on core level classes.
+=== Using Packages in Plugins
-*Plugin declaration in `solrconfig.xml`*
+Any class name can be prefixed with the package name, e.g., `mypkg:fully.qualified.ClassName` and Solr would use the latest version of the package to load the classes from. The plugins loaded from packages cannot depend on core level classes.
+.Plugin declaration in `solrconfig.xml`
[source, xml]
----
<requestHandler name="/myhandler" class="mypkg:full.path.to.MyClass">
@@ -185,8 +209,8 @@ Any class name can be prefixed with the packagename e.g : `mypkg:fully.qualified
=== Full Working Example
-1) create a package
-
+. Create a package:
++
[source,bash]
----
curl http://localhost:8983/api/cluster/package -H 'Content-type:application/json' -d '
@@ -195,13 +219,14 @@ curl http://localhost:8983/api/cluster/package -H 'Content-type:application/jso
"version":"1.0",
"files" :["/mypkg/1.0/myplugins.jar"]}}'
----
-2) Verify the created package
+. Verify the created package:
++
[source,bash]
----
curl http://localhost:8983/api/cluster/package?omitHeader=true
----
-
++
[source,json]
----
{"result":{
@@ -211,20 +236,22 @@ curl http://localhost:8983/api/cluster/package?omitHeader=true
"files":["/mypkg/1.0/myplugins.jar"]}]}}}
----
-3) The Package is ready to use now. Now, register a plugin in your collection from the package. Note the `"mypkg"` prefix applied to the `class` attribute. The same result can be achieved by editing your `solrconfig.xml` as well
+. The package should be ready to use at this point. Next, register a plugin in your collection from the package. Note the `mypkg:` prefix applied to the `class` attribute. The same result can be achieved by editing your `solrconfig.xml` as well:
++
[source,bash]
----
curl http://localhost:8983/solr/gettingstarted/config -H 'Content-type:application/json' -d '{
- "create-requesthandler" : { "name" : "/test",
+ "create-requesthandler": { "name": "/test",
"class": "mypkg:org.apache.solr.core.RuntimeLibReqHandler" }}'
----
-4) Verify that the component is created and it is using the correct version of the package to load classes from
-
+. Verify that the component is created and it is using the correct version of the package to load classes from:
++
[source,bash]
----
curl http://localhost:8983/solr/gettingstarted/config/requestHandler?componentName=/test&meta=true&omitHeader=true
----
++
[source,json]
----
{
@@ -237,11 +264,13 @@ curl http://localhost:8983/solr/gettingstarted/config/requestHandler?componentNa
"files":["/mypkg/1.0/myplugins.jar"]}}}}}
----
-5) Test the request handler
+. Test the request handler:
++
[source,bash]
----
$ curl http://localhost:8983/solr/gettingstarted/test?omitHeader=true
----
++
[source,json]
----
{
@@ -255,23 +284,24 @@ $ curl http://localhost:8983/solr/gettingstarted/test?omitHeader=true
"loader":"java.net.FactoryURLClassLoader"}
----
-6) Update the version of our component
-
-Get a new version of the jar, sign and upload it
-
+. Update the version of our component. Get a new version of the jar, sign and upload it:
++
[source, bash]
----
$ curl -o runtimelibs3.jar -LO https://github.com/apache/lucene-solr/blob/master/solr/core/src/test-files/runtimecode/runtimelibs_v3.jar.bin?raw=true
+
$ openssl dgst -sha1 -sign my_key.pem runtimelibs.jar | openssl enc -base64 | sed 's/+/%2B/g' | tr -d \\n | sed
+
$ curl --data-binary @runtimelibs3.jar -X PUT http://localhost:8983/api/cluster/files/mypkg/2.0/myplugins.jar?sig=<signature>
----
-7) Verify it
+. Verify it:
++
[source, bash]
----
$ curl http://localhost:8983/api/node/files/mypkg/2.0?omitHeader=true
----
-
++
[source, json]
----
{
@@ -282,8 +312,8 @@ $ curl http://localhost:8983/api/node/files/mypkg/2.0?omitHeader=true
"sig":["ICkC+nGE+AqiANM0ajhVPNCQsbPbHLSWlIe5ETV5835e5HqndWrFHiV2R6nLVjDCxov/wLPo1uK0VzvAPIioUQ=="]}]}}
----
-8) Add a new version of the package
-
+. Add a new version of the package:
++
[source,bash]
----
$ curl http://localhost:8983/api/cluster/package -H 'Content-type:application/json' -d '
@@ -293,12 +323,13 @@ $ curl http://localhost:8983/api/cluster/package -H 'Content-type:application/j
"files" :["/mypkg/2.0/myplugins.jar"]}}'
----
-9) Verify the plugin to see if the correct version of the package is being used
+. Verify the plugin to see if the correct version of the package is being used:
++
[source,bash]
----
$ curl http://localhost:8983/solr/gettingstarted/config/requestHandler?componentName=/test&meta=true&omitHeader=true
----
-
++
[source,json]
----
{
@@ -316,12 +347,13 @@ $ curl http://localhost:8983/solr/gettingstarted/config/requestHandler?component
}}}}}
----
-10) Test the plugin
-
+. Test the plugin:
++
[source,bash]
----
$ curl http://localhost:8983/solr/gettingstarted/test?omitHeader=true
----
++
[source,json]
----
{
@@ -338,11 +370,11 @@ $ curl http://localhost:8983/solr/gettingstarted/test?omitHeader=true
"Version": "2"
}
----
-Note that the `Version` value is `"2"` . So the plugin is updated
+Note that the `Version` value is `"2"`, which means the plugin is updated.
=== How to Avoid Automatic Upgrade
-The default version used in any collection is always the latest. However, setting a per-collection property in the `params.json` ensures that the versions are always fixed irrespective of the new versions added
+The default version used in any collection is always the latest. However, setting a per-collection property in `params.json` ensures that the versions are always fixed irrespective of the new versions added.
[source,bash]
----
diff --git a/solr/solr-ref-guide/src/package-manager.adoc b/solr/solr-ref-guide/src/package-manager.adoc
index 94e948e..57a0212 100644
--- a/solr/solr-ref-guide/src/package-manager.adoc
+++ b/solr/solr-ref-guide/src/package-manager.adoc
@@ -1,7 +1,6 @@
= Package Management
:page-children: package-manager-internals
:page-tocclass: right
-
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information