You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by ja...@apache.org on 2022/01/18 22:24:54 UTC
[solr] branch main updated: Developer docs for modules and packages (#531)
This is an automated email from the ASF dual-hosted git repository.
janhoy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/solr.git
The following commit(s) were added to refs/heads/main by this push:
new 402ff11 Developer docs for modules and packages (#531)
402ff11 is described below
commit 402ff11a8b1d827660e1deaaa63b9dccba1d857e
Author: Jan Høydahl <ja...@users.noreply.github.com>
AuthorDate: Tue Jan 18 23:23:33 2022 +0100
Developer docs for modules and packages (#531)
Co-authored-by: David Smiley <ds...@apache.org>
Co-authored-by: Eric Pugh <ep...@opensourceconnections.com>
Co-authored-by: Houston Putman <ho...@gmail.com>
---
dev-docs/plugins-modules-packages.adoc | 139 +++++++++++++++++++++++++++++++++
1 file changed, 139 insertions(+)
diff --git a/dev-docs/plugins-modules-packages.adoc b/dev-docs/plugins-modules-packages.adoc
new file mode 100644
index 0000000..7dc064d
--- /dev/null
+++ b/dev-docs/plugins-modules-packages.adoc
@@ -0,0 +1,139 @@
+= Plugins, Modules and Packages
+:toc: left
+
+This document discusses how developers can group features into Plugins, Modules and Packages.
+
+== Definitions
+
+=== Plugin
+
+A plugin is a low level class that implements a certain Lucene or Solr interface, such as
+`QParserPlugin`, `AuthPlugin`, `UpdateRequestProcessor`, `QueryResponseWriter` etc.
+See ref guide page https://solr.apache.org/guide/solr-plugins.html[Solr Plugins] for details.
+
+=== Module
+
+Solr modules are addon Solr plugins that are not part of solr-core, but officially maintained
+by the Solr project. They provide well-defined features such as the "extracting" module which lets
+users index rich text documents with Apache Tika. Modules were earlier known as "contribs".
+
+Each module produces a separate `.jar` file in the build, and additional dependencies required by each module are packaged with that module's `.jar` file. This helps keep the solr-core small and lean,
+and also reduces risk by including only needed java classes and dependencies on the class-path.
+
+A single module can contain multiple Plugins.
+
+=== Package
+
+A Package is a module or a 3rd party plugin for Solr provisioned with a `manifest.json`
+to enable discovering and loading through https://solr.apache.org/guide/package-manager.html[Solr's package manager]. Packages are loaded from a "package repository" which is
+simply a json file on a HTTP server.
+
+Packages can be maintained by 3rd party devs and hosted on
+GitHub, like https://github.com/yasa-org/yasa[YASA],
+https://github.com/rohitbemax/dataimporthandler[DataImportHandler],
+https://github.com/erikhatcher/solr-velocity[Velocity],
+https://github.com/cominvent/request-sanitizer-component[RequestSanitizerComponent]
+and https://solr.cool/[several others]. Such packages are not endorsed by the Solr project.
+
+The Solr project is also working on packaging our own (1st party) modules as packages,
+and host them either inside the binary tarball distribution of Solr or later also in the
+ASF download repos.
+
+== Dev guide for creating modules
+
+=== When to create a module
+
+Any functionality added to Solr that is not obviously useful for all Solr users and that
+spans more than a single source file, should be considered for a new module.
+
+Solr functionality that requires risky dependencies can more easily be added as a module, because Solr users will not have to load those dependencies by default.
+
+=== Module layout
+
+Modules consist of a `solr-<moduleName>-X.Y.Z.jar` artifact as often also a set of dependency
+`.jar` files that it needs to work (but not jars that are otherwise default a part of Solr).
+
+Module sources live in the `solr/modules/` folder in the checkout, and have their own gradle
+build file. See "Bootstrapping" paragraph on how to scaffold a new module.
+
+The layout of modules in the binary distribution is produced by the gradle build and is
+as follows:
+```
+<install-dir>/modules/<moduleName>/
+ +-- README.md
+ +-- lib/
+ +-- solr-<moduleName>-X.Y.Z.jar
+ +-- dependency-1.jar
+ +-- other-dependency.jar
+```
+
+=== Bootstrapping a new module
+
+It is not hard to create a new module. You can do it by hand, or you can run the helper tool:
+
+```bash
+dev-tools/scripts/scaffoldNewModule.py <short-name> <full name> <one line description>
+```
+
+Once you have the scaffold, start creating classes, or move out classes from solr-core.
+Solr-core and test-framework are added as dependencies by default, and you can add custom
+dependencies as necessary. Those will automatically end up in `<moduleName>/lib` in the
+binary distro.
+
+=== Module naming
+
+As the number of modules grow, we should strive to keep naming logical. The proposal is to
+structure module name as `<type>-<name>`.
+
+Types can be:
+
+* `analysis-` - for fieldTypes, tokenizers, token filters (lucene level)
+* `update-` - for UpdateRequestHandlers, UpdateRequestProcessors and other indexing related
+* `search-` - for QParser, ResponseWriters, SearchHandler, SearchComponent
+* `auth-` - for Authentication and Autorization
+* `backup-`- for backup repositories
+* ...more here...
+
+Examples:
+
+* `search-clustering`
+* `update-extraction`
+* `backup-gcs`
+
+== Dev guide for turning a module into a package
+
+=== Adding a manifest
+
+Create a file `<my-module>/src/main/resources/manifest.json`. It contents should be at a
+minimum something like below:
+
+```json
+{
+ "version-constraint": "9",
+ "plugins": [
+ {
+ "name": "update-extraction",
+ "setup-command": {
+ "path": "/api/collections/${collection}/config",
+ "payload": {"add-requesthandler": {"name": "${NAME}", "class": "update-extraction:org.apache.solr.handler.extraction.ExtractingRequestHandler"}},
+ "method": "POST"
+ },
+ "uninstall-command": {
+ "path": "/api/collections/${collection}/config",
+ "payload": {"delete-requesthandler": "${NAME}"},
+ "method": "POST"
+ }
+ }
+ ],
+ "parameter-defaults": {
+ "NAME": "/update/extract"
+ }
+}
+```
+
+Version constraint must follow SemVer expression syntax, `9` means it is
+compatible with any 9.x version of Solr. It is dangerous to assume compatibility with future versions, and for 1st party modules, we release them with every minor version, so consider
+using e.g. `9.1`, which will be compatible with all bugfix 9.1 releases.
+
+For details about how the package management internals work, please see
+https://solr.apache.org/guide/8_11/package-manager-internals.html[Package Manager Internals] chapter of the reference guide.
\ No newline at end of file