You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hop.apache.org by mc...@apache.org on 2020/11/26 09:43:18 UTC
[incubator-hop-docs] branch asf-site updated: additional doc updates
This is an automated email from the ASF dual-hosted git repository.
mcasters pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-hop-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new edc65ac additional doc updates
new 44ebe5d Merge pull request #15 from mattcasters/asf-site
edc65ac is described below
commit edc65ac88286a8f6ae3fdcafa63fce07be7d5230
Author: Matt Casters <ma...@gmail.com>
AuthorDate: Thu Nov 26 10:41:39 2020 +0100
additional doc updates
---
.../modules/ROOT/pages/plugin-development.adoc | 86 ++++++++++++++++------
.../modules/ROOT/pages/setup-dev-environment.adoc | 37 +++++-----
2 files changed, 85 insertions(+), 38 deletions(-)
diff --git a/hop-dev-manual/modules/ROOT/pages/plugin-development.adoc b/hop-dev-manual/modules/ROOT/pages/plugin-development.adoc
index 1a8dac7..5832010 100644
--- a/hop-dev-manual/modules/ROOT/pages/plugin-development.adoc
+++ b/hop-dev-manual/modules/ROOT/pages/plugin-development.adoc
@@ -1,33 +1,73 @@
[[PluginDevelopment-PluginDevelopment]]
-= HOP Plugin Development
+= Hop Plugin Development
-Almost all the tranform capabilities and actions in HOP are plugins. This page explains how to develop new plugin with references to make the new plugin development easy.
+This page explains how to develop new plugins with references to make development easy.
-== Categorize the plugin type
+== Plugin Registry
-In HOP, all the plugins sources are placed under https://github.com/apache/incubator-hop/tree/master/plugins[hop/plugins] directory based on the category.
+Hop keeps a central registry of all the available Hop plugins. They are organized per plugin type (see below).
+The registry also keeps track of the class loaders that were created and used for the various plugins to avoid duplication.
+You can get a hold of this plugin registry at any place using
+[source]
+PluginRegistry.getInstance()
-- database -> for the plugins which provide database connectivity.
-- actions ->
-- engines ->
-- transforms ->
-- vfs -> Virutal files system
-- misc -> misceleneous (choose this only when other plugin types doesn't fit)
-TODO: table format would be nice with best/simple reference plugin for new plugin developer
+== Plugin types
-== Key Interfaces and Classes for plugins
-- IPluginType
-- BasePluginType
-- PluginRegistry
+Plugins allow you to extend Hop functionality in over 20 different areas.
+Each of these plugin types are represented by a class which implements ```IPluginType<T extends Annotation>```
+For your convenience you can extend a class called ```BasePluginType<T extends Annotation>```.
+The responsibility of these classes is to register plugins of the given type in the Hop Plugin Registry.
+It looks for these plugins in various ways:
-TODO: class diagram
+1. It reads an XML file if the plugin classes to load are in the classpath. This technique is used for the core plugins and a number of GUI plugins.
+You can find them in ```core/src/resources```, ```engine/src/resources``` and ```ui/src/resources```. The location of the file is defined using the ```getXmlPluginFile()``` method.
-=== Database
-- DatabaseMetaPlugin
-- DatabasePluginType
+2. It scans the ```plugins/``` subfolders and scans the jar files on the lookout for classes that have the annotation and interface specified in the plugin type. It then loads these classes and adds them to the plugin registry.
+== Plugins
+
+You can use the plugin registry to search for plugins using an ID, a class name. You can list all plugins and types.
+Plugins contain a lot of information and implement the ```IPlugin``` interface.
+Here are a few of the main fields:
+
+|===
+|Field |Type|Optional| Description
+
+|ids|String[]|Mandatory|The unique ID of the plugin within its type.
+A plugin can have more than 1 ID to make it possible to merge multiple plugins into one and still remain backward compatible.
+
+|name|String|Optional|The name of the plugin as shown in the user interface
+
+|description|String|Optional|The description of the plugin as shown in the user interface
+
+|category|String|Optional|used to organize certain types of plugins in the UI, for example transforms and actions
+
+|imageFile|String|Optional|The path to the SVG file representing the icon of this plugin.
+
+|documentationUrl|String|Optional|The URL of the documentation for the plugin
+
+|keywords|String[]|Optional|The keywords which represent this plugin to make it easier to find it in the UI
+
+|nativePlugin|boolean|false|Indicates if the plugin can be found in the classpath (using XML or API) or in a plugin classpath
+
+|===
+
+Aside from this list you can find the main class, the libraries in the plugin classpath and so on.
+The information in the IPlugin objects is copied from the plugin annotations on the plugin classes by the PluginType class.
+
+
+== Where are the Hop plugins?
+
+In Hop all optional plugins are in the https://github.com/apache/incubator-hop/tree/master/plugins[hop/plugins] folder which contains the plugin categories:
+
+- database: relational database plugins
+- actions: workflow actions
+- engines: pipeline and workflow engines
+- transforms: pipeline transforms
+- vfs: Virtual Files System plugins
+- misc: miscellaneous plugins. Pick this category only when other plugin types don't fit.
=== Actions
@@ -37,7 +77,8 @@ TODO: class diagram
Once you identified the category, create the plugin as a new maven sub module by carefully mentioning parent in your plugin pom file and adding new module in the parent project. This ensures adding the new plugin into maven build cycle.
-You can duplicate the reference plugin mentioned above as a starting point. When duplicating take time to modify the documentation and remove irrelavent to get the skeleton.
+You can duplicate the reference plugin mentioned above as a starting point.
+When duplicating take time to modify the documentation and remove irrelevant to get the skeleton.
THOUGHT: Can we create maven archetype? Contribution welcome.
@@ -45,5 +86,8 @@ THOUGHT: Can we create maven archetype? Contribution welcome.
To add the plugins into final artifacts, you need to add it to hop/assemblies/plugins directory under right category as a maven sub module by carefully mentioning parent in your plugin pom file and adding new module in the parent project. This ensures adding the new plugin into maven build cycle.
-Define src/assembly/assembly.xml with necessary dependent jars requred for the plugin to function. Review the jars which endup going into the plugin zip. If any jar which is already present in hop/lib directory, exclude it. This is for keeping the target hop artifact size as small as possible.
+Define src/assembly/assembly.xml with necessary dependent jars required for the plugin to function.
+Review the jars which end up going into the plugin zip.
+If any jar which is already present in hop/lib directory, exclude it.
+This is done to keep the target hop artifact size as small as possible.
diff --git a/hop-dev-manual/modules/ROOT/pages/setup-dev-environment.adoc b/hop-dev-manual/modules/ROOT/pages/setup-dev-environment.adoc
index c011fa3..4242b0d 100644
--- a/hop-dev-manual/modules/ROOT/pages/setup-dev-environment.adoc
+++ b/hop-dev-manual/modules/ROOT/pages/setup-dev-environment.adoc
@@ -50,26 +50,20 @@ To build your fork you can use Maven.
Run the following command to build Hop and run all unit tests:
[source]
-----
mvn clean install
-----
Please make sure all the files you added or changed have the proper license header.
You can run the following command to verify this:
[source]
-----
mvn apache-rat:check
-----
*IMPORTANT: At the very least make sure to run the above 2 commands before generating a pull request against the "upstream" Hop source code.*
*Tip* : to run Maven quicker in parallel and by skipping the unit tests you can use the following command:
[source]
-----
mvn -T4 -DskipTests=true clean install
-----
Replace 4 by the number of threads you can spend on your computer.
@@ -94,41 +88,34 @@ Updating your fork can be done simply by committing locally and then pushing tho
Make sure to always reference a HOP-xxxxx JIRA case! For example:
[source]
-----
git commit -m "HOP-98765 : My work description" .
-----
== Updating your fork
After a while it's likely your "origin" fork will be lagging behind with the main "upstream" Hop codebase.
+You can develop against the ```master``` branch of the codebase or you can create different branches for the features you're building or the JIRA cases you're fixing.
+In the commands below we assume you want to update your ```master``` branch. If this is not the case, replace ```master``` with the branch you're interested in and later want to generate a pull request for.
+
To update it you can add "upstream" to your local configuration:
[source]
-----
git remote add upstream git@github.com:apache/incubator-hop.git
-----
-Then you can fetch all the changes from "upstream"
+Then you can fetch all the changes from "upstream":
[source]
-----
git fetch upstream master
-----
We now want to catch up by pulling all the changes from "upstream" and by doing a rebase at the same time.
[source]
-----
git pull --rebase upstream master
-----
Now of-course we want to update our "origin" fork as well:
[source]
-----
git push --force-with-lease origin master
-----
== Generating a pull request
@@ -156,3 +143,19 @@ image::github-pull-request-checking.png[]
If the pull request doesn't build you can look at the details and fix it easily by simply pushing another commit to your "origin" fork.
It will be automatically added to the pull request and it will re-run the build and tests.
+== Evaluating a pull request
+
+If you want to review someone else's pull request you can check out a pull request in a different branch.
+Before you do this, commit or stash all the changes you're making yourself in the branch you're currently using.
+
+Then fetch the pull request changes themselves, with ```1234``` being the pull request number and ```pr1234``` the name of the new branch you want to create locally:
+
+[source]
+git fetch upstream pull/1234/head:pr1234
+
+Now we can check out this branch:
+
+[source]
+git checkout pr1234
+
+You now have the state of the source code in the pull request and you can build and test the project.