You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by ottobackwards <gi...@git.apache.org> on 2017/09/27 11:12:13 UTC
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/774
METRON-1212 The bundle system and maven plugin
This PR contains the Bundle system and Maven Plugin.
The bundle system and the plugin are adapted from the Apache Nifi project.
## bundles-maven-plugin
The bundles-maven-plugin is an adapted version of the jar dependency plugin who's function is to bundle a jar of jars based on the dependencies for a project. It also creates metadata attributes.
A project's jar, and it's non-provided dependency jars are place in a /lib entry in the bundle, with the bundle itself being in jar format.
## bundles-lib
The bundles-lib contains the functionality required to:
- discover bundles
- inspect bundles for exposed extension types
- load the bundles
- create special class loaders for bundles
- deliver instances of extension types for use
NAR exposed the bundles through many classes. I have created the BundleSystem interface to expose a more usable, simplified api for our use cases.
### From the original PR for METRON-777:
Metron Bundle Plugin
- adaptation of the nifi plugin
- more configurable wrt file extension/dependency and metadata naming
bundle-lib
- adaptation of nifi-nar-utils to be used outside of the nifi project
- rudimentary extensibility to allow configuration and injection of service types and other things that were hard coded to nifi
- refactored from File based to VFS based
- rebranding to Bundle from Nar ( although the lib and the plugin allow that to be configured now )
- added capability to the properties class to write to stream, adapted to uri from paths
- added integration tests for hdfs
- changed to be ClassIndex based instead of ServiceLoader. Service loader is slower, and Casey's ClassIndex work is great. This also removes the NAR's required manual maintenance of the service file.
- refactored to use VFS to load the bundle/nar into the classloader AND to use VFS to load the dependency jars -> VFS as a composite filesystem. Thus going from NAR's 'working directory', exploded NARS to just loading the bundle/nar.
## Previous Review
Please see [@mattf_apache's review](https://github.com/apache/metron/pull/530/files/c5f8c34e4de8e6d456b97edd6f8a0d33b4819d69)
## changes from that review
In preparing this PR I have:
- made checkstyle fixes
- fixed several types
- added a requested set of tests loading and executing simple interface/implementation from bundle beyond what is already in the bundle-lib tests
## Testing
This review is code review and test code review and running only
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Have you ensured that the full suite of tests and checks have been executed in the root metron \
- [x] Have you written or updated unit tests and or integration tests to verify your changes?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron fifth_bundles
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/774.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #774
----
commit 8aa3372c506b065849a4d868d1fd920733d0396d
Author: Otto Fowler <ot...@gmail.com>
Date: 2017-09-27T10:29:42Z
The bundle=lib and maven plugin for bundles.
as compared to the feature branch the differences are:
- Added some modules to test loading a bundle, and executing a function
- checkstyle changes
- some minor typo fixes
----
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
The new Trusty image breaks my build. I need to figure out why exactly, so I'll be trying a few things.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150077476
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
I refer to the discussion of this effort, that this PR's review would be theoretical and based on code review, unit tests, integration tests and the provided test of loading without direct dependencies ( bundles-tests ).
As stated in the PR description itself:
> - This review is code review and test code review and running only
Where are we missing each other on this? It seems we have a disconnect.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149783542
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
I should say the getting started and the quick-start
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
Bringing over this from #530,
https://github.com/apache/nifi/commit/d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6#diff-83e1afb34470ca47809f82aa1caf2138
is the commit in nifi for the nat-utils to diff.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
@nickwallen I have added a usage document in the last commit. Is this what you are looking for?
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
> @nickwallen: I have just started into this, but seems like I can create a bundle using the Maven plugin with this PR, right?
> @ottobackwards: yes you can
This was more of a leading question. Can we add this step to the test plan? Can you point to the directions on creating a bungle?
Is there anything else that should be part of the test plan?
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
@mattf-horton I have added some more documentation, and I think the bundles-testing project provides a simple, integrated example. @nickwallen and I talked it over, and he is going to try that.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
From a quick check of other PR travis failures, it looks like my build was the only one to have that problem ( which as an expected 1000 got 908 issue )
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149993579
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
I had thought that the point of your `bundles-maven-plugin` was to create the files and directory structure of a bundle; ie something like the ` bundles-testing` project. But based on your answer, I am assuming that with this PR I need to manually create that directory structure and files. Is that correct?
And then maybe in a future PR you have the code that creates this directory structure auto-magically?
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
yes you can
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
Sounds good @JonZeolla thanks
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
I have refactored to have a top level metron-bundles directory, with the bundles lib and bundles testing under there.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r151198323
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
no problem, let me know if the new doc is in line with what you are looking for
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
@nickwallen hopefully the Readme changes help.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
@nickwallen If is OK with you, i'll make it a separate file to the README.md
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150003217
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
The Archetype for creating **Extensions** does more than lay down the folder/pom.xml for creating the bundle. An Extension may or may not include a bundle, but also includes other things.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
> @ottobackwards hopefully the Readme changes help.
The Getting Started information that you added to the README is outstanding!
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
Thanks for looking at this again Nick, let me try to answer your questions.
TL;DR
--------
I'm sorry for my confusion with your original statement. I had in mind this part of @mattf-horton 's email on these pull requests:
> The first PR will be just the “bundle” mechanism and the maven plugin.
> Both are adaptations from the Apache NiFi project, and have already been
> reviewed as part of PR# 530. Review is acknowledged to be purely
> theoretical, and testing is based on the junit tests and integration
> tests.
I thought an example project to look at would answer your question, not that you actually wanted to do it, or have instructions to do it. Especial since you didn't say "the readme should have better instructions for creating a project that uses the plugin".
While creating bundles by hand is not in the initial use case, if you feel it is important I'll write a readme.
---------
As with the NiFi version of the maven plugin, the 100% use case in practice ( and for initial dev in metron ) is to have bundles create as part of the archetype project generation. When I 'adapted' the maven plugin, which consisted of adding the ability to rename the extension of the product and change the metadata prefixes, I did not make changes to the documentation of the plugin beyond documenting my changes. I also did not attempt to add maven test harness support for the plugin, as it was not present in the original project. Thus, the create by hand scenario for our version is as lacking as the create by hand scenario is in the origin nifi project. I tried to avoid changing the plugin as much as possible, as it is already in use in NiFi.
As we had discussed in the threads and the meetings, this initial PR would contain the plugin and the bundle, and the practical testing would be limited. I created and added reference to the test project because it is a simple example of a project that utilizes the plugin to create a bundle. I did this as opposed to writing out a step by step guide to how to create a multi module maven project, as I mistakenly thought this would be helpful given the context of the review.
@mattf-horton, would it be possible for you to help us with a comment as to your approach when you reviewed this functionality? It might help if you could summarize your feelings as well. The context is different, as the review within 777 could not be a +1 for the PR since this was only part of the total, but since you are the only one to have had reviewed this, your views are pretty important.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
I checkout your PR and then attempt to do an install and the build fails.
```
$ mvn clean install
[INFO] Scanning for projects...
Downloading: https://repo.maven.apache.org/maven2/org/apache/metron/bundles-maven-plugin/0.4.2/bundles-maven-plugin-0.4.2.pom
[WARNING] The POM for org.apache.metron:bundles-maven-plugin:jar:0.4.2 is missing, no dependency information available
Downloading: https://repo.maven.apache.org/maven2/org/apache/metron/bundles-maven-plugin/0.4.2/bundles-maven-plugin-0.4.2.jar
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Unresolveable build extension: Plugin org.apache.metron:bundles-maven-plugin:0.4.2 or one of its dependencies could not be resolved: Could not find artifact org.apache.metron:bundles-maven-plugin:jar:0.4.2 in central (https://repo.maven.apache.org/maven2) @
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project org.apache.metron:bundles-testing:0.4.2 (/Users/nallen/tmp/metron-pr774/bundles-testing/pom.xml) has 1 error
[ERROR] Unresolveable build extension: Plugin org.apache.metron:bundles-maven-plugin:0.4.2 or one of its dependencies could not be resolved: Could not find artifact org.apache.metron:bundles-maven-plugin:jar:0.4.2 in central (https://repo.maven.apache.org/maven2) -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException
```
I noticed that you changed the .travis build instructions so that you have to manually go into the `bundles-maven-plugin` project and install that first.
```
$ git diff origin/master -- .travis.yml
diff --git a/.travis.yml b/.travis.yml
index f5edfb27..8023830a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -33,6 +33,7 @@ before_install:
- npm config set prefix $HOME/.npm-prefix --global
install:
+ - cd bundles-maven-plugin && mvn -q install && cd ..
- time mvn -q -T 2C -DskipTests clean install
script:
```
Ideally we need to be able to just build Metron by running an install from the root directory. Is there no way around this? What troubles did you run into that caused you to have to do this?
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
> Test Project can be examined as a simple example of how to create bundles.
@ottobackwards Maybe I'm being really stupid, but I don't understand how the test-bundles-plugin shows me how to create bundles.
Wouldn't I use the code in this PR to create something like the test-bundles-plugin? Does that make sense?
What command do I run to create a bundle? How do I validate that the bundle created is valid?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149994944
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
In a [previous comment](https://github.com/apache/metron/pull/774#issuecomment-335012079), you have a link to a [`test-plugin-bundles`](https://github.com/ottobackwards/test-bundles-plugin) project. In that README, you have the following.
> A module generated from the maven quickstart archetype. Note that...
This has made me believe that there is a Maven archetype to create the bundle project.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
I have created a new feature branch feature/METRON-1211-extensions-parsers-gradual to track this.
and have rebased this PR on to that.
I have also updated confluence and jira : https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions
1. Feature Branch still makes sense
2. now that we are splitting it up it will work better
3. I don't want to have to worry about regression for short periods
Will email list
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150620372
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Let me clarify, when I say that nifi doesn't have this documented, I mean that their documentation, as mine ( though not included in this PR ) revolves around creating NARs of NiFi components, and not generic NARs.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150558916
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Nifi does not have this documented, and does not have a 'generic' archetype. @nickwallen I wonder if an archetype that just creates a simple bundle project would be useful? Would that be more of what you are looking for than the bundles-testing? Or would a written guide be the thing? Does this guide have to document how to create the multi module project? Does it have to document both existing and from scratch scenarios?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149755588
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Overall to test your PR, I want to do the following...
1. Create a bundle
2. Write some custom code in that bundle
3. Deploy the bundle somehow and see the code in that bundle execute
I am now trying to complete step 1 and just create a bundle. Isn't there a `mvn archetype:generate` command that I have to run to create a bundle? Where are the docs outlining those steps? I would have expected to see them here, I think, but I don't
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
On that day the Flat File test failed. Since it worked after open/close I wrote it off to a load / timing issue.
https://travis-ci.org/apache/metron/builds/280393026
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r151196532
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Sorry @ottobackwards. Not ignoring you. I have just been busy. Will get back to this soon. Thanks for your hard work in helping me through this.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
> This review is code review and test code review and running only
I have just started into this, but seems like I can create a bundle using the Maven plugin with this PR, right?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150000166
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
The point of the bundles maven plugin is to take the product of the module declared in it's dependency section, and create a bundle out of that module's jar, and all of that jar's dependencies. That bundle having a specific 'directory'/entry structure
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards closed the pull request at:
https://github.com/apache/metron/pull/774
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
GitHub user ottobackwards reopened a pull request:
https://github.com/apache/metron/pull/774
METRON-1212 The bundle system and maven plugin
This PR contains the Bundle system and Maven Plugin.
The bundle system and the plugin are adapted from the Apache Nifi project.
## bundles-maven-plugin
The bundles-maven-plugin is an adapted version of the jar dependency plugin whose function is to bundle a jar of jars based on the dependencies for a project. It also creates metadata attributes.
A project's jar, and it's non-provided dependency jars are place in a /lib entry in the bundle, with the bundle itself being in jar format.
## bundles-lib
The bundles-lib contains the functionality required to:
- discover bundles
- inspect bundles for exposed extension types
- load the bundles
- create special class loaders for bundles
- deliver instances of extension types for use
NAR exposed the bundles through many classes. I have created the BundleSystem interface to expose a more usable, simplified api for our use cases.
### From the original PR for METRON-777:
Metron Bundle Plugin
- adaptation of the nifi plugin
- more configurable wrt file extension/dependency and metadata naming
bundle-lib
- adaptation of nifi-nar-utils to be used outside of the nifi project
- rudimentary extensibility to allow configuration and injection of service types and other things that were hard coded to nifi
- refactored from File based to VFS based
- rebranding to Bundle from Nar ( although the lib and the plugin allow that to be configured now )
- added capability to the properties class to write to stream, adapted to uri from paths
- added integration tests for hdfs
- changed to be ClassIndex based instead of ServiceLoader. Service loader is slower, and Casey's ClassIndex work is great. This also removes the NAR's required manual maintenance of the service file.
- refactored to use VFS to load the bundle/nar into the classloader AND to use VFS to load the dependency jars -> VFS as a composite filesystem. Thus going from NAR's 'working directory', exploded NARS to just loading the bundle/nar.
## Previous Review
Please see [@mattf_apache's review](https://github.com/apache/metron/pull/530/files/c5f8c34e4de8e6d456b97edd6f8a0d33b4819d69)
## changes from that review
In preparing this PR I have:
- made checkstyle fixes
- fixed several types
- added a requested set of tests loading and executing simple interface/implementation from bundle beyond what is already in the bundle-lib tests
## Testing
This review is code review and test code review and running only
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Have you ensured that the full suite of tests and checks have been executed in the root metron \
- [x] Have you written or updated unit tests and or integration tests to verify your changes?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron fifth_bundles
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/774.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #774
----
commit 8aa3372c506b065849a4d868d1fd920733d0396d
Author: Otto Fowler <ot...@gmail.com>
Date: 2017-09-27T10:29:42Z
The bundle=lib and maven plugin for bundles.
as compared to the feature branch the differences are:
- Added some modules to test loading a bundle, and executing a function
- checkstyle changes
- some minor typo fixes
commit cc141ed36e24eaca1772ad237202e69da88158db
Author: Otto Fowler <ot...@gmail.com>
Date: 2017-09-27T11:48:50Z
update readme based on new build options around delay loading
----
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by JonZeolla <gi...@git.apache.org>.
Github user JonZeolla commented on the issue:
https://github.com/apache/metron/pull/774
@nickwallen I am doing some functional testing, but I'm not comfortable enough with it to give a full +1 on my own. When I'm done I will report back.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
No, I need to get back to this. Are you? Don't let me stop your own review.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149999458
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Ok, let me try to detangle this:
- ` A module generated from the maven quickstart archetype` : there is an actual maven quickstart archetype you an use to create a simple maven module project. This is not related to the plugin:
The expanded version of that comment in the readme would be:
"This module, whose product jar is what we are going to make a bundle of, was created using the maven quickstart archetype"
I am sorry, I did not understand you were looking at test-plugin-bundles when you made your comment.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
FlatFile tests are failing with timeouts, kicking
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards closed the pull request at:
https://github.com/apache/metron/pull/774
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
# Testing the bundle plugin
I have created a sample project that can be examined for testing the plugin:
[test-bundles-plugin](https://github.com/ottobackwards/test-bundles-plugin)
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150002599
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
There is not archetype for creating a bundle setup. To create a bundle of another module, you create the module folder and a simple pom.xml, listing the target module as a dependency. As I put in the readme, and tried to show in the sample projects.
Of the sample projects:
test-plugin-bundles: Is a simple project to create a bundle of a project
bundles-testing : is a more complicated but still simple project that has:
- A module for interfaces
- A module for implementation of an interface
- A module that creates a bundle of the implementation module
- An integration test that only references the bundles-lib and the interface module. This test uses the BundleSystem to load and execute to the implementation
Between these projects and the unit test it is my hope that a reviewer may not only get a handle from example on the basics of bundle projects and loading a bundle in a project, but also back within the agreed upon scope of the review see code examples as well.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/774
@ottobackwards Are we have intermittent test failures again?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150053090
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
> but also back within the agreed upon scope of the review see code examples as well.
Am I going outside some kind of boundaries here? Please clarify
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149985390
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Can you point me to a specific file and section? I can't find instructions in the 3 new READMEs that I see in the PR. Maybe I am just missing it somehow.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by mattf-horton <gi...@git.apache.org>.
Github user mattf-horton commented on the issue:
https://github.com/apache/metron/pull/774
@ottobackwards says:
> would it be possible for you to help us with a comment as to your approach when you reviewed this functionality? It might help if you could summarize your feelings as well
{with music}
Oh spelunking we shall go, spelunking we shall go... :-)
{with music}
While reviewing the original incarnation of this, I tried to focus on the part most other Metron community members wouldn't be familiar with, the NiFi Bundle implementation, which I have some (small) experience with due to past work with NiFi. The review I did was an eyeball line-by-line code review. Of course actually running the code is also necessary, and at the time @mmiklavc was focusing on that side of things.
Knowing that (a) the NiFi feature works in NiFi, and (b) Otto's intent was to change it as little as possible while porting it into Metron, I took an approach to reviewing it that focused on the **changes.** Specifically, as best I remember, I applied the following logic:
* I pulled a copy of the version of NiFi that Otto used as a base
* I pulled a copy of Otto's dev branch (submitted in PR#530)
* I observed relationships between directory paths of corresponding files (code organization relationships) in the two code sets, including filename changes (which were mostly pattern-based)
* I applied "diff -w" to corresponding file pairs
* I reviewed THAT as the code deltas of interest, the way we would code review a simple PR based on master
* I also reviewed in detail the small number of newly created files, and missing/deleted files, compared to the related NiFi sources
* I mapped my questions and remarks back to the submitted code in Otto's PR#530, and made the usual comments in that PR
* We interacted and followed up as usual.
After many days and some untold number of comments, responses, and changes, I felt I understood and agreed with the state of the code, and gave it a +1 here: https://github.com/apache/metron/pull/530#issuecomment-322970038 ,
contingent on the practical trials being completed.
Now, we've changed gears a bit. Faced with the difficulty of trying to review the offering all at once, whether in a PR or in a "feature branch" (turned out that wasn't much help after all), @ottobackwards and I proposed, and you'all accepted, the idea of breaking it up into five or so chunks. This was specified in my email to dev@metron.apache.org, 25 Sep 2017, Subject: Re: feature branch bumps (see http://mail-archives.apache.org/mod_mbox/metron-dev/201709.mbox/browser , Thread view, page 2, about 3/4 way down the page).
It was specifically called out that for the first chunk, consisting of just the “bundle” mechanism and the maven plugin,
> "Review is acknowledged to be purely theoretical, and testing is based on the junit tests and integration tests. We’ll add a simple integration test with a synthetic test jar containing a trivial
Class, unrelated to parsers, invoked in a test case via reflection (hence needing no glue
code). After passing that level of review and testing, it would be committed, with an understanding that additional revisions might be required as the result of subsequent PR comments.
So on the one hand, we agreed to a fairly light-weight review process, since by the nature of a piece-wise submission, the first part can't be tested on parsers. I encourage you to stick with that resolve.
On the other hand, the simple integration test suggested does call for being able to see a "hello-world" Stellar command loaded and run from a bundle, which actually would be kinda exciting.
@ottobackwards , can your test bundle be turned into a Stellar function, loaded, and invoked (by reflection so you don't need to worry about registration)? That would look more like an integration test than that bundle hanging out by itself.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
@bbende
this is going in as a separate PR now. I am not sure if you are interest or not, but I figure I'll give you a heads up.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
I'm sorry nick, the instruction for 'how to build metron' from the top level readme didn't get pulled over when I did the reconstruction.
The plugin is not in any repo for maven to install from. I don't know how to get it installed since we don't publish it. I will check my list history, but I think we (not you necessarily) talked about doing it this way until we figured out if we were going to publish the plugin.
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149989360
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
Are you looking for instructions on how to create the bundles-testing project step by step?
like : create a folder. in that folder create 4 folders. create a file pom.xml.... ?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150092533
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
> This review is code review and test code review and running only
I disagree with that statement. I don't know why I shouldn't be able to exercise this code like any other PR. The functionality is here to do that. You told me that the bundles-testing project has an integration test, that "has load and execute of a bundle." All I want to do, is do that manually so I can get a feel for the functionality.
But apologies if you'd prefer not to do this. If you'd like to get someone else's thumbs up on this according to those ground rules, then feel free.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by JonZeolla <gi...@git.apache.org>.
Github user JonZeolla commented on the issue:
https://github.com/apache/metron/pull/774
@nickwallen are you still looking at this?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r150106587
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
I wish you would have stated your disagreement during the community meeting, or in the email list on the 26th when you gave a +1 to this.
It is unfortunate that we again have gone months on a PR in this now year long effort to no effect, especially since as stated this was supposed to be the 'lighter' review of all the pr's.
That being said, we should not be discouraging reviews under any circumstances, and your review certainly matters.
So please. Can you explain what would would like to do based on your current understanding now that I have attempted to clarify things, and tried to describe the projects that are present?
---
[GitHub] metron pull request #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/774#discussion_r149772760
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building
+
+Building the bundles-maven-plugin module should be rare since it will be released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin module, you should next
+ go to the [metron](../metron) directory and follow its instructions.
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a toolkit etc, you may want to create on manually, or may need to create a project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 'bundle'.
+
+```xml
+<packaging>bundle</packaging>
+```
+
+This means that when you package this module, any of it's non-provided dependencies will be packaged into the produced bundle ( and all of their non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│ ├── pom.xml
+│ ├── src
+│ │ ├── main
+│ │ │ └── java
+│ │ │ └── org
+│ │ │ └── apache
+│ │ │ └── test
+│ │ │ └── App.java
+│ │ └── test
+│ │ └── java
+│ │ └── org
+│ │ └── apache
+│ │ └── test
+│ │ └── AppTest.java
+└── testappbundle
+ ├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, and contains the following pom.xml:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>test.bundles.plugin</artifactId>
+ <groupId>org.apache.test</groupId>
+ <version>1.0-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>test.app.bundle</artifactId>
+
+ <!-- Packaging is bundle -->
+ <packaging>bundle</packaging>
+
+ <!-- All dependencies of this module, and all the dependencies of THAT dependency will
+ be included in the produced bundle -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.test</groupId>
+ <artifactId>test.app</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <!-- OUR PLUGIN REFERENCES -->
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ <configuration>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>bundles-maven-plugin</artifactId>
+ <version>0.4.2</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
+```
+When the module is packaged, it packages all of it's non-provided dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided dependencies, you add that module to your
+bundle modules dependencies. You can unzip and examine the bundle in the target directory, and verify
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+ ├── MANIFEST.MF
+ ├── bundled-dependencies
+ │ ├── log4j-1.2.17.jar
+ │ ├── metron-common-0.4.1.jar
+ │ ├── slf4j-api-1.7.7.jar
+ │ ├── slf4j-log4j12-1.7.7.jar
+ │ └── test.app-1.0-SNAPSHOT.jar
+ └── maven
+ └── org.apache.test
+ └── test.app.bundle
+ ├── pom.properties
+ └── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+<dependencies>
+ <dependency>
+ <groupId>org.apache.metron</groupId>
+ <artifactId>metron-common</artifactId>
+ <version>0.4.1</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>3.8.1</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+```
+metron-common itself is a shaded jar, but it depends on log4j and slf4j, so those libraries are pulled in..
+
+## Quickstart
+
+* Create a new multi module maven project (if you do not have one already)
+* Add a new module for your bundle, it needs only to have a pom.xml
+* Create the pom.xml as above, with the correct plugin and packaging entries, and add dependencies
+for the module you want to bundle.
+* `mvn package`
--- End diff --
The getting started section of the plugin documentation tells you how to create a bundle.
The bundles-testing module is a simple result of following those instructions, with an added integration module with an example of using a bundle in code like you are looking to do.
The archetype is for creating specific types of projects that produce artifacts that may include a bundle. For example an archetype for creating a parser extension that produces an assembly with a bundle in it.
---
[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/774
I will extend the readme
---