You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by justinleet <gi...@git.apache.org> on 2017/04/20 17:39:53 UTC
[GitHub] incubator-metron pull request #538: METRON-868 Fix documentation on building...
GitHub user justinleet opened a pull request:
https://github.com/apache/incubator-metron/pull/538
METRON-868 Fix documentation on building RPMs
## Contributor Comments
Update doc to run install instead of package as a prereq to the RPM build.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron (Incubating).
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
bin/generate-md.sh
mvn site:site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/justinleet/incubator-metron rpm_instructions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/538.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #538
----
commit e18e84af2a40e2b828828001a181ecbd3348bcfe
Author: justinjleet <ju...@gmail.com>
Date: 2017-04-20T17:22:35Z
changing from package to install
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-metron issue #538: METRON-868 Fix
documentation on building RPMs
Posted by Otto Fowler <ot...@gmail.com>.
Sorry to be late to this, but since METRON-777 will have to add to this
list, can we get a summary of what we think is going on?
On April 24, 2017 at 10:23:57, mmiklavc (git@git.apache.org) wrote:
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@justinleet Works for me. Please indicate that we have a dependency across
projects that is not shown via the dependency mechanism. Specifically, we
have a dependency on the tarballs output via these projects:
```
<dependencies>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-common</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-data-management</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-elasticsearch</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-enrichment</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-indexing</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-parsers</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-pcap-backend</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-solr</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-profiler</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-config</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
```
@dlyle65535 I had tried that also re: explicit jar dependency in our
project. But there's still something not quite right in our deps. The build
doesn't seem to find the artifacts even though they're in the reactor tree
when I do that. I don't think it's trying to game the dependency mechanism,
as Maven doesn't really differentiate between the artifacts and the module
itself - they're pretty tightly coupled afa Maven is concerned. But
absolutely agreed about the build order with the pom deps not having worked
as intended (per your interleaved output above). I suspect you're right
about the pom declaration behind handled differently, at least from a
multi-threaded perspective. I'm unclear if this is a bug in Maven's
multi-threaded mode or intended behavior, but we certainly don't need to
deal with it here.
We should look at this further in a separate PR. While we're at it, we
might even simplify things further and remove the profiles, which are
typically used for building for different environments, not acting as
virtual-modules.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@justinleet Works for me. Please indicate that we have a dependency across projects that is not shown via the dependency mechanism. Specifically, we have a dependency on the tarballs output via these projects:
```
<dependencies>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-common</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-data-management</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-elasticsearch</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-enrichment</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-indexing</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-parsers</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-pcap-backend</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-solr</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-profiler</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-config</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
```
@dlyle65535 I had tried that also re: explicit jar dependency in our project. But there's still something not quite right in our deps. The build doesn't seem to find the artifacts even though they're in the reactor tree when I do that. I don't think it's trying to game the dependency mechanism, as Maven doesn't really differentiate between the artifacts and the module itself - they're pretty tightly coupled afa Maven is concerned. But absolutely agreed about the build order with the pom deps not having worked as intended (per your interleaved output above). I suspect you're right about the pom declaration behind handled differently, at least from a multi-threaded perspective. I'm unclear if this is a bug in Maven's multi-threaded mode or intended behavior, but we certainly don't need to deal with it here.
We should look at this further in a separate PR. While we're at it, we might even simplify things further and remove the profiles, which are typically used for building for different environments, not acting as virtual-modules.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Also @mmiklavc, please remember if the intended effect was to make the rpm build happen after everything else completed then install + pom dependencies didn't work. If you take a close look during the multi-threaded build, you'll see docker-build output interleaved with the output from the other module builds.
e.g. (from a mvn clean install -Pbuild-rpms run)
```
INFO]
[INFO] --- maven-shade-plugin:2.4.3:shade (default) @ elasticsearch-shaded ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-wait-for-it-to-hbase) @ metron-docker ---
[INFO] Executing tasks
[echo] ******** Displaying value of property ********
[echo] ../../../..
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-rpm-sources) @ metron-rpm ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-wait-for-it-to-kafkazk) @ metron-docker ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO]
[INFO] --- maven-resources-plugin:3.0.1:copy-resources (copy-wait-for-it-to-elasticsearch) @ metron-docker ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
```
In this example, you'll see that metron-rpm and metron-docker are happening at the same time at the beginning of the build even though they're later in the build order.
```
[INFO] metron-writer
[INFO] metron-storm-kafka
[INFO] metron-hbase
[INFO] metron-profiler-common
[INFO] metron-profiler-client
[INFO] metron-profiler
[INFO] metron-enrichment
[INFO] metron-indexing
[INFO] metron-solr
[INFO] metron-pcap
[INFO] metron-parsers
[INFO] metron-pcap-backend
[INFO] metron-data-management
[INFO] metron-api
[INFO] metron-management
[INFO] elasticsearch-shaded
[INFO] metron-elasticsearch
[INFO] metron-deployment
[INFO] metron-rpm
[INFO] metron-docker
```
Here's the output from the build that shows the actual build executed order:
```
[INFO] Building metron-analytics 0.4.0
[INFO] Building metron-docker 0.4.0
[INFO] Building metron-interface 0.4.0
[INFO] Building metron-platform 0.4.0
[INFO] Building metron-maas-common 0.4.0
[INFO] Building metron-test-utilities 0.4.0
[INFO] Building metron-deployment 0.4.0
[INFO] Building metron-config 0.4.0
[INFO] Building metron-rpm 0.4.0
[INFO] Building metron-integration-test 0.4.0
[INFO] Building metron-maas-service 0.4.0
[INFO] Building metron-common 0.4.0
[INFO] Building metron-rest-client 0.4.0
[INFO] Building metron-statistics 0.4.0
[INFO] Building metron-writer 0.4.0
[INFO] Building metron-hbase 0.4.0
[INFO] Building metron-storm-kafka 0.4.0
[INFO] Building metron-profiler-common 0.4.0
[INFO] Building metron-pcap 0.4.0
[INFO] Building metron-profiler-client 0.4.0
[INFO] Building metron-pcap-backend 0.4.0
[INFO] Building metron-api 0.4.0
[INFO] Building metron-profiler 0.4.0
[INFO] Building metron-enrichment 0.4.0
[INFO] Building metron-indexing 0.4.0
[INFO] Building metron-parsers 0.4.0
[INFO] Building metron-data-management 0.4.0
[INFO] Building metron-elasticsearch 0.4.0
[INFO] Building metron-solr 0.4.0
[INFO] Building metron-management 0.4.0
[INFO] Building metron-rest 0.4.0
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/incubator-metron/pull/538
also: http://blog.sonatype.com/2008/04/how-to-share-resources-across-projects-in-maven/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #538: METRON-868 Fix documentation on building...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet closed the pull request at:
https://github.com/apache/incubator-metron/pull/538
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Rather than call install, I'd remove the dependency introduced in METRON-827. That will allow package or install work as expected (both should).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-metron issue #538: METRON-868 Fix
documentation on building RPMs
Posted by Otto Fowler <ot...@gmail.com>.
Maybe it is the relative parent path?
On April 20, 2017 at 20:05:31, mmiklavc (git@git.apache.org) wrote:
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Further, the reason that install worked is not because it needs the
artifacts in the cache, per se. It's because of something in the way our
multi-module build is structured that isn't allowing artifacts in the
metron-deployment project hierarchy to see projects in metron-platform. The
reason "package" would normally work on a typical build (without ever
having done an install) is because sibling modules are able to reference
the pom info of each other, and therefore the appropriate target
directories containing the standard build artifacts, e.g. jars.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Further, the reason that install worked is not because it needs the artifacts in the cache, per se. It's because of something in the way our multi-module build is structured that isn't allowing artifacts in the metron-deployment project hierarchy to see projects in metron-platform. The reason "package" would normally work on a typical build (without ever having done an install) is because sibling modules are able to reference the pom info of each other, and therefore the appropriate target directories containing the standard build artifacts, e.g. jars.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Travis!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
+1 - pulled in the dependency changes, deleted my .m2 cache and the build ran as expected. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@dlyle65535 I'm all for that. I'll update the ticket and the PR in a little bit
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@justinleet - works for me, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #538: METRON-868 Fix documentation on building...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-metron/pull/538
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
In your above test, you're depending on default (jar) packages rather than pom as we do in the rpm project. Try depending on pom.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
As I stated in https://issues.apache.org/jira/browse/METRON-856, this is a fix that will work given the current configuration, but will quite possibly break with future changes. Given that I don't have a good answer for Maven's ordering, I'm ok with David's suggestion providing we document the limitation, preferably in the rpm pom as well as the deployment README.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
+1 from me
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@ottobackwards This actually causes issues in just building metron-deployment with mvn package alone (regardless of which target is being run), so it wouldn't help.
The purpose of this PR, at this point, is to just get things back in the state they were in, with anything more elaborate being part of METRON-874
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #538: METRON-868 Fix documentation on building...
Posted by justinleet <gi...@git.apache.org>.
GitHub user justinleet reopened a pull request:
https://github.com/apache/incubator-metron/pull/538
METRON-868 Fix documentation on building RPMs
## Contributor Comments
Update doc to run install instead of package as a prereq to the RPM build.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron (Incubating).
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
bin/generate-md.sh
mvn site:site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/justinleet/incubator-metron rpm_instructions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/538.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #538
----
commit e18e84af2a40e2b828828001a181ecbd3348bcfe
Author: justinjleet <ju...@gmail.com>
Date: 2017-04-20T17:22:35Z
changing from package to install
commit b3e2cb1a7f92a9039fc57dafc15341a2b64e358d
Author: justinjleet <ju...@gmail.com>
Date: 2017-04-24T14:20:15Z
Rolling back dependencies in metron-deployment to make package work
commit 1467ec71d30412329d7c1a2a7381afa54cc11530
Author: justinjleet <ju...@gmail.com>
Date: 2017-04-24T14:28:31Z
Updating README
commit 5ab4689d0ead35ee70436cc12d7b51f4f3a12297
Author: justinjleet <ju...@gmail.com>
Date: 2017-04-24T14:29:20Z
fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
The .m2 cache is an extension of the local target directories and you're correct that upping the version to 0.4.0 caused the initial issue with missing artifacts. Regardless of maven placing artifacts in the cache, what still doesn't make sense to me is why Maven doesn't seem to be ordering the reactor properly with a multi-module project with the parallel build setting enabled. I tried validation on multiple configurations with and without parallel settings. The non-parallel build sets up the order as expected whereas the parallel build doesn't seem to honor the dependency hierarchy fully. It would be great to know why this is so we can run the build with whatever profile or install/package command we choose, with a single command, in parallel.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/538
In fact, having given it additional thought, the dependency mechanism is a way of saying "hey Maven, make sure you pull those artifacts into our .m2 cache" which isn't really the desired result here.
We need to make sure the reactor orderer understands that we cannot build the RPMs until we've built the other tar.gzs.
The reason it started becoming a problem was that we didn't have the 0.4.0 artifacts in our m2 cache so Maven complained. Running an install "fixed" the issue but, we don't need them there to build the rpms, so not really a fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by anandsubbu <gi...@git.apache.org>.
Github user anandsubbu commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Hi @justinleet - can you modify this one as well as a part of this pull request?
https://github.com/apache/incubator-metron/tree/master/metron-deployment/vagrant/quick-dev-platform#deploy-metron
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@mmiklavc I made the proposed change and pushed it so it can at least be seen. I'll can easily adjust if you either don't want to do this or have suggestions or anything else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
For posterity: https://issues.apache.org/jira/browse/METRON-868
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@mmiklavc @dlyle65535 Do we want to compromise on this for now by removing the dependency, and kick out another ticket to investigate a more robust way to manage this?
Mike, you wanted some documentation in the deployment README, but I'm not sure what you specifically wanted in there
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/incubator-metron/pull/538
Would this be fixed if we just ran build-rpms every time? In other words got rid of the profile flag requirement?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/538
@mmiklavc Added a quick note, and listed those dependencies off. Let me know if there's anything else you want in there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/incubator-metron/pull/538
fwiw, I ran a local test that seems to work fine. There must a be a subtle difference somewhere in our build, but I haven't found it yet.
```
{18:45}~/devprojects/temp/mvn-dep-order/metron \u27ad find .
.
./m1
./m1/pom.xml
./m2
./m2/pom.xml
./m3
./m3/pom.xml
./m4
./m4/m41
./m4/m41/pom.xml
./m4/m42
./m4/m42/pom.xml
./m4/pom.xml
./pom.xml
{18:45}~/devprojects/temp/mvn-dep-order/metron \u27ad find . -name pom.xml -exec sh -c "echo '\n{}'" \; -exec cat {} \;
./m1/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m1</artifactId>
<name>m1</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>mproj</artifactId>
<version>1.0</version>
</parent>
<dependencies>
<dependency>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m2</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
</project>
./m2/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m2</artifactId>
<name>m2</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>mproj</artifactId>
<version>1.0</version>
</parent>
</project>
./m3/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m3</artifactId>
<name>m3</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>mproj</artifactId>
<version>1.0</version>
</parent>
</project>
./m4/m41/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m41</artifactId>
<name>m41</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m4</artifactId>
<version>1.0</version>
</parent>
<dependencies>
<dependency>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m1</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
</project>
./m4/m42/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m42</artifactId>
<name>m42</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m4</artifactId>
<version>1.0</version>
</parent>
<dependencies>
<dependency>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m41</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
</project>
./m4/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>m4</artifactId>
<name>m4</name>
<url>http://maven.apache.org</url>
<packaging>pom</packaging>
<parent>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>mproj</artifactId>
<version>1.0</version>
</parent>
<modules>
<module>m41</module>
<module>m42</module>
</modules>
</project>
./pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.michaelmiklavcic</groupId>
<artifactId>mproj</artifactId>
<packaging>pom</packaging>
<version>1.0</version>
<name>mproj</name>
<url>http://maven.apache.org</url>
<modules>
<module>m1</module>
<module>m2</module>
<module>m3</module>
<module>m4</module>
</modules>
</project>
{18:47}~/devprojects/temp/mvn-dep-order/metron \u27ad mvn clean package -T 2C
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] mproj
[INFO] m2
[INFO] m1
[INFO] m3
[INFO] m4
[INFO] m41
[INFO] m42
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 16
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mproj 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ mproj ---
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m2 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m3 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m4 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m4 ---
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m3 ---
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m2 ---
[INFO] Deleting /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m3/target
[INFO] Deleting /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m2/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m3 ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m2 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m3/src/main/resources
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m2/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m3 ---
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m3 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m3/src/test/resources
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m2/src/test/resources
[INFO]
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m2 ---
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m3 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m3 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m2 ---
[INFO] No tests to run.
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m3 ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m2 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m2/target/m2-1.0.jar
[INFO] Building jar: /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m3/target/m3-1.0.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m1 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m1 ---
[INFO] Deleting /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m1/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m1 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m1/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m1 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m1 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m1/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m1 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m1 ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m1 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m1/target/m1-1.0.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m41 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m41 ---
[INFO] Deleting /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m41/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m41 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m41/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m41 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m41 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m41/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m41 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m41 ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m41 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m41/target/m41-1.0.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m42 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m42 ---
[INFO] Deleting /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m42/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m42 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m42/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m42 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m42 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m42/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m42 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m42 ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m42 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /Users/mmiklavcic/devprojects/temp/mvn-dep-order/metron/m4/m42/target/m42-1.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] mproj .............................................. SUCCESS [ 0.119 s]
[INFO] m2 ................................................. SUCCESS [ 0.557 s]
[INFO] m1 ................................................. SUCCESS [ 0.016 s]
[INFO] m3 ................................................. SUCCESS [ 0.556 s]
[INFO] m4 ................................................. SUCCESS [ 0.030 s]
[INFO] m41 ................................................ SUCCESS [ 0.015 s]
[INFO] m42 ................................................ SUCCESS [ 0.015 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.819 s (Wall Clock)
[INFO] Finished at: 2017-04-20T18:48:05-06:00
[INFO] Final Memory: 10M/309M
[INFO] ------------------------------------------------------------------------
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---