You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/08/29 13:38:00 UTC
[jira] [Commented] (METRON-1136) Metron Extensions System and
Parser Extensions Feature Branch
[ https://issues.apache.org/jira/browse/METRON-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145299#comment-16145299 ]
ASF GitHub Bot commented on METRON-1136:
----------------------------------------
GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/720
METRON-1136 Initial Merge of Metron Extensions and Parser Extensions
This pull request will land the Metron Extensions and Parser Extensions functionality in the new feature branch.
It represents the total combined work on this functionality:
- [METRON-777](https://github.com/apache/metron/pull/530)
- [METRON-942](https://github.com/apache/metron/pull/580)
- [METRON-947](https://github.com/apache/metron/pull/626)
This feature branch work is now being tracked is [CONFLUENCE](https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions#) and [JIRA](https://issues.apache.org/jira/browse/METRON-1136)
## Status
[x] Creation of Metron Bundles ( adaptation and refactoring of Nifi NAR )
[x] Adaption of nar maven plugin to bundles maven plugin
[x] Archetype for creating parser extensions
[x] metron-parsers reduced to library for interfaces, base types, and base parsers
[x] Metron parser types moved to parser extensions
[x] Metron parser extensions packaged by rpm, installed by ambari including hdfs deployment
[x] Metron parser extensions loaded by bundle from parser bolt, from hdfs
[x] Refactored grok building to support variable metron hdfs apps root, parsers with directories of groks
[x] Rest api for parser extensions ( install, delete, list, find )
[x] Zookeeper registration for extensions and their metadata, including default configurations
[x] Stellar management for parser extension configurations
[x] Guide to adding parsers to the metron system
> Currently there is no work in Metron Config UI for installing or managing parser extensions
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
> see contributing PR's
- [x] Have you included steps or a guide to how the change may be verified and tested manually?
> see contributing PR's
- [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to verify your changes?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron ext-feature
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/720.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #720
----
commit d68019dd48b17f6961fb6be1a87afada9ece57ab
Author: Otto Fowler <ot...@gmail.com>
Date: 2017-08-29T13:06:12Z
Squashed commit of the following:
commit 2b54f853fadd1ffb4b7c70cfb70d1c28d8220eaf
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 11:46:28 2017 -0400
conform to logging norms
commit 2c1eab5e266b5cab94708a05ec292fb4d28cc9de
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 11:11:50 2017 -0400
more test tweaks
commit 76affe12d52e7dea30a417dc7b8a1ac8292943ca
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 10:04:49 2017 -0400
do not need the resource copy
commit 5c38f00ba1d7cbdf72908105d469fe6ad4797f0b
Merge: dcf9cee4c c8e84fa3b
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 09:40:51 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit dcf9cee4c2ef9cc5945b92f085baa3465f029418
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 09:18:25 2017 -0400
copy resources before class
commit e18fe49305dfe06c637b6d491fd098a43a7aeb0f
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 08:44:12 2017 -0400
javadoc and some saftey options for ResourceCopier
commit 799b0b73a57cc127ea9b25cb3f92b02f97d07f73
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 28 00:02:54 2017 -0400
integrate with 777 + 777-grok
commit 49788639e91e1649eacd7b8c4f36475a44f0ceac
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 26 00:20:59 2017 -0400
fix up imports
commit 239df5f5e8d21253edaf6127e839b9b293d2a19f
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 9 10:26:28 2017 -0400
missed a fix, rename folder
commit bacd11508eaac0635d62afe3825dc40eb35ebf93
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 9 10:00:16 2017 -0400
adding the metron management function for parser extension configs from METRON-947, no reason to maintain two prs
commit 94d223ed4badbc9f109dfbe17027cefacc1ee1ae
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 7 16:43:26 2017 -0400
merge fixes
commit 97edf2f869ff234613db8b74830b57e84c0231a9
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 30 16:33:24 2017 -0400
change documenation to reflect uninstall
commit b0fc2e4cc8b0637f9b619ac5c48e4b716245d16e
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 30 16:31:19 2017 -0400
After merge, fix tests. Problems re-activating kafka after it was taken out in master.
Instead - do not delete kafka topic or storm topology, since this service doesn't create
them.
commit a02b6fc2b80fe0a69ddef61db9ac73b6aab3c1bf
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 6 10:24:17 2017 -0400
better readme for usage
commit beee5d94e10f8a46fd3c0a632b2ae29c7e05d6e8
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 10 09:25:24 2017 -0400
Add endpoints to readme
commit 2d0edf6d6ef557370e517ec09e18df966bc05cbd
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 9 09:50:23 2017 -0400
Use the ambari managed spring configuration to properly get the grok pattern dir for hdfs
Be sure to rollback on install exception, and delete grok dir for installed parsers
commit fda824e38d36b12cc394cbdad40c3323f37ad9aa
Author: Otto Fowler <ot...@gmail.com>
Date: Fri May 5 10:43:25 2017 -0400
Do not install an extension if it is already installed
commit e536adf08d499e4c0408c97f0042fc82206f3907
Author: Otto Fowler <ot...@gmail.com>
Date: Fri May 5 10:17:09 2017 -0400
fix extensions alt lib path in config
commit cf3ad6991a6eea3987e8eeae62890d28abeed882
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 4 14:58:29 2017 -0400
cleanup a couple of imports
commit 40048dbe953fa11dac20415d7ee3d6f57dd3e005
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 4 14:53:27 2017 -0400
implementation of delete
commit 1912795bfd5a6395d55cc7222b73bf37370aa432
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 3 07:59:49 2017 -0400
async delete call
commit 2e2c7a2d462eb4aab0b4cc2581fd52ab47cb0b54
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 3 07:15:36 2017 -0400
fix to return th correct httpstatus of created for install, use async dispatch in mock tests to get right result
commit a3a5d47032e02a1ac6a474bfeb7326311c21ce62
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 23:54:48 2017 -0400
added delete test, need to implement
commit 69378114704060d20dd5312d7b78c1b071b89fad
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 23:44:10 2017 -0400
fix json path issue for test of getAll
commit c2839078e730226f9fdb491b6181fff7729a63f0
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 18:29:55 2017 -0400
test for getall()
the call works, but I cannot get the jsonpath verification right
a single statement works in a verification tool, but not here
commit f9429494417fad9181400c08d4804a5f8780ee83
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 13:20:13 2017 -0400
test for getAll
change ConfigurationType to allow changing zookeeper root
generate a new extension name, must not have 0.0.0 version at end...
will be the archive.tar.gz file name, with . -> _ and dropping the archive.tar.gz
this foo-0.4.0-archive.tar.gz -> foo-0_5_0 as the extension name
commit ade08a9505ce899bef502c05848a5ce777de424c
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 10:37:32 2017 -0400
update goal comment
commit b193fe8604b4a75a84b2d122d9e43feed4db204f
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 2 10:11:33 2017 -0400
support for putting the default es templates in the extension config
fixed bug where we were calling wrong function in ConfigurationUtils
commit e62b0216fce099f8641074ef0f7f33a2fea93d3c
Author: Otto Fowler <ot...@gmail.com>
Date: Mon May 1 17:51:11 2017 -0400
grok fix, after some manual testing. need to provide an override for the patterns dir in hdfs while testing
commit 98f0c40273b790514b5cdad88cb0f998a33a8dd1
Author: Otto Fowler <ot...@gmail.com>
Date: Mon May 1 16:39:11 2017 -0400
Write out files to grok.
At this time, the location of the hdfs grok directory is hard coded to the default
If it is changed in ambari, this will not work.
commit 098a288fd7754c9164c97f7aa8ee97ed4e3110ec
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jul 4 00:46:15 2017 -0400
track extension installs, along with metadata and defaults
Needs more tests, but this should allow implmementation of delete()
commit 180215a7393ddf33d095257baf27e7284dbbc32d
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Apr 30 11:22:11 2017 -0400
Create an actual context object and build it out with more information, including bundle information
The goal is to have enough information to fill out a configuration object that is in zk, that will allow
for some kind of removal/uninstall of a parser
commit 7ca11b63bed268cf333db9407fc431eb88a441a2
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Apr 29 13:09:38 2017 -0400
do not assume empty content, unless you are going to clear out everything
commit 6f5257311cec9c9169d99065b4f4d56ba4668ef5
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Apr 29 00:11:26 2017 -0400
do not need to add this, with the right intellij project ref to metron-common this works fine
commit 37def88edf8599b930c0f7f19855a61af00498bb
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Apr 28 23:41:27 2017 -0400
start of controller test
commit cd2f589da0ab1f97978043c8ecbd1372a35197d3
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Apr 28 10:50:15 2017 -0400
correct path to write bundle to
commit 6f9d170e7ff46539ffbd237b3d1d48a4bd97ef6f
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Apr 28 10:39:31 2017 -0400
first pass implementation, no tests
commit c0c8312582d0423afb8cd85a1e326e0afce82386
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Apr 27 23:49:24 2017 -0400
use context to track paths that are created, delete tmp files on exit
commit d4183732019084383dccb5249ec65db3a0dada18
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Apr 27 23:08:47 2017 -0400
wire up the services we will use
commit 6e0ec09301e8c1c6748a764f5e2979758536634b
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Apr 27 23:00:38 2017 -0400
start of implementation of service, validation of extensions
commit 5dc7e2fa044b7485600533f70f9f80f7aa710f72
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Apr 27 17:02:49 2017 -0400
create controller class shell
commit 1f71847dfec1f261d03f75238c091b0af393cc75
Merge: ae3a07ad3 219012b49
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Aug 27 12:48:15 2017 -0400
Merge branch '777-grok' into METRON-777
commit ae3a07ad3d89d5fd0a54692fb722f3b8ac26f857
Merge: a53aadc20 a2bae0bce
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 26 00:12:07 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 219012b4952d5695c867df9eec701f7ab6df77f8
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 15:21:18 2017 -0400
remove debug and fixup docs and tests for ResourceLoader
handle paths missing / better
commit 97cf0d738e381a1587e9a5b0ff71fb27cb6ea772
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 12:08:04 2017 -0400
cleanup SesnsorParserConfigServiceImpl
commit 4fa2423be52784591977ccaadc09b1196c0126c6
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 12:01:50 2017 -0400
cleanup BundleSystemConfig
commit 152100ced5a011008a6396dafdf513f1c9321463
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 11:41:00 2017 -0400
correct the tests
commit 5f7e52b7495cdc271c8d5905effa2d649196da23
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 09:30:18 2017 -0400
fix grok tests, add new test for save statement
commit b2eec55443fd9eae693758abadf97d895ac861b2
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 25 07:43:51 2017 -0400
working integegration with parser extensions as bundles and rest/ui
cleanup and refactor todo
commit 9c7f98f3e05f81ca2b8239085918bc74ea9b909d
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 22 16:20:05 2017 -0400
refactor hdfs fixups so they are not required
commit 5c98a1573655826cc7740ff211143eeba39e1f31
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 22 02:36:33 2017 -0400
first pass integration of rest with bundles and extensions
commit f0923a6ca4ca0cbfa5fe541311ebe20126a28bad
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 16 16:35:20 2017 -0400
refactor hdfs and pattern loading
Create GrokBuilder and ResourceLoader for re-usable and consistant file loading and path fixups
tests create the correct ./target environment for rules to be loaded from
ParserBolt injects the global configuration into the parser configuration
Global configuration has new key for metron hdfs root
new key is setup by ambari
Now, give a parser configuration with a simple grokPath "/patterns/yaf"
- it will be fixed to work in hdfs or local for tests
- it will support yaf as a directory not just a file
commit a53aadc2005d60dbea459821050ea56f9cc3a10a
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 23 02:13:43 2017 -0400
fix test, missing comma from merge
commit 4dc1650021034882459b0fbe0d9549c9227f1169
Merge: b25d5f1f1 d64afbc3f
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 22 16:25:47 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit b25d5f1f1c976cf5716dc8af4786a75e1cba4e7c
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Aug 17 00:13:38 2017 -0400
Changes for synchronization and other things per latest review
commit c0fd02d0c8296f79e8a8a479d1a6099e31900583
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 16 16:40:37 2017 -0400
fix indentation error
commit 8455bbed2f3c5fb0935e17b2c574c4fd4fa65bab
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 19:04:25 2017 -0400
update doc per review
commit 13942173cf4cd174d790a3ca835e8756c1a686e5
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 19:01:59 2017 -0400
sigh, fix tests
commit c5f8c34e4de8e6d456b97edd6f8a0d33b4819d69
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 18:36:10 2017 -0400
address review comments around exceptions, singltons
commit 9526fed864aff3209dce7b319eda59961681f03d
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 15:09:50 2017 -0400
javadoc for extension mapping
commit d652fa093a89916bd84347984e3ec0d6a00c42e1
Merge: 88c005fd8 adb378f35
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 14:54:00 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 88c005fd8a63b13cc28ec27c66b5925566618d7a
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 14:50:00 2017 -0400
per review - add comments about extensiable extensions namespaces and difference with nar
commit fd887c1bfb107fa4633a0f50f48083e45b36bd63
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 14 14:40:37 2017 -0400
add dummy implementation of FileObject for system bundle
commit 72abee73f00085ddd2019dd84f689b81d81da4d3
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Aug 13 13:11:26 2017 -0400
document usage of BundleSystem
commit b672ef3d31c2a1255566e120d2654b615b24009f
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Aug 13 12:50:10 2017 -0400
javadoc additions
commit ce8d98dcb69fc7e249fbfce5edd4afadaf2a14af
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Aug 13 12:14:17 2017 -0400
simplify, the extensionmanager has the classes, no need to do the init dance with the BundleThreadContextClassLoader
commit d1b9871dd6f85afcfa9536b3bfd0ae1f650336f4
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 22:47:23 2017 -0400
test fix: need to clear after class
commit 5cb6bdf4cf69396664489f5d620e057c386c6a3b
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 22:05:28 2017 -0400
javadoc and formatting
commit 5bcc8c6f066f5339578c329b24d92d746a803562
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 22:01:45 2017 -0400
unit test and some fixups
commit 97295d72470dbf5fccc68cb0073f123fe3a16391
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 21:44:32 2017 -0400
returning a value would work better
commit 70f2106f9b58d04bee3d7cee981535194d2285c5
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 18:53:57 2017 -0400
simplified the call interface for creating classes
commit 103f869f4901de0ae1c007702b526a213793b9c5
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 17:38:33 2017 -0400
we do not need to do the mapping, we are not using them
commit 2f93bfc3356230a3362989c1d4c0d825236e31a3
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 17:11:16 2017 -0400
more javadoc detail
commit 3edafdb5ff2b09b58bddd900ff1951347f0679ff
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 16:58:09 2017 -0400
enforce immutability in the BundleClassLoader as well
commit 4cf9cb4c7aad3458ed40880dd5695b1d2acb0d44
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 12 13:15:28 2017 -0400
based on review, refactor the ExtensionManager for Immutability and Thread Safety
commit 97ef4f0b509ad9b110165665382223373392e25a
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 11 22:21:49 2017 -0400
per review, add clarification
commit 22ba18b569b05bf040d1fca1f331602210e2e5ec
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 11 22:19:13 2017 -0400
per review fix spelling
commit 571168d897cdd10bc616cef92b613763f76a1842
Merge: ea6240a0e 86b0f137e
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 11 14:15:26 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit ea6240a0e17c5ac7261b6aa41ffc75f512573cca
Merge: 8b5d66582 b3148a182
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Aug 11 08:51:50 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 8b5d66582ccfe391ff139ecd8f4bbc58d128542f
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Aug 10 17:30:42 2017 -0400
remove reference to nifi
commit 27c7de4e563820d2722ac7d7a7e79105882da400
Merge: 0b583e0ff 7d554444e
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Aug 10 10:12:51 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 0b583e0ffb31c04b75d9b7375c4058999a3930ce
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 9 15:36:47 2017 -0400
per review, added some validation, also renamed variables and accessors
commit b402517e858554761a27f7624c65a2538e5c0c89
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 8 16:57:49 2017 -0400
the headers, the headers
commit 6690398f242ab9279786eaa83f5a1e67cb6e4103
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 8 16:52:02 2017 -0400
add header
commit c6e9a6abc147a2c07a7e2a884a054c4f7fd29679
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 8 16:50:20 2017 -0400
remove unused import
commit 4bad9483cc00b6632c13c482deeba7b2cc349bf4
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 8 16:48:16 2017 -0400
fix working dir reference
commit f42eef2146b539e47914f625df4b3808bcc9491f
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 8 16:43:06 2017 -0400
Changes per reviews
commit 4f8409abc06191c3453e30756afa50941870a1d8
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 7 16:09:12 2017 -0400
fix post merge bits
commit 54602cf3a706beb75bb5da17e71c08ce20b7057d
Merge: 0ffac1616 813adf2da
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Aug 7 15:35:37 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 0ffac16167576bb99128892868f99f852ea0810e
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 5 08:47:26 2017 -0400
remove notice about versions matching
commit b1a96448c44e20768638fb925191eb66f2ca7d7f
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 5 08:41:59 2017 -0400
change plugin inclusion in archetype pom, this resolves requirment for version to match metron version
commit bf5ec2f0e4df5688c12813da17651e6d0d2c3080
Merge: 04727ca8c df25d81c4
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Aug 5 08:00:32 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 04727ca8c448e2845e48a8a95812e6bf27b2c13d
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Aug 3 13:30:29 2017 -0400
update document to remove references to the working dir
commit 3212c1bc9a0c31c607c7652a27519ddc97879999
Merge: c78592921 b2375a1f1
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Aug 3 08:11:57 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit c78592921d6be4c2f694b93245cea77f72e3581c
Merge: 4b9a1245a 27ab28923
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Aug 2 17:24:52 2017 -0400
Merge branch 'master' of https://github.com/apache/metron into METRON-777
commit 4b9a1245a24f7d4b6dbac95fa40334c0fb0bea69
Merge: 7a97e41de ec959d20e
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 1 12:00:14 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 7a97e41de770e4c95e3715feae080dc96f20a9c7
Merge: afe4b9e29 7b9f20c93
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Aug 1 09:24:49 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit afe4b9e29a1fb711cf170d8a844275ca22929965
Merge: 5bb6a33b3 c99df8ff1
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Jul 31 21:12:28 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 5bb6a33b36962a8cec8a7a3f3f1941f87dc068bc
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 23:15:16 2017 -0400
refactor/rename VFSUtil to FileSystemManagerFactory, which is what it is
commit 2e48f087a9c320ab2094f606005a87030da1e153
Merge: f1cdcd793 ba46fa734
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 13:00:51 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit f1cdcd79374e603dd3a90e4e067c119d1bb8bf7a
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 12:17:34 2017 -0400
for BundleThreadContextClassloaderTest, fix last path. in travis if the folder is empty it may not be there?
commit 98545d1526e61564bbabf2268f4743f7a2b7c646
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 12:03:39 2017 -0400
more travis fix attempts
commit 868cf6cb103775ef30cf45a0f4f042b50ebdd9f3
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 11:45:01 2017 -0400
attempt to fix travis only failure
commit 66a0c8205c778469d99dd1108f7735ca5668c3f6
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jul 29 10:46:37 2017 -0400
Based on feedback from reviews and discussion with mattf, this commit represent a major refactoring for the bundle functionality.
While trying to resolve the ugliness bought about by working around VFS's read only support for HDFS, I have come up with a different approach.
While this approach makes bundles-lib significantly different than nar-utils, it makes it much more simple, and may arguably be an approach worth
concidering.
The original bundles-lib maintained the nar system's method of unpacking the bundle into a 'working' directory, and then using having the classloader
load from that directory structure. Trying to avoid too far a departure ( at the time ) from nifi, bundle-lib maintained the 'create folders and files as nec'
scheme found in nifi. Since nifi is always working with local disk and File objects, this is ok. But with VFS and HDFS this did not work. The work around
was to create a poop static 'class cluster' type step with multiple implemenations of FileUtils ( VFS and HDFS ).
The new approach, which is a more willing departure from the nifi approach, is to avoid having to ever do writes ( file or folder creation ) from bundles-lib at all.
This allows for a simplier, single implementation of the file utils and the removal of the anti-pattern. It also removes the need for a 'working' area.
The new approach is to better utilize the capabilities of VFS such that we never have to unpack the bundle at all. VFS can load JAR files as Filesystems. I have created
a new Classloader ( based on the VFS classloader implemenation from the VFS project ) that is bundle aware. It can load the bundle and it's dependencies as resources
and make them available. This is what removes the requirement to unpack the bundles into a working directory.
As such, the bundle-lib has been refactored to move from the 'working directory' to only needing to work with the bundle in the lib. I have also removed the doc directory support
since we are not using it.
We no longer need multiple file utils, the hdfs working directory, and other things.
Thanks to mattf for the time and discussion.
commit 9fffd6187b2a36448a3a4caf7472d1b5da10b426
Merge: bbc022857 9355a0465
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 23:32:23 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit bbc022857ffbc553cdb9519910b3a466765c0c62
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 19:00:19 2017 -0400
use MessageParser for consistantcy
commit 05797505d5ef4f0e562a2e1621ff455b709bd515
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 18:40:57 2017 -0400
remove more nar-isms. base tests on archetype build parser projects instead of old nifi processors
commit 7c84fc49f0010b517d72f87843efc7de6a93d527
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 14:16:46 2017 -0400
refactore type property to packageType per review
commit 4650125713bb9bc0e4d3184b9434eb216b750524
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 08:25:40 2017 -0400
remove nar references per review
commit f84efb72047e19d0c9bc8e66e96c62bba627d878
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 08:18:42 2017 -0400
remove nar references
commit c4f346867c6aa0e46a6f6a596c81eb3457cf0804
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 08:12:07 2017 -0400
refactor package per review
commit fc38b3df915b47495d9f7791736255aa0fbb6df6
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 08:06:52 2017 -0400
more incubator instances
commit 132763c75435dfc804b435d16580bb1d8a48a273
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 27 08:04:10 2017 -0400
remove incubator references
commit 8f274d835e0fb787bcb74a53c98a3ba53bc7e2ba
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 26 11:24:30 2017 -0400
reformat based on review
commit a6dffb246599b79abbfc03c63755f01f221b7c54
Merge: 0d9a43a93 badc6cf97
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 26 11:23:37 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 0d9a43a9393a20309845c668a206005f45ea4617
Merge: fcb8ef72e 309d8097a
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Jul 24 10:05:28 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit fcb8ef72e243c34e7e2f2191b968431f3b01fe0a
Merge: 7ee99ef3a cf7043c59
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jul 20 10:44:54 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 7ee99ef3aa42b016f87fe4b6b69a507712aa0102
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jul 18 09:52:54 2017 -0400
reverting last change, it worked in other travis repo
commit 1b095e5ffce8ca9add693e223470ce4f42660874
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jul 18 09:49:06 2017 -0400
try to resolve travis not building bundle
commit 06e4d7626bb9b40f704816109735b88973f85fcf
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jul 14 08:58:32 2017 -0400
need to install bundle now that we moved the other install phase
commit 3ef2059c72f00108e52a4f106cda3280e170b355
Merge: 6d4be0f4a 954b5ea13
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jul 14 08:54:09 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 6d4be0f4a25bdb7e87b89a2a3622e913425d97c3
Merge: b673fc7b9 dcf768297
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 12 10:43:40 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit b673fc7b933cdfefc64db81f7853275161d4bc7d
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jul 7 15:59:16 2017 -0400
fix version in new poms
commit 8b70e08071738eeb7406f8df39581753068daca9
Merge: d12d3fcb5 a8f35363f
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jul 7 15:35:30 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit d12d3fcb5a72437f87e53f0d2a34e84d4ffba354
Merge: efecf3a9a f7a94f2e6
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jul 7 12:20:26 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit efecf3a9a3240c0404be547fc153bdf7825add23
Merge: 31d52d436 df270feea
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 5 12:56:01 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 31d52d43632faf2720bc9dfecd1f0d5cb230db71
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 5 11:02:58 2017 -0400
add documentation of multiple configurations of multiple parsers in one extension
commit 3e9c3972c053e96cf0feea380028c69dae062870
Merge: 8a53de80a cc7bbc948
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jul 5 09:10:24 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 8a53de80a4bf146759eabff3e58aab6e9a28d684
Merge: e67a55edf a5b13777a
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jul 4 00:43:25 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit e67a55edfd4c0734dcde3e65c9ec660ff2d38e59
Merge: 679d56fd7 df94ed405
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 30 12:36:20 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
Fix integration tests to work with extension paths
commit 679d56fd7df8819f8ebf631bf045f34b23f27290
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 29 20:58:54 2017 -0400
per review, changed version in plugin to 2.4
also updated metron version in the archetype test files
commit 2a7e8d3acba321174330b160fa98bef072632251
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 29 07:57:58 2017 -0400
fix extra project node per review
add ASF headers to other poms
commit 6d984823fec9696854b94d0c94c42aabb0c32739
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 29 00:08:59 2017 -0400
removed deployed pattern directory and common pattern files for parsers that do not use patterns
commit 2db581469e40a5dbb000ba9c5dc430586f68ee5e
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 28 23:23:27 2017 -0400
fix spacing, fix name
commit b8e65fbe8cc7847731bcb22b3e80be68bc9dee1c
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 28 23:11:13 2017 -0400
deployment readme
commit 86190f830e25846d1b4f89f7d2a10377b5d59e74
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 28 22:19:31 2017 -0400
readme changes per review
commit a8f34eff42e575f110e4260e050ab95393bfbff0
Merge: b3daa9ed2 095be23dc
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 28 22:06:44 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit b3daa9ed211da74007ec22b63184002e60672b19
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 27 18:13:28 2017 -0400
start of deployment documenation
commit 50290332ffed4027766578e7ec6195ffe4ee9a08
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 27 13:07:08 2017 -0400
remove reporting tag, we will inherit here - per review
commit 16f79994cb7147e8a3e60d97def49199b99cf397
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 27 11:29:14 2017 -0400
more documentation work around packaging and terms
commit d4f28ed8f5bc23b1cb3dac29834265831a9843ff
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 27 08:05:25 2017 -0400
fix readme links
commit 72eed83899ebeebd2d63aaf9e324579a310f9865
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Jun 26 18:15:46 2017 -0400
start documentation of packaging and terms
commit 33682eba5fa0e7883f57184c9c97752a05fd6661
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 22 09:40:26 2017 -0400
more fge requirements
commit 269f739dcfb47038497b2249ef315037b6d0cc5f
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 22 09:16:28 2017 -0400
put the fge json stuff back in the parsers that use them in test, as opposed to adding fge as a shipped dependency
commit bfafd30d356766f9408b6e34ac81d7bf7053f81b
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 21 16:22:20 2017 -0400
create a testing pom project to hold testing only dependencies - per review
remove some nar references
commit ee6fbd4b8307f5ae7032db88dd7fd1935414b000
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 21 09:35:55 2017 -0400
Per review, remove .gitignore left over from nifi plugin as separate project
commit ced63804f40438eb8e3bef44acddc1e2d57ca90d
Merge: 96957271c b76bcd5f2
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 21 09:30:59 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 96957271c8ed849b786b18373054347bbcb09bd6
Merge: 22c0cba89 de2c871ab
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Jun 19 11:09:46 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 22c0cba895625a7020b49ddf6875b8b05c8ad700
Merge: 02ef46ec7 f523c1795
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 16 10:56:02 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 02ef46ec72161b952b56b72faa4d8067418486c3
Author: Otto Fowler <ot...@gmail.com>
Date: Tue Jun 13 14:21:34 2017 -0400
per review, move metron-extensions under metron-platform
associated fixes for path related things
commit a6408061e2122a0c61ca6d82b2dca3a2de3745d3
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Jun 11 17:00:03 2017 -0400
add readme for bundles-lib
this readme specifically references the relationship to apache nifi nars
this readme is based on the nifi developer guide for nars
commit df78af106ff74f97675950145736f5c73282b739
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Jun 11 10:52:41 2017 -0400
refactor based on feedback to use the metron integration test classes
this should be OK because that jar does not pull in other metron jars and is test only
commit 9e012a5e37b94737ff4faa5a26aa27b4a0c52e0f
Merge: 9099f3682 30d0e2a6f
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 9 16:07:08 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 9099f3682e1397f9901e568f1efab85d0bcb3356
Merge: e9b8b9bbf 7d6121bfd
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Jun 9 07:01:34 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit e9b8b9bbf3756920eaaddee6a47891993a4e6aa6
Merge: 350ac94e4 61cbab46e
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 8 08:54:00 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 350ac94e43652d7b3916bd07b61dba88f12090af
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 7 23:36:39 2017 -0400
fix markdown so site-book does not crash
commit 145465fce07a734637aafca0416227e94c1c841e
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 7 23:13:50 2017 -0400
Added a guide for adding parsers to the system
commit 572212ff4dc6c1c63c1b1fd37030b2fa6c4d392e
Merge: 9bee17ce9 5b72da7be
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 7 12:45:44 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 9bee17ce99e89477a67eacf0d181d38301b3d884
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 7 09:12:55 2017 -0400
per review: use inherited reporting in pom
commit d8b09c6c31a44e7342dd2b7b3ecf294a670d2c9c
Merge: b25358f60 605a3cc6d
Author: Otto Fowler <ot...@gmail.com>
Date: Wed Jun 7 09:07:42 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit b25358f6044d68b3a49a70aae1cd8abd63638f3b
Merge: bf726813d d0e1ba508
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Jun 5 21:43:47 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit bf726813dbb6a0f899c77f103af967ce0e0a47b6
Merge: 48ce8e26b e31ed2f7d
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Jun 3 08:58:45 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 48ce8e26bcecae166856834fc323a431727a63b2
Merge: 23e2a0bce 85872bd68
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Jun 1 11:34:03 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 23e2a0bce63ffcd0b20666c25cde5dc3b03e82b9
Merge: f12922d6d 61105c715
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 31 10:20:37 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit f12922d6de8485c21d28ca12b85f72ed37357feb
Merge: 0dd8cf8c9 a2634f31d
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 25 22:01:56 2017 -0400
Merge branch 'jz-metron-777' into METRON-777
commit a2634f31d2b16f9ece00dbae49a8b0265f091cfd
Merge: 0dd8cf8c9 2e66ea397
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 25 22:00:07 2017 -0400
Merge branch 'METRON-777' of https://github.com/JonZeolla/metron into jz-metron-777
commit 0dd8cf8c9ab89b43d543de8b5237010a948540a9
Merge: 382a29e18 08f95dc8f
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 25 21:58:05 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 2e66ea397ffc82caf6a76d47ba3651754361dd51
Author: Jon Zeolla <ze...@gmail.com>
Date: Thu May 25 18:07:23 2017 -0400
Trivial documentation changes
commit 90a51deb14078b57966f3ad8e2f7e000cbb74ca3
Merge: 7ef929ae4 382a29e18
Author: Jon Zeolla <ze...@gmail.com>
Date: Thu May 25 17:52:55 2017 -0400
Merge branch 'METRON-777' of https://github.com/ottobackwards/metron into METRON-777
commit 382a29e18f9ab161f0a21d92c1ba3ce6f2601eeb
Merge: eb2062be4 e2197316d
Author: Otto Fowler <ot...@gmail.com>
Date: Mon May 22 08:01:38 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit eb2062be4c14d0549a6ce2504dbbe29ea5510141
Merge: b73f35fc2 356881ad2
Author: Otto Fowler <ot...@gmail.com>
Date: Mon May 22 07:58:50 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit b73f35fc25982556ad469de409e15c72b0305a8a
Merge: 97c1be132 ce381f3e0
Author: Otto Fowler <ot...@gmail.com>
Date: Fri May 19 11:33:30 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 97c1be132c114a0f3f729e5f9887adb5aa49f582
Merge: 225fb4e38 be0307659
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 16 22:22:33 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 225fb4e38d6e5ec5eff877be2a84c8fd29af0c3e
Merge: 5565ad661 345eba8d5
Author: Otto Fowler <ot...@gmail.com>
Date: Fri May 12 14:12:03 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 5565ad661f4143234b67230f7cfd420ea1bce91a
Merge: ab0aad849 e375936a1
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 10 15:47:58 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit ab0aad849319117dd3d964247861cf09bc3e7822
Merge: 6c0201b85 8532e2848
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 10 08:57:21 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 6c0201b853ffd079f02589011c57cb6a960475e8
Merge: e3b71aed7 e08b7d47f
Author: Otto Fowler <ot...@gmail.com>
Date: Tue May 9 09:55:31 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit e3b71aed74271cf255be5fad072a99506f553d2e
Author: Otto Fowler <ot...@gmail.com>
Date: Sat May 6 12:08:58 2017 -0400
use simple json and not the JSONUtil Instance, there are issues in storm loading
commit 1e0e30562b4b526d9512bb1dc06077674d6b3277
Author: Otto Fowler <ot...@gmail.com>
Date: Fri May 5 14:18:32 2017 -0400
update archetype based on changes to loading and configuration
commit a3e63555e4e76486be58ddd6f818006ff42cdd1f
Merge: 5f74fc3a1 716bda326
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 4 23:42:54 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 5f74fc3a15fe5e56f9e3483fa4c25961ffb284dd
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 4 16:25:18 2017 -0400
be sure to clear out before and after, we cannot assume order of maven build
commit 995a4d746fa21400a55e9571957878ef2d5b48d8
Merge: de4884563 38d26d432
Author: Otto Fowler <ot...@gmail.com>
Date: Thu May 4 09:46:58 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit de48845634ae1cecb258175326592a28f5b3f8fe
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 3 22:40:55 2017 -0400
second attempt to fix plugin errors with jacoco
commit 9cafe970b40d551a846fc442ac121591ab1d0d6d
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 3 22:18:29 2017 -0400
specify the plugin to fix dependency problem
commit fd021e42a2cdb80676426b65ae227ac9f44fcd6a
Merge: ac92d7b88 1d27a32d5
Author: Otto Fowler <ot...@gmail.com>
Date: Wed May 3 10:34:31 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit ac92d7b88404f8b8bc825c71a4aeee5aa015d757
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Apr 30 09:44:58 2017 -0400
instead of hard-coding in metron-parsers-extensions, overload in the archetype to use metronVersion
commit e21efb89a37d034e36cb82dab6c0b862cad3777b
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Apr 30 09:16:41 2017 -0400
make dependency explicit, I think parent version is not what we think it is when running from archetype, causing the extension versio to be used for this dependency
commit 158f463f121cbca2cf2c4d3c1cd9cf7b707e1d23
Merge: c25585d10 47e5aa70a
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Apr 29 08:58:41 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit c25585d10529ca2048d2ed0df4d8dbd904d3db17
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Apr 29 00:07:11 2017 -0400
use the configured extension list, do not hardcode
commit bc7cc2d823d54be58fae13ef94f671457d4c10f0
Author: Otto Fowler <ot...@gmail.com>
Date: Sat Apr 29 00:05:41 2017 -0400
use correct testing bundle
commit 7760425b02142f01f37d58876aa09fb88b197c82
Merge: c18cbe3a8 5dd87886d
Author: Otto Fowler <ot...@gmail.com>
Date: Thu Apr 27 06:45:37 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit c18cbe3a8847fbed7b5f6ba9182b58f01d142c2c
Merge: e8caaf51e 41b5b1050
Author: Otto Fowler <ot...@gmail.com>
Date: Sun Apr 23 12:14:15 2017 -0400
merge apache/master
commit e8caaf51e3f3d640e9ec1c3c548bd722d595687d
Merge: 850154cb0 e6992d568
Author: Otto Fowler <ot...@gmail.com>
Date: Mon Apr 17 08:19:02 2017 -0400
Merge remote-tracking branch 'apache/master' into METRON-777
commit 850154cb086bd9f825daa6797b575cde95392a87
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Apr 14 08:55:37 2017 -0400
fix package names
commit 864d320d91c522dfc2eb63fc12341f316a3f8952
Author: Otto Fowler <ot...@gmail.com>
Date: Fri Mar 17 00:56:49 2017 -0400
Metron Extension system
Based on Apache Nifi Nars
NAR changes
* new lib , rebrand to bundles from NAR
* port to VFS/FileObject from File based
* ability to set property values
* Rework FileUtils so that you can derive and override
* added initializers to set 'classes' that we care about instead of hard coding them, still needs defaults
* added components nec. for integration tests ( do not want dep. on metron-* )
* VFSClassloader for NarClassLoader
* Hdfs based integration test version of unpacknars tests
* HDFS ( filesystem ) based fileutilities to cover for writes to hdfs, since VFS is currently R/O HDFS
* modified plugin to support configuration of outputs
* use class index not service loader ( both subclass and annotated supported )
Archetype
* Parser Extension archetyp
* incudes all configuration
* creates tar.gz with bundle and configuration
* class index support ( automatic generation )
Extensions
* new extensions modules
* parser
* archetype built module for each parser type
* support for configuration only parsers with tests
Parsers
* moved all but json, csv, grok to extensions
* Bolt now loads from bundle properties
Deployment
* rpms for parsers
* create extension directories
* ambari initializes zookeeper per parser
* amabri creates hdfs directories
* ISSUE: Writing to hdfs
Rest-API
* only test against parsers in metron-parsers
* still needs integration
----
> Metron Extensions System and Parser Extensions Feature Branch
> -------------------------------------------------------------
>
> Key: METRON-1136
> URL: https://issues.apache.org/jira/browse/METRON-1136
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> This jira is for the feature branch effort to create an extension system for Metron, and parser extensions.
> The feature branch is picking up from several other jiras and prs, consolidating the efforts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)