You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/01/12 14:32:52 UTC
[jira] [Commented] (METRON-652) Extract indexing config from
enrichment config
[ https://issues.apache.org/jira/browse/METRON-652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821122#comment-15821122 ]
ASF GitHub Bot commented on METRON-652:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/415
METRON-652: Extract indexing config from enrichment config
Currently, the indexing configuration is bound and coupled with the sensor enrichment configuration. This was done historically because indexing and enrichment were part of the same topology. When the topologies were separated, the configurations were never separated, leaving a confusing section about indexes in the middle of the enrichment configuration. This effort will separate out the configuration.
Because of the configuration's simplicity, we are treating the config as just a `Map<String, Object>` and supporting the two existing configurations:
* `batchSize`
* `index`
Because this change is fairly deep, a few non-obvious things were also updated for consistency and necessity:
* The Stellar Management functions `ENRICHMENT_SET_BATCH` and `ENRICHMENT_SET_INDEX` functions have been renamed to `INDEXING_SET_BATCH` and `INDEXING_SET_INDEX`
* Support in `CONFIG_GET` and `CONFIG_PUT` for retrieving/storing indexing configurations was added.
* Support for indexing configurations were added to the zookeeper utility function
* The indexing RPM was updated to include the new configs for the shipped sensors.
I have *not* updated the management pack to create a new panel for managing indexing configs. That will be part of a follow-on JIRA (METRON-653).
I have done smoke testing on quickdev, but will include more comprehensive testing explanation in the comments. I feel that this has the possibility of causing regressions, so I want to be explicit in testing methodology.
Furthermore, I plan on creating a wiki section about about migrations, because this will necessitate migrating configs when upgrading from 0.3.0 to the next version.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron METRON-652
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/415.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #415
----
commit 1d03bc57235af2c1b90c901e7895f3c037a054c2
Author: cstella <ce...@gmail.com>
Date: 2017-01-10T14:25:33Z
initial update
commit 4c9227cb618a62bfea63b147c32054bfeadf7d08
Author: cstella <ce...@gmail.com>
Date: 2017-01-10T15:48:33Z
Updating testing components.
commit 3681edb32dca2876ecb2c0895c58a392ce558ada
Author: cstella <ce...@gmail.com>
Date: 2017-01-10T20:32:47Z
second round of changes.
commit facb6518b27fd19fd5c3f40671b57b84013c4076
Author: cstella <ce...@gmail.com>
Date: 2017-01-11T15:03:14Z
Merge branch 'master' into METRON-652
commit ec91c279af078c94c4d762bd8ed166a18fa78da3
Author: cstella <ce...@gmail.com>
Date: 2017-01-11T15:03:40Z
Merge branch 'master' into METRON-652
commit ed98341e904575f4d1e89533440cb920ee7a6e3d
Author: cstella <ce...@gmail.com>
Date: 2017-01-11T17:54:59Z
updating.
commit 375e0260ce2d12e38e66e8ca05740bf6f71e348c
Author: cstella <ce...@gmail.com>
Date: 2017-01-11T21:10:16Z
Updating documentation.
commit 3736294bfe9bb4f3f6c3862af300da49960da81f
Author: cstella <ce...@gmail.com>
Date: 2017-01-11T21:15:24Z
Updating RPM.
----
> Extract indexing config from enrichment config
> ----------------------------------------------
>
> Key: METRON-652
> URL: https://issues.apache.org/jira/browse/METRON-652
> Project: Metron
> Issue Type: Sub-task
> Reporter: Casey Stella
> Assignee: Casey Stella
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)