You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/11/10 05:26:00 UTC
[jira] [Commented] (METRON-939) Upgrade ElasticSearch and Kibana
[ https://issues.apache.org/jira/browse/METRON-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247028#comment-16247028 ]
ASF GitHub Bot commented on METRON-939:
---------------------------------------
GitHub user mmiklavc opened a pull request:
https://github.com/apache/metron/pull/840
METRON-939: Upgrade ElasticSearch and Kibana
DO NOT MERGE
This PR builds on the great work in https://github.com/apache/metron/pull/619
## Contributor Comments
I wanted to get eyes on this despite some outstanding work left. It was no small undertaking moving 3 major versions, so there's quite a bit to review. In the future I would like to recommend we make feature branches for changes of this magnitude.
This work upgrades Elasticsearch and Kibana both to 5.6. I'm running a local full dev build now, but assuming no failures this will spin up full dev with everything, including the Kibana dashboard (had to be completely rewritten).
### TODO
- [ ] Another merge with master
- [ ] Update docs
- [ ] Docs for managing Kibana dashboards (no more pickle file - straight JSON using ES's bulk load API. This was not as easy as you would think it would be.)
- [ ] Test with Kerberos
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [ ] Have you written or updated unit tests and or integration tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmiklavc/metron elasticsearch562
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/840.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #840
----
commit 0805d20b95e3cdcf55bb0dfde91a08d8b9f58395
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-02T18:51:23Z
Version changes for MPack
commit 889c1bd5d24ee359d7d8113f500539ea10750a40
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-04T14:55:08Z
save demo code
commit 42f82180208dd7235214810406596bfe896665ab
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-17T12:12:43Z
Working through tests
commit 084db5ab4258a1dacb792ba504c1aeb18838327d
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-17T12:30:23Z
fix licensing
commit b33575c0b2131b7f44141342ebaaf3664fe6eab9
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-17T20:23:20Z
fix elasticsearchdao tests
commit 132b3330260dfe5a4b97e91d6210b381eda7222d
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-19T01:17:32Z
working through ES setting migrations for 2.x to 5.x
commit f71321485ea7b54be4ba45e969991bf3696c5197
Author: cstella <ce...@gmail.com>
Date: 2017-10-19T20:07:47Z
Updating to fix kibana mpack and log4j issue.
commit 54c2486e4e618a6d445667ed2040faacf165ca8c
Author: cstella <ce...@gmail.com>
Date: 2017-10-19T20:29:19Z
Removing argline..we should figure out why this is necessary, though. I tried to up the version of surefire.
commit bd48988cf61344eeb2eebdd61caacf1ce83be1d5
Author: cstella <ce...@gmail.com>
Date: 2017-10-19T23:02:23Z
Avoiding log4j version mismatch.
commit 785e6a08558dec9e38468bc7ed50253f4e97ecdd
Author: cstella <ce...@gmail.com>
Date: 2017-10-20T16:43:01Z
Fixed some parameter name changes.
commit b8c81273de0fd82fa4c2978e8e62b3819173aaa7
Author: cstella <ce...@gmail.com>
Date: 2017-10-20T18:33:14Z
Removing and relocating old properties.
commit dcb2dbd1a96a31501fe4a0e0ffbdac70bc4e18b2
Author: cstella <ce...@gmail.com>
Date: 2017-10-20T21:38:28Z
Updating yamls.
commit d5629f0f53207012ee78c862d23c100431f8d2b1
Author: cstella <ce...@gmail.com>
Date: 2017-10-21T02:30:44Z
updating yaml.
commit d5e5cf7f9c353c45ce49b5fd47b2d37acf55d7c3
Author: cstella <ce...@gmail.com>
Date: 2017-10-21T02:51:11Z
Added a limits role to adjust security. We should consider doing this in the mpack possibly..not sure how...
commit 67cc97d922ec06f231e45852dfc71a57618432c2
Author: cstella <ce...@gmail.com>
Date: 2017-10-21T03:20:46Z
typo
commit 297ed1d938892f5de48cc66caab013262be8cb4d
Author: cstella <ce...@gmail.com>
Date: 2017-10-21T04:22:41Z
forgot to remove bootstrap filter checking from master
commit 216b401382d43a9b7db0f9c8d3b410907e93810d
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-23T00:14:15Z
Merge pull request #11 from cestella/elasticsearch562_cstella
Day's work 10/19/2018 - 10/20/2018:)
commit fc92aa9edaacdfa3c618411a9f3f2426fcf51f1f
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-23T03:55:32Z
Get ES templates working
commit f3475bf7a50c733241f2aaf31dc03c08efcc89ad
Author: cstella <ce...@gmail.com>
Date: 2017-10-23T14:58:26Z
updating tests.
commit 12354430487f2a02fd50bc5ac2a6aa24c141dd6b
Author: cstella <ce...@gmail.com>
Date: 2017-10-23T16:21:01Z
Removing log4j-core.
commit 46e0a05b34e43fb2a42aa3ecb5f509678534ae79
Author: cstella <ce...@gmail.com>
Date: 2017-10-23T21:48:31Z
Updating poms to exclude the storm log4j.
commit 798f954ed6fb8598a27f37173a3f0ef66f0164b7
Author: cstella <ce...@gmail.com>
Date: 2017-10-23T23:47:43Z
Updating pom to include log4j
commit 9e2de3bf883b710a1c50f14f2fd4e119ff88e24a
Author: cstella <ce...@gmail.com>
Date: 2017-10-24T00:28:40Z
Adding back in teh log4j but with the right version.
commit c160837ae2863e71c5d392499af575ac1893fb0c
Author: cstella <ce...@gmail.com>
Date: 2017-10-24T13:42:55Z
Updating dependencies with url.
commit a134e230454db7d4dfff0ed8a308ab205efa4285
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-24T13:58:37Z
get netty conflicts with rest api working
commit 30795d78f4801bffdf50f52fa7aa566246f14e7c
Author: cstella <ce...@gmail.com>
Date: 2017-10-24T14:00:12Z
Merge remote-tracking branch 'mmiklavcic/elasticsearch562' into elasticsearch562_tests
commit bdabc5b01f6f80f70394a1fa1088c512b2050ffe
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-24T14:35:52Z
Merge pull request #12 from cestella/elasticsearch562_tests
Testing fixes in travis.
commit e13688fbfcb09624fcad439959c285dd41e32a0b
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-24T17:12:54Z
Fix text/keyword types according to ES mapping recommendations
commit 193891fdee0ace19fde9f1c51ee7560c550c946b
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2017-10-24T17:13:09Z
Merge branch 'elasticsearch562' of https://github.com/mmiklavc/metron into elasticsearch562
commit feaa843ae745ddf66ab1f7504593326d09dda11f
Author: cstella <ce...@gmail.com>
Date: 2017-10-24T17:58:32Z
Merge branch 'master' into elasticsearch562_master
----
> Upgrade ElasticSearch and Kibana
> --------------------------------
>
> Key: METRON-939
> URL: https://issues.apache.org/jira/browse/METRON-939
> Project: Metron
> Issue Type: Improvement
> Reporter: Jon Zeolla
> Assignee: Michael Miklavcic
>
> Upgrade ElasticSearch and Kibana (latest is 5.4 as of writing this). Among other benefits, this allows us to use periods in field names (https://github.com/elastic/elasticsearch/pull/19937/files), which has been available as of 5.0 and 2.4, and the ability to index an IPv6 address properly (https://www.elastic.co/blog/indexing-ipv6-addresses-in-elasticsearch).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)