You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by merrimanr <gi...@git.apache.org> on 2018/05/02 20:12:26 UTC
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/1008
METRON-1545: Upgrade Spring and Spring Boot
## Contributor Comments
This PR upgrades Spring to version [5.0.5.RELEASE](https://docs.spring.io/spring/docs/5.0.5.RELEASE/spring-framework-reference/) and Spring Boot to version [2.0.1.RELEASE](https://docs.spring.io/spring-boot/docs/2.0.1.RELEASE/reference/htmlsingle/). A summary of the changes included in this PR (all related to metron-rest):
- Incremented Spring maven dependency versions
- Added documentation for creating authentication tables
- Incremented Jackson version to match the version Spring depends on
- Added an explicit dependency on JsonPath to avoid conflicts with the provided metron-parsers uber jar
- Disabled auto-configuration for Gson and Kafka to avoid version conflict problems that result from metron-parsers and metron-elasticsearch including these classes on the classpath.
- Disabled automatic logging configuration to avoid version conflict problems that result from metron-elasticsearch including log4j2 classes on the classpath
- Updated the JpaConfiguration class to match newer constructor
- Configured a NoOpPasswordEncoder to satisfy the password encoder requirements introduced in newer versions of Spring
- Fixed a bug in the test profile related to the Spring banner
- Added the default application.yml as a configuration file since it is no longer added by default when the --spring.config.location option is used
- Updated depedencies_with_url.csv to include all of the recent versions introduced by the upgrade
I tested this in full dev by manually interacting with all the REST endpoints including:
- creating and editing alert UI settings
- creating, updating, and deleting the various configs (global, parser, enrichment, indexing)
- listing, getting and validating grok statements
- creating, reading, updating, and deleting HDFS files
- creating, listing, deleting Kakfa topics as well as consuming and producing messages
- performing searches, lookups, groupings, and updates with our DAO endpoints
- creating and editing metaalerts
- stopping, starting and getting status of Storm topologies
I also went through the exercise of configuring a MySQL database for authentication.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
```
mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to verify your changes?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/merrimanr/incubator-metron spring-upgrade
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1008.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1008
----
commit 09aab7e47b34dd996a13372d5233605cc182ae0d
Author: merrimanr <me...@...>
Date: 2018-05-02T18:02:24Z
initial commit
commit 420399c0e8879721869edf7c56e6afede5e605fa
Author: merrimanr <me...@...>
Date: 2018-05-02T19:29:42Z
updated dependencies_with_url.csv
----
---
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by merrimanr <gi...@git.apache.org>.
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/1008#discussion_r185898854
--- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java ---
@@ -87,13 +91,18 @@ public void configureJdbc(AuthenticationManagerBuilder auth) throws Exception {
List<String> activeProfiles = Arrays.asList(environment.getActiveProfiles());
if (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) ||
activeProfiles.contains(MetronRestConstants.TEST_PROFILE)) {
- auth.jdbcAuthentication().dataSource(dataSource)
- .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
+ auth.jdbcAuthentication().dataSource(dataSource)
+ .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
} else {
auth.jdbcAuthentication().dataSource(dataSource);
}
}
+
+ @Bean
+ public PasswordEncoder passwordEncoder() {
+ return NoOpPasswordEncoder.getInstance();
--- End diff --
I agree we should have that discussion anyways. I will start a thread on that topic.
---
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/1008
---
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/1008#discussion_r185860632
--- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java ---
@@ -87,13 +91,18 @@ public void configureJdbc(AuthenticationManagerBuilder auth) throws Exception {
List<String> activeProfiles = Arrays.asList(environment.getActiveProfiles());
if (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) ||
activeProfiles.contains(MetronRestConstants.TEST_PROFILE)) {
- auth.jdbcAuthentication().dataSource(dataSource)
- .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
+ auth.jdbcAuthentication().dataSource(dataSource)
+ .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
} else {
auth.jdbcAuthentication().dataSource(dataSource);
}
}
+
+ @Bean
+ public PasswordEncoder passwordEncoder() {
+ return NoOpPasswordEncoder.getInstance();
--- End diff --
Admitting that I have near total ignorance here, why are we using a NoOpPasswordEncoder instead of something else? What are the implications of this?
---
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by merrimanr <gi...@git.apache.org>.
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/1008#discussion_r185883201
--- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java ---
@@ -87,13 +91,18 @@ public void configureJdbc(AuthenticationManagerBuilder auth) throws Exception {
List<String> activeProfiles = Arrays.asList(environment.getActiveProfiles());
if (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) ||
activeProfiles.contains(MetronRestConstants.TEST_PROFILE)) {
- auth.jdbcAuthentication().dataSource(dataSource)
- .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
+ auth.jdbcAuthentication().dataSource(dataSource)
+ .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
} else {
auth.jdbcAuthentication().dataSource(dataSource);
}
}
+
+ @Bean
+ public PasswordEncoder passwordEncoder() {
+ return NoOpPasswordEncoder.getInstance();
--- End diff --
The reason I used that particular encoder is to keep the system working the same way it does now. Using a different encoder would significantly alter the way we manage users and passwords. It will also force people to migrate all passwords to a new encoding when upgrading.
I would like to see a broader discussion on our authentication strategy outside of this PR. I suspect we will end up moving away from JDBC authentication anyways and more towards a unified architecture that aligns with other components in our stack.
---
[GitHub] metron issue #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/1008
I'm +1 by inspection. Thanks for the contribution!
---
[GitHub] metron issue #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1008
+1 Thanks for the upgrade and all the fixes. This look solid. I ran this up in CentOS 6 and validated the Alerts UI and ran a Service Check successfully.
---
[GitHub] metron pull request #1008: METRON-1545: Upgrade Spring and Spring Boot
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/1008#discussion_r185894824
--- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java ---
@@ -87,13 +91,18 @@ public void configureJdbc(AuthenticationManagerBuilder auth) throws Exception {
List<String> activeProfiles = Arrays.asList(environment.getActiveProfiles());
if (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) ||
activeProfiles.contains(MetronRestConstants.TEST_PROFILE)) {
- auth.jdbcAuthentication().dataSource(dataSource)
- .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
- .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
+ auth.jdbcAuthentication().dataSource(dataSource)
+ .withUser("user").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and()
+ .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN);
} else {
auth.jdbcAuthentication().dataSource(dataSource);
}
}
+
+ @Bean
+ public PasswordEncoder passwordEncoder() {
+ return NoOpPasswordEncoder.getInstance();
--- End diff --
Assuming it matches what was happening pre-upgrade, I'm fine with it for an upgrade ticket.
Having a broader discussion outside this PR is a good idea. Would you be able to kick off a discuss thread regarding current vs. proposed feature state? It seems like that discussion should be had regardless of this PR.
---