[GitHub] [metron] nickwallen opened a new pull request #1465: METRON-2172 Solr Updates Not Tested in Integration Test

[GitBox <gi...@apache.org>]

nickwallen opened a new pull request #1465: METRON-2172 Solr Updates Not Tested in Integration Test
URL: https://github.com/apache/metron/pull/1465
 
 
   The `SolrUpdateIntegrationTest` is not testing that the `SolrDao` can update and retrieve values. This fix is similar to what was done for Elasticsearch in #1451.
   
   This replaces #1454 which was a more involved refactor that fixed some issues with code duplication and provided additional unit tests.  I am going to break-off just the bare minimum of what we need to fix the broken tests, rather than fix and refactor in the same PR.
   
   ### What?
   This gap in the integration test is hiding a couple bugs.  
   
   1. A timestamp is not being populated in the documents returned from Solr.
   
   1. The integration test is not accounting for the fact that comments are serialized differently when written to Solr versus Elasticsearch.  When comments are written to Solr, they are serialized as a list of JSON strings.  When comments are written to Elasticsearch, they are serialized as a list of maps.
   
   ### Why? 
   These problems arise because of the way the test is setup.  Instead of directly testing a `SolrDao` this test runs against a `MultiIndexDao` initialized with both a `SolrDao` and an `HBaseDao`. On retrievals the `MultIndexDao` will return the document from whichever index responds first.
   
   With the current test setup, the underlying `SolrDao` will never retrieve the document that the test case is expecting.  In all cases where the test passes, the document is actually being returned from the `HBaseDao` which is actually just interacting with a mock backend.  The test needs to actually test that we can update and retrieve documents from Elasticsearch.
   
   ### Changes
   
   1. The `SolrUpdateIntegrationTest` was changed so that it specifically tests a `SolrDao`.
   
   1. The `UpdateIntegrationTest` was changed so that it can handle comments stored as either strings or maps.  This allows the same integration tests to continue to work when run against either Solr or Elasticsearch.
   
   ### Acceptance Testing
   
   - [ ] Validate against Elasticsearch.
       1. Spin-up Full Dev with Elasticsearch running.
       1. Ensure alerts are visible in the Alerts UI.
       1. Add a comment to an alert.
   
   - [ ] Validate against Solr.
       1. Spin-up Full Dev with Solr running.
       1. Ensure alerts are visible in the Alerts UI.
       1. Add a comment to an alert.
   
   ## Pull Request Checklist
   - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
   - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
   - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
   - [x] Have you included steps or a guide to how the change may be verified and tested manually?
   - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
   - [x] Have you written or updated unit tests and or integration tests to verify your changes?
   - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services