[GitHub] [metron] nickwallen opened a new pull request #1456: METRON-2175 Introduce HBase Connection Abstractions for HBase 2.0.2

[GitBox <gi...@apache.org>]

nickwallen opened a new pull request #1456: METRON-2175 Introduce HBase Connection Abstractions for HBase 2.0.2
URL: https://github.com/apache/metron/pull/1456
 
 
   This PR is for the `feature/METRON-2088-support-HDP-3.1` feature branch.
   
   ## Description
   
   The major change in migration to HBase 2.0.2 is that connection management must now be managed by the client (us).  Previously this was managed internally within the HBase client.
   
   More specifically, previously you could just instantiate a Table whenever you like and not worry about closing it.  Now you must first establish a `Connection` and from that `Connection`, you can create a `Table`. Both of those need to be closed after use.  A `Connection` can be shared by multiple threads, but a `Table` cannot be shared.
   
   Most of our existing abstractions were not designed to handle connection management, specifically closing the `Connection` and `Table` resources after use.  This is why the HBase upgrade is quite involved.
   
   This Jira will introduce the new abstractions that allow Metron to interact with HBase 2.0.2, but not remove the existing ones.  I will then introduce additional follow-on PRs to start using these new abstractions in various parts of the Metron code base.  Finally, the existing abstractions that do not work with HBase 2.0.2 will be removed as a final step.
   
   ## Changes
   
   1. Moved the legacy HBase client from `HBaseClient` to `LegacyHBaseClient`.  This allows the code to continue to function while the new abstractions are introduced.  The `LegacyHBaseClient` will be removed before the feature branch for HDP-3.1 is merged into master.
   
   1. Introduced two new profiles `HDP-2.6` and `HDP-3.1`. The `HDP-2.6` profile is currently the default and will continue to compile Metron with all the same versions that we do today.  After the HBase upgrade work is complete, we will change the default profile to `HDP-3.1`.
   
   1. Created the `HBaseClient` interface that expands on the methods provided by the `LegacyHBaseClient`.  These additional methods will allow it to be used more broadly across the code base in future PRs.
   
       1. Created the `HBaseTableClient` which uses HBase's Table to interact with HBase.
       1. Created the `FakeHBaseClient` which will be used for testing purposes.
   
   1. Created the `HBaseClientFactory` that allows an `HBaseClient` to be injected as needed.
   
   1. Created a `HBaseConnectionFactory` that is responsible for creating connections to HBase.
   
       1. Created the `FakeHBaseConnectionFactory` which will be used for testing purposes.
       1. Created the `StaticMockHBaseConnectionFactory` which will be used for testing purposes.
   
   ## Acceptance Testing
   
   This should not impact the deployment and functioning of the system.  As a sanity check, spin-up Full Dev and ensure alerts are visible.
   
   ## Pull Request Checklist
   
   - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
   - [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
   - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
   - [ ] Have you included steps or a guide to how the change may be verified and tested manually?
   - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
   - [ ] Have you written or updated unit tests and or integration tests to verify your changes?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services