You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by baolsen <gi...@git.apache.org> on 2017/03/23 14:07:55 UTC
[GitHub] nifi pull request #1615: NIFI-3639 - Implemented HBase Get functionality for...
GitHub user baolsen opened a pull request:
https://github.com/apache/nifi/pull/1615
NIFI-3639 - Implemented HBase Get functionality for HBaseClientService
NIFI-3639 - Implemented HBase Get functionality for HBaseClientService
Code changes have been tested with current NiFi / maven build, which completed successfully.
- This is my first PR on Github, please let me know if there is anything I could improve on or if anything is incomplete.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/baolsen/nifi master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1615.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1615
----
commit 8c0285b5efb6afd1607bb050650b758fed7d06e3
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T12:35:43Z
Update HBaseClientService.java
Added "get" function call for doing single row lookup on HBase (HBase get)
commit 03d1b36376c6954d8bdcf4056314fced0cf0d1fc
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:20:41Z
Update HBase_1_1_2_ClientService.java
Implemented "get" function for retrieval of single HBase rows.
commit 6dbca10e82b3b6b8ac94f8f0152b8fff85008082
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:33:15Z
Update HBase_1_1_2_ClientService.java
commit df30a22a3ba71fedfe1dffedefcc0eb64c3670b0
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:40:08Z
Update HBase_1_1_2_ClientService.java
commit 6d8036cc03ef49e41b92dbb5fa7e0de41cc15c3d
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:44:12Z
Update MockHBaseClientService.java
Implemented "get" function with UnsupportedException
commit 4bcb26fd6a99a23852097f4f3db02cbeb6b8a3b5
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:46:23Z
Update HBase_1_1_2_ClientService.java
commit 4b266d9d1d112e2bf8aa198f87253d17c055dbbc
Author: baolsen <bj...@gmail.com>
Date: 2017-03-23T13:50:09Z
Update MockHBaseClientService.java
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1615: NIFI-3639 - Implemented HBase Get functionality for...
Posted by baolsen <gi...@git.apache.org>.
Github user baolsen closed the pull request at:
https://github.com/apache/nifi/pull/1615
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1615: NIFI-3639 - Implemented HBase Get functionality for HBaseC...
Posted by wangrenlei <gi...@git.apache.org>.
Github user wangrenlei commented on the issue:
https://github.com/apache/nifi/pull/1615
@baolsen you should check your style of code .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1615: NIFI-3639 - Implemented HBase Get functionality for HBaseC...
Posted by baolsen <gi...@git.apache.org>.
Github user baolsen commented on the issue:
https://github.com/apache/nifi/pull/1615
Hi @bbende, my pleasure!
Thanks for the quick response.
The intention was to use the Get for another processor I am writing which just needs single row lookups (didn't at first realise that FetchHBaseRow was also doing single row lookups).
I had assumed that a Get would be more efficient than a Scan for fetching single rows.
However, upon further reading it seems that the HBase client API uses a Scan implementation for Gets as well. https://www.cloudera.com/documentation/enterprise/5-4-x/topics/admin_hbase_scanning.html
There are some stack overflow questions regarding Get performance being poorer than Scan, especially when using a key prefix in the scan as opposed to a full rowkey.
https://www.quora.com/What-is-the-difference-between-get-and-scan-in-HBase
It's a little unclear what scenarios cause this performance difference, or whether one approach is more performant in general eg. when the rowkey is a full rowkey as in our case.
In summary, seeing as the HBase client API uses a Scanner under the hood when doing a Get, there should be no real benefit to having a Get added to the code (at least not without doing some practical benchmarks).
I'll use the Scan for my processor instead since it already has the functionality I need. Will add the processor as a separate PR.
Can I close this PR, or does that need to be done on your side?
Also, I see that the automated checks have failed (looks like other components' tests). Is this something I should worry about for my next PR? :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1615: NIFI-3639 - Implemented HBase Get functionality for HBaseC...
Posted by baolsen <gi...@git.apache.org>.
Github user baolsen commented on the issue:
https://github.com/apache/nifi/pull/1615
Great, thanks for all the info.
Will close this PR meanwhile.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1615: NIFI-3639 - Implemented HBase Get functionality for HBaseC...
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1615
@baolsen Thanks for the info. I believe only you can close the pull request on your end.
As far as the automated checks, you can ignore appveryor, but the travis build should be passing as of yesterday. Looking at the travis logs for your build it shows:
[INFO] --- maven-checkstyle-plugin:2.15:check (check-style) @ nifi-hbase-client-service-api ---
[WARNING] src/main/java/org/apache/nifi/hbase/HBaseClientService.java[131] (regexp) RegexpSinglelineJava: Line has trailing whitespace.
You can detect this locally by running "mvn clean install -Pcontrib-check" before submitting anything. The contrib-check will fail if anything doesn't pass checkstyles or the RAT licensing check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1615: NIFI-3639 - Implemented HBase Get functionality for HBaseC...
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1615
Hi @baolsen , thanks for contributing!
One question, is there any difference between the get method being added, and using the existing scan method with startRow and endRow being the same value?
There is a new FetchHBaseRow processor in master that hasn't been released yet, and its using scan to fetch a single row:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/FetchHBaseRow.java#L259-L262
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---