You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by melrief <gi...@git.apache.org> on 2016/07/16 23:02:50 UTC
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
GitHub user melrief opened a pull request:
https://github.com/apache/accumulo/pull/125
ACCUMULO-4375 Add missing byte[]-based Key constructors
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/melrief/accumulo ACCUMULO-4375
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/accumulo/pull/125.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #125
----
commit e9644f2c7e382019c62c4ea3ed612a88a63729dd
Author: Mario Pastorelli <ma...@teralytics.ch>
Date: 2016-07-16T23:01:31Z
ACCUMULO-4375 Add missing byte[]-based Key constructors
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
Great! Feel free to ping me when you have things updated and I'll give you another review. The unit tests should always be passing reliably so feel free to use those as verification that your changes are working as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
@joshelser done. I don't know were my head was yesterday. I was so sure about `getBytes` and `getLength` behaviour that I simply didn't check. I wrote a simple tests comparing created keys and I was sure that was enough to check the patch. Not my best "first contribution" to an OSS project :). I'll try to be more careful in future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
I think aside from the minor javadoc request I had, this looks good. If you have time, feel free to throw that up; otherwise, I can just modify it on commit. LMK
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/125#discussion_r71085502
--- Diff: core/src/main/java/org/apache/accumulo/core/data/Key.java ---
@@ -107,6 +107,17 @@ public Key(Text row) {
}
/**
+ * Creates a key with the specified row, empty column family, empty column qualifier, empty column visibility, timestamp {@link Long#MAX_VALUE}, and delete
+ * marker false.
--- End diff --
I'll do it. I wonder if this is the right behaviour or if the user would expect the byte array to be used as it is. In my version of this that I'm using at work, I'm setting `copy` to false all the time because I know that's safe considering how I'm using the arrays. When I submitted this patch, I decided to switch to true because it was more safe. Maybe we should add this parameter to each constructor using `byte[]` explicitly so that the user knows that he can avoid copying the array?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
@dlmarion I've deleted the old branch because rebasing was painful, I had too many conflicts for a very small patch. I've recreated the branch on top of 1.8 and created a new PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
This cannot be merged as-is. The code does not work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/125#discussion_r71086520
--- Diff: core/src/main/java/org/apache/accumulo/core/data/Key.java ---
@@ -107,6 +107,17 @@ public Key(Text row) {
}
/**
+ * Creates a key with the specified row, empty column family, empty column qualifier, empty column visibility, timestamp {@link Long#MAX_VALUE}, and delete
+ * marker false.
--- End diff --
> I wonder if this is the right behaviour or if the user would expect the byte array to be used as it is.
Copying the underlying array is definitely the safe/consistent thing to do. I think we should avoid making another set of constructors just to support users providing `copy=false`. That's pretty much what I was thinking about with ACCUMULO-4376.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
Wops, sorry for that. I had a look at the tests but they didn't look related. My mistake, I'll fix the PR. I'm going to `--amend` the commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
Tests are failing but on unrelated tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/125#discussion_r71085334
--- Diff: core/src/main/java/org/apache/accumulo/core/data/Key.java ---
@@ -107,6 +107,17 @@ public Key(Text row) {
}
/**
+ * Creates a key with the specified row, empty column family, empty column qualifier, empty column visibility, timestamp {@link Long#MAX_VALUE}, and delete
+ * marker false.
--- End diff --
I think adding a Javadoc comment to your method additions (all of them, not just this one) that explicitly state that they copy the provided byte arrays is important. This can be a big performance impact -- you could point to the `Key(byte[] row, byte[] cf, byte[] cq, byte[] cv, long ts, boolean deleted, boolean copy)` constructor which does allow users to control whether or not the constructor will do a copy of the provided byte arrays.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
Cool, this looks good to me, @melrief. Let me run it through some tests and will commit if it passes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by keith-turner <gi...@git.apache.org>.
Github user keith-turner commented on the issue:
https://github.com/apache/accumulo/pull/125
@melrief this looks good. The behavior is consistent w/ the Text constructors, which is really nice. Looking at the test and javadoc you added inspired me to open [ACCUMULO-4377](https://issues.apache.org/jira/browse/ACCUMULO-4377). Adding the test for copy behavior is more general than this PR, since existing constructors do not have those test.
@melrief one additional thing thats needed is `@since 1.8.0` tags. Would you be interested in doing another PR to add those?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/125#discussion_r71082922
--- Diff: core/src/main/java/org/apache/accumulo/core/data/Key.java ---
@@ -103,7 +103,18 @@ public Key() {
* row ID
*/
public Key(Text row) {
- init(row.getBytes(), 0, row.getLength(), EMPTY_BYTES, 0, 0, EMPTY_BYTES, 0, 0, EMPTY_BYTES, 0, 0, Long.MAX_VALUE, false, true);
+ this(row.getBytes());
--- End diff --
This is wrong. `Text.getBytes()` will return the backing array for the `Text` object, but the contents of the data in the `Text` object may be less than the length of the array. Make sure you use the initial position of `0` and a length of `Text.getLength()`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by keith-turner <gi...@git.apache.org>.
Github user keith-turner commented on the issue:
https://github.com/apache/accumulo/pull/125
I opened ACCUMULO-4378 as subtask of ACCUMULO-4375 and made it blocker (just so we dont forget about it for 1.8).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
Haha, not worries @melrief! There are definitely no feelings of animosity for less-than-perfect contributions (don't google for mine ;)). Taking a look at your latest now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
@joshelser yes, I'm interested in helping with it. I do agree overloading the constructor of `Key` so many times can be confusing and not very maintainable or flexible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on the issue:
https://github.com/apache/accumulo/pull/125
@keith-turner sure, I'll do it tonight.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/125
@melrief, these changes also inspired me to create https://issues.apache.org/jira/browse/ACCUMULO-4376. We're getting to lots of constructors for `Key`. Mentioning it because I thought you may also be interested in helping out there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #125: ACCUMULO-4375 Add missing byte[]-based Key constructors
Posted by dlmarion <gi...@git.apache.org>.
Github user dlmarion commented on the issue:
https://github.com/apache/accumulo/pull/125
LGTM. @mjwall - can we get this into the 1.8 release?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/accumulo/pull/125
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #125: ACCUMULO-4375 Add missing byte[]-based Key const...
Posted by melrief <gi...@git.apache.org>.
Github user melrief commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/125#discussion_r71086057
--- Diff: core/src/main/java/org/apache/accumulo/core/data/Key.java ---
@@ -107,6 +107,17 @@ public Key(Text row) {
}
/**
+ * Creates a key with the specified row, empty column family, empty column qualifier, empty column visibility, timestamp {@link Long#MAX_VALUE}, and delete
+ * marker false.
--- End diff --
Done, now each method has a note
```java
This constructor creates a copy of each specified array. If you don't want to create a copy of the arrays, you should call
* {@link Key#Key(byte[] row, byte[] cf, byte[] cq, byte[] cv, long ts, boolean deleted, boolean copy)} instead.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---