You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by keith-turner <gi...@git.apache.org> on 2017/03/06 23:51:16 UTC
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
GitHub user keith-turner opened a pull request:
https://github.com/apache/accumulo/pull/229
ACCUMULO-4597 fixed bug in rfile-info
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/keith-turner/accumulo ACCUMULO-4597
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/accumulo/pull/229.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #229
----
commit de80cf53b66f58a9a99d8a9dd54b5a716fa8df44
Author: Keith Turner <kt...@apache.org>
Date: 2017-03-06T23:48:29Z
ACCUMULO-4597 fixed bug in rfile-info
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by keith-turner <gi...@git.apache.org>.
Github user keith-turner commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104807544
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
You can thank findbugs for the message. I had some different code that accomplished the same thing and made findbugs unhappy. So I changed it to this to soothe findbugs. I don't expect this case to happen, and if it does I would have been happy with an NPE. There are many places that could have an NPE when there is a bug, we don't handle them all with detailed error messages.
Findbugs has found some nasty bugs. The false positives can be a little annoying, but I think its worth it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by keith-turner <gi...@git.apache.org>.
Github user keith-turner commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104850830
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
@phrocker did you clear that meme with @mjwall
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/accumulo/pull/229
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104833643
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
> Findbugs has found some nasty bugs. The false positives can be a little annoying, but I think its worth it.
+1000
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104804673
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
Is there any other useful information that could be provided? For example, would it make sense to print out the rest of the locality group column families and help identify if others are listed as a the default and have no column families?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by mjwall <gi...@git.apache.org>.
Github user mjwall commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104904485
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104804870
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
haha and it was merged!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/229#discussion_r104833952
--- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
@@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
}
+ /**
+ * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
+ * {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
+ * more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
+ * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
+ */
public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
for (LocalityGroupMetadata lcg : localityGroups) {
- ArrayList<ByteSequence> setCF = new ArrayList<>();
+ ArrayList<ByteSequence> setCF;
- for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
- setCF.add(entry.getKey());
+ if (lcg.columnFamilies == null) {
+ Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
--- End diff --
"There are many places that could have an NPE when there is a bug, we don't handle them all with detailed error messages."
https://s-media-cache-ak0.pinimg.com/736x/88/15/5e/88155efa961948d0627e61236c7d1f2b.jpg
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---