You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "cshannon (via GitHub)" <gi...@apache.org> on 2025/02/14 00:16:42 UTC
[PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
cshannon opened a new pull request, #5335:
URL: https://github.com/apache/accumulo/pull/5335
Adds filtering to TabletMetadataCheckIterator that will use matching families for the provided columns in order to reduce the amount of data that is fetched. A ResolvedColumns instance is returned by each TabletMetadataCheck instance that contains the set of columns and set of matching families to read.
CompactionReservationCheck caches the ResolvedColumns instance statically to avoid having to recreate the set of columns and familes for each scan.
This closes #5254
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956435764
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
> The methods are a bit different so we may need to do something like have some immutable maps that map between ColumnType and ColumnFQ or families.
That would be nice if instead of the switch stmt we had a static `Map<ColumnType, List<Family>>` that was populated in a static code block. Looked for a way to automatically construct this in a static code block and do not see a way w/ the current code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#issuecomment-2657976351
Follow on to #5317
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956580061
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletsMetadata.java:
##########
@@ -326,82 +318,10 @@ public Options checkConsistency() {
public Options fetch(ColumnType... colsToFetch) {
Preconditions.checkArgument(colsToFetch.length > 0);
- for (ColumnType colToFetch : colsToFetch) {
-
- fetchedCols.add(colToFetch);
Review Comment:
Will still need to add things to the set of fetchedColumns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956127173
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
There is a similar method in `TabletsMetadata.Builder.fetch`. I wonder if the logic in that method to decode ColumnTypes to families and qualifiers could be pulled out and re-used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956394406
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
The methods are a bit different so we may need to do something like have some immutable maps that map between ColumnType and ColumnFQ or families. `TabletsMetadata.Builder.fetch` is adding families for some columns and column qualifiers for others where the other method is only resolving families.
I also noticed that the fetch() method is missing `FLUSH_NONCE` from the ColumnType enum, which I suppose means it isn't being used by the fetch() method anywhere but it probably should be added. Maybe we need a test that will iterate over all the values in the enum to verify it's handled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956140704
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
That's a good idea, I'll take a look at refactoring it. One nice benefit of consolidation is that if we add a new column family it reduces the chance of it being forgotten to be handled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956584008
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletsMetadata.java:
##########
@@ -326,82 +318,10 @@ public Options checkConsistency() {
public Options fetch(ColumnType... colsToFetch) {
Preconditions.checkArgument(colsToFetch.length > 0);
- for (ColumnType colToFetch : colsToFetch) {
-
- fetchedCols.add(colToFetch);
Review Comment:
Yep that would indeed be a problem and is what caused the test failures, should be fixed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#issuecomment-2660362049
i made some updates based on feedback, I think the current tests are sufficient now because the static initialization will throw an exception if a new enum is added and not handled for either the family or column mapping. I'm running the sunny build and can merge after that if it passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956489298
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
Nevermind, I think it probably makes sense to just do the mapping/enum changes as part of this PR vs a follow on as it's easier I think to just do it all at once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956435764
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
> The methods are a bit different so we may need to do something like have some immutable maps that map between ColumnType and ColumnFQ or families.
That would be nice if instead of the switch stmt we had a static `Map<ColumnType, List<Family>>` that was populated in a static code block. Looked for a way to automatically construct this in a static code block and do not see a way w/ the current code.
##########
core/src/test/java/org/apache/accumulo/core/metadata/schema/TabletMetadataCheckTest.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.accumulo.core.metadata.schema;
+
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.DIR;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.ECOMP;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.FILES;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.PREV_ROW;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.SCANS;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.SELECTED;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.accumulo.core.data.ArrayByteSequence;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.DataFileColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ExternalCompactionColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ScanFileColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ServerColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.TabletColumnFamily;
+import org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType;
+import org.apache.accumulo.core.metadata.schema.TabletMetadataCheck.ResolvedColumns;
+import org.junit.jupiter.api.Test;
+
+public class TabletMetadataCheckTest {
+
Review Comment:
Could add a test that loops over each column type and makes sure it resolves to some family. Its tricky knowing if its the correct family, but could at least know each column type is covered. This will help someone know they need to make an update when adding a new column type.
##########
server/base/src/main/java/org/apache/accumulo/server/metadata/iterators/TabletMetadataCheckIterator.java:
##########
@@ -98,11 +97,11 @@ public void seek(Range range, Collection<ByteSequence> columnFamilies, boolean i
var colsToRead = check.columnsToRead();
- source.seek(new Range(tabletRow), Set.of(), false);
+ source.seek(new Range(tabletRow), colsToRead.getFamilies(), true);
Review Comment:
When want to read everything for the empty set need to set inclusive to false.
```suggestion
source.seek(new Range(tabletRow), colsToRead.getFamilies(), !colsToRead.getFamilies().isEmpty());
```
##########
server/base/src/main/java/org/apache/accumulo/server/metadata/iterators/TabletMetadataCheckIterator.java:
##########
@@ -98,11 +97,11 @@ public void seek(Range range, Collection<ByteSequence> columnFamilies, boolean i
var colsToRead = check.columnsToRead();
- source.seek(new Range(tabletRow), Set.of(), false);
+ source.seek(new Range(tabletRow), colsToRead.getFamilies(), true);
if (source.hasTop()) {
var tabletMetadata = TabletMetadata.convertRow(new IteratorAdapter(source),
- EnumSet.copyOf(colsToRead), false, false);
+ EnumSet.copyOf(colsToRead.getColumns()), false, false);
Review Comment:
Could move this copy of the set into ResolvedColumns.getColumns() for encapsulation. Then the real internal set never escapes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956435764
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
> The methods are a bit different so we may need to do something like have some immutable maps that map between ColumnType and ColumnFQ or families.
That would be nice if instead of the switch stmt we had a static `Map<ColumnType, Family>` that was populated in a static code block. Looked for a way to automatically construct this in a static code block and do not see a way w/ the current code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956458410
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
The only issue I see is that for fetch(), for some ColumnTypes we fetch families and for some we fetch columns. So I think we may need to split the columns into two groups to grab the type we need which should be doable. I assume we don't want to load the entire family if we only care about one column so we still need to keep this.
i'm working on changes for this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956467340
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
yeah, definitely want to keep the qualifier aspect for code that cares about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956482629
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletMetadata.java:
##########
@@ -202,7 +206,74 @@ public enum ColumnType {
COMPACTED,
USER_COMPACTION_REQUESTED,
UNSPLITTABLE,
- MERGEABILITY
+ MERGEABILITY;
+
+ public static Set<ByteSequence> resolveFamilies(Set<ColumnType> columns) {
Review Comment:
Should this be a follow on issue or should be it be done as part of this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956612982
##########
core/src/main/java/org/apache/accumulo/core/metadata/schema/TabletsMetadata.java:
##########
@@ -326,82 +318,12 @@ public Options checkConsistency() {
public Options fetch(ColumnType... colsToFetch) {
Preconditions.checkArgument(colsToFetch.length > 0);
- for (ColumnType colToFetch : colsToFetch) {
-
- fetchedCols.add(colToFetch);
-
- switch (colToFetch) {
- case CLONED:
- families.add(ClonedColumnFamily.NAME);
- break;
- case DIR:
- qualifiers.add(DIRECTORY_COLUMN);
- break;
- case FILES:
- families.add(DataFileColumnFamily.NAME);
- break;
- case FLUSH_ID:
- qualifiers.add(FLUSH_COLUMN);
- break;
- case AVAILABILITY:
- qualifiers.add(TabletsSection.TabletColumnFamily.AVAILABILITY_COLUMN);
- break;
- case HOSTING_REQUESTED:
- qualifiers.add(TabletsSection.TabletColumnFamily.REQUESTED_COLUMN);
- break;
- case LAST:
- families.add(LastLocationColumnFamily.NAME);
- break;
- case LOADED:
- families.add(BulkFileColumnFamily.NAME);
- break;
- case LOCATION:
- families.add(CurrentLocationColumnFamily.NAME);
- families.add(FutureLocationColumnFamily.NAME);
- break;
- case LOGS:
- families.add(LogColumnFamily.NAME);
- break;
- case PREV_ROW:
- qualifiers.add(PREV_ROW_COLUMN);
- break;
- case SCANS:
- families.add(ScanFileColumnFamily.NAME);
- break;
- case SUSPEND:
- families.add(SuspendLocationColumn.SUSPEND_COLUMN.getColumnFamily());
- break;
- case TIME:
- qualifiers.add(TIME_COLUMN);
- break;
- case ECOMP:
- families.add(ExternalCompactionColumnFamily.NAME);
- break;
- case MERGED:
- families.add(MergedColumnFamily.NAME);
- break;
- case OPID:
- qualifiers.add(OPID_COLUMN);
- break;
- case SELECTED:
- qualifiers.add(SELECTED_COLUMN);
- break;
- case COMPACTED:
- families.add(CompactedColumnFamily.NAME);
- break;
- case USER_COMPACTION_REQUESTED:
- families.add(UserCompactionRequestedColumnFamily.NAME);
- break;
- case UNSPLITTABLE:
- qualifiers.add(SplitColumnFamily.UNSPLITTABLE_COLUMN);
- break;
- case MERGEABILITY:
- qualifiers.add(TabletColumnFamily.MERGEABILITY_COLUMN);
- break;
- default:
- throw new IllegalArgumentException("Unknown col type " + colToFetch);
- }
- }
+ var columns = Set.of(colsToFetch);
+ fetchedCols.addAll(columns);
+ Map<Boolean,Set<ColumnType>> groups = columns.stream()
+ .collect(Collectors.partitioningBy(FETCH_QUALIFIERS::contains, Collectors.toSet()));
+ qualifiers.addAll(ColumnType.resolveQualifiers(groups.get(true)));
+ families.addAll(ColumnType.resolveFamiliesAsText(groups.get(false)));
Review Comment:
The following may do the same thing w/o allocating the intermediate sets, maps, and streams.
```suggestion
for(var col : fetchedCols) {
fetchedCols.add(col);
var colFq = COLUMNS_TO_QUALIFIERS.get(col);
if(colFq != null) {
qualifiers.add(colFq);
} else {
families.addAll(COLUMNS_TO_FAMILIES.get(col));
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#issuecomment-2659954111
@keith-turner - I pushed up something that will map column types to families/qualifiers in https://github.com/apache/accumulo/pull/5335/commits/7fd265a041f06c538d36769d6598b805e95771c4
Can you see what you think? I didn't write any extra tests or anything yet (or make changes for some of the other feedback) as I wanted to get your thoughts before continuing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5335:
URL: https://github.com/apache/accumulo/pull/5335#discussion_r1956759917
##########
core/src/test/java/org/apache/accumulo/core/metadata/schema/TabletMetadataCheckTest.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.accumulo.core.metadata.schema;
+
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.DIR;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.ECOMP;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.FILES;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.PREV_ROW;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.SCANS;
+import static org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType.SELECTED;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.accumulo.core.data.ArrayByteSequence;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.DataFileColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ExternalCompactionColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ScanFileColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.ServerColumnFamily;
+import org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.TabletColumnFamily;
+import org.apache.accumulo.core.metadata.schema.TabletMetadata.ColumnType;
+import org.apache.accumulo.core.metadata.schema.TabletMetadataCheck.ResolvedColumns;
+import org.junit.jupiter.api.Test;
+
+public class TabletMetadataCheckTest {
+
Review Comment:
This test isn't needed now with the latest changes as any new type would throw an exception if not handled during the static map creations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Filter based on columns declared in TabletMetadataCheck [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon merged PR #5335:
URL: https://github.com/apache/accumulo/pull/5335
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org