You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Tejaskriya (via GitHub)" <gi...@apache.org> on 2025/02/24 07:49:58 UTC
[PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Tejaskriya opened a new pull request, #7957:
URL: https://github.com/apache/ozone/pull/7957
## What changes were proposed in this pull request?
A repair command which performs compaction for a particular column-family for rocksDB would be helpful in solving performance issues. RocksDB's older versions (being used by ozone) has some known issues like [seekToFirst slowness](https://github.com/facebook/rocksdb/issues/261). This repair command would be a quick fix for these scenarios.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12310
## How was this patch tested?
Tested manually on a cluster:
```
bash-4.2$ ozone repair om compact --db=/opt/hadoop/mytmp/om.db/ --cf=fileTable
ATTENTION: Running as user hadoop. Make sure this is the same user used to run the Ozone process. Are you sure you want to continue (y/N)? y
Run as user: hadoop
Running compaction on fileTable
Compaction completed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2695224664
BTW Yugabyte supports [Admin-Triggered Full Compactions](https://www.yugabyte.com/blog/maximize-database-performance-full-compactions/#admin-triggered-full-compactions). We can follow a similar model with Ozone.
https://www.yugabyte.com/blog/maximize-database-performance-full-compactions/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2002416957
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OMAdminProtocolServerSideImpl.java:
##########
@@ -101,4 +103,20 @@ public DecommissionOMResponse decommission(RpcController controller,
.setSuccess(true)
.build();
}
+
+ @Override
+ public CompactResponse compactDB(RpcController controller, CompactRequest compactRequest)
+ throws ServiceException {
+ try {
+ ozoneManager.compactOMDB(compactRequest.getColumnFamily());
Review Comment:
Thanks for the review! I have made this change now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972639464
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
Review Comment:
Let us also support initializing Compaction with CompactRangeOptions that is a much needed thing.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
Review Comment:
Let us not have an option to compact the entire DB. We can do it on a column family level basis.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
I dont really like the idea of opening a rocksdb directly and performing operations on it. It would be great if all the repair commands open the DB via DBStoreBuilder class. Eventually we can work on initilializing the DBStoreBuilder based on some ini file or json file directly creating an object(setting parameters) based on an input file.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
One question I have is why are we not directly providing a wrapper on top of the rocksdb ldb tool and doing it via java? Given that the cli command is also gonna do the same thing. What is the additional feature are we going to provide here. We are not even initializing the compaction listeners that is needed for Snapshot diff. If we want to go that route we should do this via the DBStoreBuilder class and have a proper ozone wrapper for rocksdb.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2696315075
@jojochuang @swamirishi do we want to support full db compaction?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2701764640
I can see use cases for both online and offline compaction. An online tool may be easier to use and would not have the problem of missed compactions for snapdiff. However, it could not be used on RocksDB instances for other services without implementing custom commands for each one. Particularly datanode and Recon DBs would be difficult to access in this way. I suggest the following:
- We have different Jiras/PRs for OM online (`ozone repair om compact`) and generic offline (`ozone repair ldb compact`)
- Offline compaction can work as initially proposed in this change, with a disclaimer in the help message about OM snapshots.
- Online compaction needs some more design discussion on how to implement the protocol:
- Should we use Ratis to get the request to all 3 OMs, should the client send the request to all 3 automatically, or should the user have to specify the hosts manually?
- The requirements for this command and other online repair commands are similar to those used for dynamic reconfiguration, see [ReconfigureProtocolClientSideTranslatorPB](https://github.com/apache/ozone/blob/194077acbb9672ca05662fca84123938cae17851/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/protocolPB/ReconfigureProtocolClientSideTranslatorPB.java#L63).
- Can we unify this into one interface for giving online commands that are outside of Ratis and the main data path?
- Existing [protos](https://github.com/apache/ozone/blob/3d648466ed53e784b0fd34399cadaaf53894f10b/hadoop-hdds/interface-client/src/main/proto/ReconfigureProtocol.proto) would need to stay the same unless we want to break this in 2.0 (which might be an option), but we might be able to do this with just client/server side code changes and leaving the reconfig protos as they are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972646037
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
One question I have is why are we not directly providing a wrapper on top of the rocksdb ldb tool and doing it via rocks java? Given that the cli command is also gonna do the same thing. What is the additional feature we are we going to provide here? We are not even initializing the compaction listeners that is needed for Snapshot diff. If we want to go that route we should do this via the DBStoreBuilder class and have a proper ozone wrapper for rocksdb.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2001645006
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OMAdminProtocolServerSideImpl.java:
##########
@@ -101,4 +103,20 @@ public DecommissionOMResponse decommission(RpcController controller,
.setSuccess(true)
.build();
}
+
+ @Override
+ public CompactResponse compactDB(RpcController controller, CompactRequest compactRequest)
+ throws ServiceException {
+ try {
+ ozoneManager.compactOMDB(compactRequest.getColumnFamily());
Review Comment:
To avoid this : `Failed to compact column family: nonExistentTable` we can check the table existence here instead of submitting invalid requests to OM
```java
boolean tableExists = ozoneManager.getMetadataManager().getStore().getTable(compactRequest.getColumnFamily())!=null;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2689454450
It would be great to make it a tool that compacts a live OM. Taking down OM and compacts its rocksdb is too much impact.
Here is a small PoC I made today: https://github.com/jojochuang/ozone/pull/563/files
With this tool, you could script it and for example, compact the tables periodically on demand. Or trigger compaction if end to end application latency exceeds a certain threshold. IMO this enhancement makes the tool more useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972646037
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
One question I have is why are we not directly providing a wrapper on top of the rocksdb ldb tool and doing it via rocks java? Given that the cli command is also gonna do the same thing. What is the additional feature are we going to provide here. We are not even initializing the compaction listeners that is needed for Snapshot diff. If we want to go that route we should do this via the DBStoreBuilder class and have a proper ozone wrapper for rocksdb.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2002416434
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/CompactOMDB.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair.om;
+
+import java.io.IOException;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.om.helpers.OMNodeDetails;
+import org.apache.hadoop.ozone.om.protocolPB.OMAdminProtocolClientSideImpl;
+import org.apache.hadoop.ozone.repair.RepairTool;
+import org.apache.hadoop.security.UserGroupInformation;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a column family of an om.db.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a column family in the om.db. " +
+ "The compaction happens asynchronously.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class CompactOMDB extends RepairTool {
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Column family name")
+ private String columnFamilyName;
+
+ @CommandLine.Option(
+ names = {"--service-id", "--om-service-id"},
+ description = "Ozone Manager Service ID",
+ required = false
+ )
+ private String omServiceId;
+
+ @CommandLine.Option(
+ names = {"--node-id"},
+ description = "NodeID of the OM for which db needs to be compacted.",
+ required = false
+ )
+ private String nodeId;
+
+ @Override
+ public void execute() throws Exception {
+
+ OzoneConfiguration conf = getOzoneConf();
+ OMNodeDetails omNodeDetails = OMNodeDetails.getOMNodeDetailsFromConf(
+ conf, omServiceId, nodeId);
+ if (!isDryRun()) {
+ try (OMAdminProtocolClientSideImpl omAdminProtocolClient =
+ OMAdminProtocolClientSideImpl.createProxyForSingleOM(conf,
+ UserGroupInformation.getCurrentUser(), omNodeDetails)) {
+ omAdminProtocolClient.compactOMDB(columnFamilyName);
+ info("Compaction request issued for om.db of om node: %s, column-family: %s.", nodeId, columnFamilyName);
+ info("Please check role logs of %s for completion status.", nodeId);
+ } catch (IOException ex) {
+ error("Couldn't compact column %s. \nException: %s", columnFamilyName, ex);
+ }
+ }
Review Comment:
Yes the compaction is tracked by the snapshot compaction logs. I triggered compaction for om1, and not for om2. (first compaction was auto triggered, the second one was through the command)
The dag generated for om1:
The dag generated for om2:
Verified it with the entries added to the compactionLogTable before and after the command is triggered as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #7957:
URL: https://github.com/apache/ozone/pull/7957
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1994450757
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java:
##########
@@ -2666,6 +2666,17 @@ public void deleteObjectTagging(OmKeyArgs args) throws IOException {
handleError(omResponse);
}
+ @Override
+ public void compactOMDB(String columnFamily) throws IOException {
+ OzoneManagerProtocolProtos.CompactRequest compactRequest =
+ OzoneManagerProtocolProtos.CompactRequest.newBuilder()
+ .setColumnFamily(columnFamily)
+ .build();
+ OMRequest omRequest = createOMRequest(Type.CompactDB)
+ .setCompactRequest(compactRequest).build();
+ handleError(submitRequest(omRequest));
Review Comment:
Yes, good find I didn't know we had this as well. Adding this to the admin protocol looks like a better option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2001619551
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/CompactDBService.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.service;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hdds.utils.db.RDBStore;
+import org.apache.hadoop.hdds.utils.db.RocksDatabase;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedCompactRangeOptions;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.apache.hadoop.util.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This service issues a compaction request for a column family of om.db.
+ */
+public class CompactDBService {
+ private static final Logger LOG = LoggerFactory.getLogger(
+ CompactDBService.class);
+ private final OzoneManager om;
+
+ public CompactDBService(OzoneManager ozoneManager) {
+ this.om = ozoneManager;
+ }
+
+ public CompletableFuture<Void> compact(String columnFamily) throws
+ IOException {
+ return CompletableFuture.supplyAsync(() -> {
+ try {
+ return compactAsync(columnFamily);
+ } catch (Exception e) {
+ LOG.warn("Failed to compact column family: {}", columnFamily, e);
+ }
+ return null;
+ });
+ }
+
+ private Void compactAsync(String columnFamilyName) throws IOException {
+ LOG.info("Compacting column family: {}", columnFamilyName);
+ long startTime = Time.monotonicNow();
+ ManagedCompactRangeOptions options =
+ new ManagedCompactRangeOptions();
+ options.setBottommostLevelCompaction(
+ ManagedCompactRangeOptions.BottommostLevelCompaction.kForce);
Review Comment:
So setting this ensures that L0 files are compacted even if the compaction criteria (num of ssts in a level >= threshold etc) are not met and it forces compaction? Also does this only compact L0 files or other upper levels too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2001637071
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/CompactDBService.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.service;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hdds.utils.db.RDBStore;
+import org.apache.hadoop.hdds.utils.db.RocksDatabase;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedCompactRangeOptions;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.apache.hadoop.util.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This service issues a compaction request for a column family of om.db.
+ */
+public class CompactDBService {
+ private static final Logger LOG = LoggerFactory.getLogger(
+ CompactDBService.class);
+ private final OzoneManager om;
+
+ public CompactDBService(OzoneManager ozoneManager) {
+ this.om = ozoneManager;
+ }
+
+ public CompletableFuture<Void> compact(String columnFamily) throws
+ IOException {
+ return CompletableFuture.supplyAsync(() -> {
+ try {
+ return compactAsync(columnFamily);
+ } catch (Exception e) {
+ LOG.warn("Failed to compact column family: {}", columnFamily, e);
+ }
+ return null;
+ });
+ }
+
+ private Void compactAsync(String columnFamilyName) throws IOException {
+ LOG.info("Compacting column family: {}", columnFamilyName);
+ long startTime = Time.monotonicNow();
+ ManagedCompactRangeOptions options =
+ new ManagedCompactRangeOptions();
+ options.setBottommostLevelCompaction(
+ ManagedCompactRangeOptions.BottommostLevelCompaction.kForce);
Review Comment:
> compacted even if the compaction criteria ... are not met
Correct.
> does this only compact L0 files or other upper levels too?
I believe it compacts everything in the whole range of the column family.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/CompactDBService.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.service;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hdds.utils.db.RDBStore;
+import org.apache.hadoop.hdds.utils.db.RocksDatabase;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedCompactRangeOptions;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.apache.hadoop.util.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This service issues a compaction request for a column family of om.db.
+ */
+public class CompactDBService {
+ private static final Logger LOG = LoggerFactory.getLogger(
+ CompactDBService.class);
+ private final OzoneManager om;
+
+ public CompactDBService(OzoneManager ozoneManager) {
+ this.om = ozoneManager;
+ }
+
+ public CompletableFuture<Void> compact(String columnFamily) throws
+ IOException {
+ return CompletableFuture.supplyAsync(() -> {
+ try {
+ return compactAsync(columnFamily);
+ } catch (Exception e) {
+ LOG.warn("Failed to compact column family: {}", columnFamily, e);
+ }
+ return null;
+ });
+ }
+
+ private Void compactAsync(String columnFamilyName) throws IOException {
+ LOG.info("Compacting column family: {}", columnFamilyName);
+ long startTime = Time.monotonicNow();
+ ManagedCompactRangeOptions options =
+ new ManagedCompactRangeOptions();
+ options.setBottommostLevelCompaction(
+ ManagedCompactRangeOptions.BottommostLevelCompaction.kForce);
Review Comment:
> compacted even if the compaction criteria ... are not met
Correct.
> does this only compact L0 files or other upper levels too?
I believe it compacts everything in the whole range of the column family.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972358210
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
Since it will called like `ozone repair om compact`, Isn't it better to override `serviceToBeOffline` in higher level class `OMRepair` or `SCMRepair`?
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
Review Comment:
Just double-check that it is a sync operation. https://github.com/facebook/rocksdb/issues/3335#issuecomment-356082491 says it is sync.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
Review Comment:
+1, it is throwing an Exception if CFH is null: https://github.com/apache/ozone/pull/7957/files#diff-dea46e7d91f145315788f638476de5de0a6289041671aed36fb703207cbf1a63R62-R65
If we decide to add an option to compact the whole DB, it would be great to have a warning to confirm that the whole DB is getting compacted.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/OMRepair.java:
##########
@@ -31,7 +32,8 @@
FSORepairTool.class,
SnapshotRepair.class,
TransactionInfoRepair.class,
- QuotaRepair.class
+ QuotaRepair.class,
+ RocksDBManualCompaction.class
Review Comment:
I agree with it but `ldb` was removed in HDDS-11999.
IMO, `ldb` was correct because the repair tool could be used for other cases as well, such as Ratis related issues e.g. HDDS-10822.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1991944831
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java:
##########
@@ -2666,6 +2666,17 @@ public void deleteObjectTagging(OmKeyArgs args) throws IOException {
handleError(omResponse);
}
+ @Override
+ public void compactOMDB(String columnFamily) throws IOException {
+ OzoneManagerProtocolProtos.CompactRequest compactRequest =
+ OzoneManagerProtocolProtos.CompactRequest.newBuilder()
+ .setColumnFamily(columnFamily)
+ .build();
+ OMRequest omRequest = createOMRequest(Type.CompactDB)
+ .setCompactRequest(compactRequest).build();
+ handleError(submitRequest(omRequest));
Review Comment:
agreed. Good catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1970817672
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
Review Comment:
```suggestion
description = "Column family name")
```
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/OMRepair.java:
##########
@@ -31,7 +32,8 @@
FSORepairTool.class,
SnapshotRepair.class,
TransactionInfoRepair.class,
- QuotaRepair.class
+ QuotaRepair.class,
+ RocksDBManualCompaction.class
Review Comment:
Since this command does not care which component's RocksDB it operates on, we can probably add an `ozone repair ldb` subcommand to put this under. It could be used for datanodes and Recon as well.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
+ info("Compaction completed.");
+ }
+ } catch (RocksDBException exception) {
+ error("Failed to compact the RocksDB for the given path: %s, column-family:%s", dbPath, columnFamilyName);
+ error("Exception: " + exception);
+ throw new IOException("Failed to compact RocksDB.", exception);
+ } finally {
+ IOUtils.closeQuietly(cfHandleList);
+ }
+ }
+
+ @Override
+ @Nonnull
+ protected Component serviceToBeOffline() {
Review Comment:
If we make this a generic cross component repair tool, we may need to return null to treat it as an "online" tool and rely on the `RocksDBException` to indicate that some component is using the DB. Not sure if there's a better way to handle this.
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
Review Comment:
```suggestion
description = "CLI to compact a column family in the DB.",
```
This is consistent with the `--column-family` named flag
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
Review Comment:
`columnFamilyName` is a required argument currently so it shouldn't be null. Do we want an option to compact the whole DB?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972484123
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/OMRepair.java:
##########
@@ -31,7 +32,8 @@
FSORepairTool.class,
SnapshotRepair.class,
TransactionInfoRepair.class,
- QuotaRepair.class
+ QuotaRepair.class,
+ RocksDBManualCompaction.class
Review Comment:
Thanks for sharing the new CLI proposal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2004107890
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/CompactOMDB.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair.om;
+
+import java.io.IOException;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.om.helpers.OMNodeDetails;
+import org.apache.hadoop.ozone.om.protocolPB.OMAdminProtocolClientSideImpl;
+import org.apache.hadoop.ozone.repair.RepairTool;
+import org.apache.hadoop.security.UserGroupInformation;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a column family of an om.db.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a column family in the om.db. " +
+ "The compaction happens asynchronously.",
Review Comment:
```suggestion
"The compaction happens asynchronously. Requires admin privilege.",
```
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OMAdminProtocolServerSideImpl.java:
##########
@@ -101,4 +103,22 @@ public DecommissionOMResponse decommission(RpcController controller,
.setSuccess(true)
.build();
}
+
+ @Override
+ public CompactResponse compactDB(RpcController controller, CompactRequest compactRequest)
+ throws ServiceException {
+ try {
+ // check if table exists. IOException is thrown if table is not found.
Review Comment:
We should only allow admin to compact it.
```suggestion
if (!ozoneManager.isAdmin(getRemoteUser())) {
throw new OMException("Only administrators are authorized to perform compaction.", PERMISSION_DENIED);
}
// check if table exists. IOException is thrown if table is not found.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2741439070
Merged. Thanks @Tejaskriya @sadanand48 @swamirishi @errose28 @hemantk-12
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2683109793
I'd like to suggest adding an option to support compaction to the bottom most level. Sample code:
```
try (RocksDB db = RocksDB.open(new Options().setCreateIfMissing(true), "/path/to/db")) {
CompactRangeOptions compactOptions = new CompactRangeOptions();
compactOptions.setBottommostLevelCompaction(CompactRangeOptions.BottommostLevelCompaction.FORCE);
db.compactRange(compactOptions, null, null);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1978684362
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/OMRepair.java:
##########
@@ -31,7 +32,8 @@
FSORepairTool.class,
SnapshotRepair.class,
TransactionInfoRepair.class,
- QuotaRepair.class
+ QuotaRepair.class,
+ RocksDBManualCompaction.class
Review Comment:
Since we are implementing online compaction, now it is an OM specific operation. I think it would be better to keep the command under `repair om`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1989516315
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java:
##########
@@ -2666,6 +2666,17 @@ public void deleteObjectTagging(OmKeyArgs args) throws IOException {
handleError(omResponse);
}
+ @Override
+ public void compactOMDB(String columnFamily) throws IOException {
+ OzoneManagerProtocolProtos.CompactRequest compactRequest =
+ OzoneManagerProtocolProtos.CompactRequest.newBuilder()
+ .setColumnFamily(columnFamily)
+ .build();
+ OMRequest omRequest = createOMRequest(Type.CompactDB)
+ .setCompactRequest(compactRequest).build();
+ handleError(submitRequest(omRequest));
Review Comment:
Digging into this, it looks like it will only go to the [leader](https://github.com/apache/ozone/blob/2f3150ab68183b8db61ee758c793fd4c1e1ee594/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/Hadoop3OmTransport.java#L107-L108). For this command, I think we would like it to go to each OM individually as specified when the command is given. We will probably need to look at re-using/refactoring the `ReconfigureProtocolClientSideTranslatorPB` to work more generally for debug/repair commands that need to be run online and not go through Ratis. Otherwise we may end up needing a lot of new/duplicate code to accomplish this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2678989350
@hemantk-12 can you add some details on how this could affect snapshots?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2712785225
Created [pull#8039](https://github.com/apache/ozone/pull/8039) for offline compaction ([HDDS-12533](https://issues.apache.org/jira/browse/HDDS-12533))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1993646076
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java:
##########
@@ -2666,6 +2666,17 @@ public void deleteObjectTagging(OmKeyArgs args) throws IOException {
handleError(omResponse);
}
+ @Override
+ public void compactOMDB(String columnFamily) throws IOException {
+ OzoneManagerProtocolProtos.CompactRequest compactRequest =
+ OzoneManagerProtocolProtos.CompactRequest.newBuilder()
+ .setColumnFamily(columnFamily)
+ .build();
+ OMRequest omRequest = createOMRequest(Type.CompactDB)
+ .setCompactRequest(compactRequest).build();
+ handleError(submitRequest(omRequest));
Review Comment:
another simpler alternative is to use this method from [omAdminProtocol](https://github.com/apache/ozone/blob/5e28acb71de7bf9fcc70c17b2fe8db85b2f354db/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OMAdminProtocolClientSideImpl.java#L79). This should serve the purpose here
``` java
/**
* Create OM Admin Protocol Client for contacting the given OM (does not
* failover to different OM). Use for admin commands such as
* getOMConfiguration which are targeted to a specific OM.
*/
public static OMAdminProtocolClientSideImpl createProxyForSingleOM(
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2004965828
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OMAdminProtocolServerSideImpl.java:
##########
@@ -101,4 +103,22 @@ public DecommissionOMResponse decommission(RpcController controller,
.setSuccess(true)
.build();
}
+
+ @Override
+ public CompactResponse compactDB(RpcController controller, CompactRequest compactRequest)
+ throws ServiceException {
+ try {
+ // check if table exists. IOException is thrown if table is not found.
Review Comment:
Currently we are doing the check in `OzoneManager` by checking the user priviledge. This is the output for when the user doesn't have permissions:
```
bash-4.2$ ozone repair om compact --cf=fileTable --service-id=omservice --node-id=om2
ATTENTION: Running as user hadoop. Make sure this is the same user used to run the Ozone process. Are you sure you want to continue (y/N)? y
Run as user: hadoop
Couldn't compact column fileTable.
Exception: java.io.IOException: Request Failed. Error: Request to compact 'fileTable', sent to om2[om2:9862] failed with error: Only Ozone admins are allowed to compact column family fileTable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Online repair command to perform compaction on om.db [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r2001661651
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/CompactOMDB.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair.om;
+
+import java.io.IOException;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ozone.om.helpers.OMNodeDetails;
+import org.apache.hadoop.ozone.om.protocolPB.OMAdminProtocolClientSideImpl;
+import org.apache.hadoop.ozone.repair.RepairTool;
+import org.apache.hadoop.security.UserGroupInformation;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a column family of an om.db.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a column family in the om.db. " +
+ "The compaction happens asynchronously.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class CompactOMDB extends RepairTool {
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Column family name")
+ private String columnFamilyName;
+
+ @CommandLine.Option(
+ names = {"--service-id", "--om-service-id"},
+ description = "Ozone Manager Service ID",
+ required = false
+ )
+ private String omServiceId;
+
+ @CommandLine.Option(
+ names = {"--node-id"},
+ description = "NodeID of the OM for which db needs to be compacted.",
+ required = false
+ )
+ private String nodeId;
+
+ @Override
+ public void execute() throws Exception {
+
+ OzoneConfiguration conf = getOzoneConf();
+ OMNodeDetails omNodeDetails = OMNodeDetails.getOMNodeDetailsFromConf(
+ conf, omServiceId, nodeId);
+ if (!isDryRun()) {
+ try (OMAdminProtocolClientSideImpl omAdminProtocolClient =
+ OMAdminProtocolClientSideImpl.createProxyForSingleOM(conf,
+ UserGroupInformation.getCurrentUser(), omNodeDetails)) {
+ omAdminProtocolClient.compactOMDB(columnFamilyName);
+ info("Compaction request issued for om.db of om node: %s, column-family: %s.", nodeId, columnFamilyName);
+ info("Please check role logs of %s for completion status.", nodeId);
+ } catch (IOException ex) {
+ error("Couldn't compact column %s. \nException: %s", columnFamilyName, ex);
+ }
+ }
Review Comment:
I hope the compaction done by the command is captured by the snapshot DAG/compaction log. I think we can verify that once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "Tejaskriya (via GitHub)" <gi...@apache.org>.
Tejaskriya commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1978770305
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RocksDBManualCompaction.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.repair;
+
+import jakarta.annotation.Nonnull;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hdds.cli.HddsVersionProvider;
+import org.apache.hadoop.hdds.utils.IOUtils;
+import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB;
+import org.apache.hadoop.ozone.debug.RocksDBUtils;
+import org.rocksdb.ColumnFamilyDescriptor;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.RocksDBException;
+import picocli.CommandLine;
+
+/**
+ * Tool to perform compaction on a table.
+ */
+@CommandLine.Command(
+ name = "compact",
+ description = "CLI to compact a table in the DB.",
+ mixinStandardHelpOptions = true,
+ versionProvider = HddsVersionProvider.class
+)
+public class RocksDBManualCompaction extends RepairTool {
+
+ @CommandLine.Option(names = {"--db"},
+ required = true,
+ description = "Database File Path")
+ private String dbPath;
+
+ @CommandLine.Option(names = {"--column_family", "--column-family", "--cf"},
+ required = true,
+ description = "Table name")
+ private String columnFamilyName;
+
+ @Override
+ public void execute() throws Exception {
+ List<ColumnFamilyHandle> cfHandleList = new ArrayList<>();
+ List<ColumnFamilyDescriptor> cfDescList = RocksDBUtils.getColumnFamilyDescriptors(
+ dbPath);
+
+ try (ManagedRocksDB db = ManagedRocksDB.open(dbPath, cfDescList, cfHandleList)) {
+ ColumnFamilyHandle cfh = RocksDBUtils.getColumnFamilyHandle(columnFamilyName, cfHandleList);
+ if (cfh == null) {
+ throw new IllegalArgumentException(columnFamilyName +
+ " is not in a column family in DB for the given path.");
+ }
+
+ info("Running compaction on " + (columnFamilyName == null ? "entire DB" : columnFamilyName));
+
+ if (!isDryRun()) {
+ db.get().compactRange(cfh);
Review Comment:
Right now on the OM side we are using compactRangeOptions, but i think we will need to determine which options specifically we would want to keep tune. For each of those options a commanLine option will need to be added.
I am not aware of any other way of doing the same, open to suggestions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12310. Repair command to perform compaction on rocksDB [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7957:
URL: https://github.com/apache/ozone/pull/7957#discussion_r1972460618
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/OMRepair.java:
##########
@@ -31,7 +32,8 @@
FSORepairTool.class,
SnapshotRepair.class,
TransactionInfoRepair.class,
- QuotaRepair.class
+ QuotaRepair.class,
+ RocksDBManualCompaction.class
Review Comment:
The commands you referenced were correctly moved from `ozone repair ldb` to `ozone repair om` because they do not work on all RocksDB instances, they only work on the OM. Commands under `ozone repair ldb` must be component agnostic for best usability. This is explained in the tools CLI proposal in #7597
> Commands from different categories should not exist under the same `ozone debug` subcommand. For example, `ozone debug ldb` should not have a mix of subcommands that work on all DB instances and some that only work on OM.
After doing this, there were no subcommands under `ozone repair ldb` at the time so it was removed as well. Now that we have at least one subcommand we can add it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org