[GitHub] [accumulo] dlmarion commented on a change in pull request #1946: re #1451: Initial external compaction thrift APIs

[GitBox <gi...@apache.org>]

dlmarion commented on a change in pull request #1946:
URL: https://github.com/apache/accumulo/pull/1946#discussion_r581268874



##########
File path: core/src/main/thrift/compaction-coordinator.thrift
##########
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+namespace java org.apache.accumulo.core.compaction.thrift
+namespace cpp org.apache.accumulo.core.compaction.thrift
+
+include "data.thrift"
+include "security.thrift"
+include "tabletserver.thrift"
+include "trace.thrift"
+
+enum Action {
+  ADD
+  UPDATE
+  CANCEL
+}
+
+enum CompactionState {
+  # Coordinator should set state to ASSIGNED when getCompactionJob is called by Compactor
+  ASSIGNED
+  # Compactor should set state to STARTED when compaction has successfully begun
+  STARTED
+  # Compactor can call repeatedly with an updated message to reflect percentage complete
+  IN_PROGRESS
+  # Compactor should set state to SUCCEEDED when compaction job has successfully finished
+  SUCCEEDED
+  # Compactor should set state to FAILED when compaction job fails, message should be mandatory
+  FAILED
+  # Compactor should set state to CANCELLED to acknowledge that it has stopped compacting 
+  CANCELLED
+}
+
+struct CompactionJob {
+  1:trace.TInfo traceInfo
+  2:security.TCredentials credentials
+  3:i64 compactionId
+  5:data.TKeyExtent extent
+  6:list<string> files
+  7:i32 priority
+  8:i32 readRate
+  9:i32 writeRate
+  10:tabletserver.IteratorConfig iteratorSettings
+  11:tabletserver.CompactionType type
+  # Need to add SELECTOR To CompactionReason, delete CompactionKind?
+  12:tabletserver.CompactionReason reason
+}
+
+struct Status {
+  1:i64 timestamp
+  2:i64 compactionId
+  3:string compactorHostAndPort
+  4:CompactionState state
+  5:string message
+}
+
+service CompactionCoordinator {
+
+  /*
+   * Called by TabletServer (or CLI) to add, update, or cancel a compaction for a tablet
+   */
+  void AddOrUpdateCompaction(
+    1:string tableId
+    2:string endRow
+    3:string queueName
+    4:i64 priority
+    5:Action action
+  )
+  
+  /*
+   * Called by TabletServer (or CLI) to get current status of compaction for a tablet
+   */
+  Status getCompactionStatus(

Review comment:
       I think there might be two cases, which might end up being the same:
   
   1. The compaction manager is constantly checking to see if the tablets need to be compacted. We had talked about, for external compactions, having the tserver just send a compaction request to the coordinator and having the coordinator just ignore the request is one is already queued. An alternate approach would be for the tserver to have (or the ability to know) the if a compaction is occurring or what the state is.
   2. Likewise, if a compaction for a tablet is started, and then the tablet is re-hosted, giving the ability to the tablet server to determine if a compaction is occurring or queued for that tablet.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org