[GitHub] [ozone] adoroszlai commented on a change in pull request #2996: HDDS-6194. EC: Freon ockg support EC write

[GitBox <gi...@apache.org>]

adoroszlai commented on a change in pull request #2996:
URL: https://github.com/apache/ozone/pull/2996#discussion_r803543129



##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OzoneClientKeyGenerator.java
##########
@@ -112,7 +140,7 @@ private void createKey(long counter) throws Exception {
 
     timer.time(() -> {
       try (OutputStream stream = bucket.createKey(key, keySize,
-              ReplicationType.RATIS, factor, metadata)) {
+          replicationConfig, metadata)) {
         contentGenerator.write(stream);
         stream.flush();

Review comment:
       Got this for EC key:
   
   ```
   $ ozone freon ockg -n1 -t1 --replication 'rs-3-2-1024k' --type EC
   ...
   NotImplementedException: The flush API is not implemented yet.
   	at org.apache.hadoop.ozone.client.io.ECKeyOutputStream.flush(ECKeyOutputStream.java:468)
   	at org.apache.hadoop.ozone.client.io.OzoneOutputStream.flush(OzoneOutputStream.java:55)
   	at org.apache.hadoop.ozone.freon.OzoneClientKeyGenerator.lambda$createKey$0(OzoneClientKeyGenerator.java:145)
   	at com.codahale.metrics.Timer.time(Timer.java:101)
   	at org.apache.hadoop.ozone.freon.OzoneClientKeyGenerator.createKey(OzoneClientKeyGenerator.java:141)
   ```

##########
File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OzoneClientKeyGenerator.java
##########
@@ -94,6 +113,15 @@ public Void call() throws Exception {
     contentGenerator = new ContentGenerator(keySize, bufferSize);
     metadata = new HashMap<>();
 
+    if (spec.commandLine().getParseResult().hasMatchedOption("--factor")) {
+      replicationConfig = ReplicationConfig
+          .fromTypeAndFactor(ReplicationType.RATIS, factor);
+    } else {
+      replicationConfig = OzoneClientUtils
+          .validateAndGetClientReplicationConfig(replicationType, replication,
+              ozoneConfiguration);
+    }

Review comment:
       Something is wrong around option handling.  Setting only `replication`, but neither `factor` nor `type`, a regular RATIS/3 key was created:
   
   ```
   $ ozone freon ockg -n1 -t1 --replication 'rs-3-2-1024k' -p replication_rs-3-2-1024k
   ...
   Successful executions: 1
   
   $ ozone sh key info /vol1/bucket1/replication_rs-3-2-1024k/0
   ...
     "replicationConfig" : {
       "replicationFactor" : "THREE",
       "requiredNodes" : 3,
       "replicationType" : "RATIS"
     },
   ...
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org