[GitHub] [ozone] symious edited a comment on pull request #2424: HDDS-5455. Check invalid keyName when creating new key

[GitBox <gi...@apache.org>]

symious edited a comment on pull request #2424:
URL: https://github.com/apache/ozone/pull/2424#issuecomment-882200589


   @adoroszlai Thanks for the review.
   
   Seems the config of "OZONE_OM_KEYNAME_CHARACTER_CHECK_ENABLED_KEY" will deny
   ```
      * "\", "{", "}", "^", "<", ">", "#", "|", "%", "`", "[", "]", "~", "?"
      * and Non-printable ASCII characters (128–255 decimal characters).
   ```
   , but if the user does create a name contains something like "U-00000000", it won't help with the issue.
   
   I think it is the catastrophic outcome that is not acceptable to users. In our scenario, SRE provides the basic env and we install the applications, we are not quite sure if SRE has set the correcting encoding. So we start the Ozone cluster, then it runs well for several months, suddenly the OM crashes because users are creating some invalid keys. 
   
   Besides, the keys are created by the scheduling system, the invalid key was created several hours after we told them to correct them, which means if we don't check the "Paths.get", the OM cluster isn't available for several hours, and it's not even decided by cluster maintainers, but the users.
   
   It took some time before I found out it's related to the encoding. When the outage happens, and the error message is so obvious, I won't even check the env, so the recovery still relies on the users.
   
   IMHO, the encoding of JVM should affect Ozone so much. If Ozone runs on "ANSI_X3.4-1968", it should provide the functions on this encoding, and if this encoding can not resolve a keyName, it should just return an INVALID_KEY_NAME and reject. If runs on "UTF-8", it should reject the user requests as well if it can not resolve. 
   
   
   Attached the original error log and keyName.
   The original error is as follows:
   ```
   java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /vol_name/bucket_name/task-exec/di_scheduler_test.monhly_dudutestshell063002??prod_2021071519_HOUR_1_0_prod/main.sh
   	at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
   	at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
   	at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
   	at java.nio.file.Paths.get(Paths.java:84)
   	at org.apache.hadoop.ozone.util.RadixTree.getLongestPrefix(RadixTree.java:190)
   	at org.apache.hadoop.ozone.om.PrefixManagerImpl.getLongestPrefixPath(PrefixManagerImpl.java:260)
   	at org.apache.hadoop.ozone.om.request.key.OMKeyRequest.getAclsForKey(OMKeyRequest.java:284)
   	at org.apache.hadoop.ozone.om.request.key.OMKeyRequest.createKeyInfo(OMKeyRequest.java:267)
   	at org.apache.hadoop.ozone.om.request.key.OMKeyRequest.prepareKeyInfo(OMKeyRequest.java:352)
   	at org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest.validateAndUpdateCache(OMKeyCreateRequest.java:278)
   	at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:227)
   	at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:417)
   	at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$applyTransaction$1(OzoneManagerStateMachine.java:242)
   	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   ```
   The keyName user wants to create is `task-exec/di_scheduler_test.monhly_dudutestshell063002\342\200\224\342\200\224prod_2021071519_HOUR_1_0_prod/main.sh`.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org