[I] [Bug] [Plugin] Can not list objects when using Huawei OBS as resource center. [dolphinscheduler]

["CloudExtreme (via GitHub)" <gi...@apache.org>]

CloudExtreme opened a new issue, #18168:
URL: https://github.com/apache/dolphinscheduler/issues/18168

   ### Search before asking
   
   - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   I'm trying to deploy ds in cluster mode. As official document guide，I need change value of resource.storage.type to another one, such as HDFS, OBS, etc. When I use huawei obs as resource center to storage files, I found no results displayed on the UI, but files actually exit in my obs bucket.
   
   ### What you expected to happen
   
   The file should be displayed on the resource center UI.
   
   ### How to reproduce
   
   - dolphinshceduler version: 3.3.1
   - Java 1.8
   - MySQL 8.0.33 
   
   Upload a file from fronted UI，it worked well, but I can not see it.
   
   ### Anything else
   
   There are also issues in the latest version.
   The problem exits in "listStorageEntity" method from package org.apache.dolphinscheduler.plugin.storage.oss.
   ```java
       public List<StorageEntity> listStorageEntity(String resourceAbsolutePath) {
           resourceAbsolutePath = transformObsKeyToAbsolutePath(resourceAbsolutePath);
   
           ListObjectsRequest request = new ListObjectsRequest();
           request.setBucketName(bucketName);
           request.setPrefix(resourceAbsolutePath);
           request.setDelimiter("/");
   
           ObjectListing result = obsClient.listObjects(request);
   
           return result.getObjects()
                   .stream()
                   .map(this::transformObsObjectToStorageEntity)
                   .collect(Collectors.toList());
       }
   
       private String transformObsKeyToAbsolutePath(String s3Key) {
           if (s3Key.endsWith("/")) {
               return s3Key.substring(0, s3Key.length() - 1);
           }
           return s3Key;
       }
   ```
   transformObsKeyToAbsolutePath method will remove the "/". This will result in an empty return result when list object.
   
   ### Version
   
   3.3.1
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] [Plugin] Can not list objects when using Huawei OBS as resource center. [dolphinscheduler]

["SbloodyS (via GitHub)" <gi...@apache.org>]

SbloodyS closed issue #18168: [Bug] [Plugin] Can not list objects when using Huawei OBS as resource center.
URL: https://github.com/apache/dolphinscheduler/issues/18168


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org