[jira] [Comment Edited] (JCLOUDS-1504) BlobStore.list(container, ListContainerOptions) returns collection containing null elements

["Енчо Белезирев (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JCLOUDS-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872085#comment-16872085 ] 

Енчо Белезирев edited comment on JCLOUDS-1504 at 6/25/19 7:03 AM:
------------------------------------------------------------------

Hi,

 

I am attaching the wire logs. There is something interesting in it. For instance, there are more than one query for certain blob:
{code:java}
Request: Sending request 782100679: HEAD https://hcp-a0b4ab71-c807-48c5-85b7-797d57d9613e.s3-eu-central-1.amazonaws.com/12bbc932-79c5-4bed-bc9e-a82c6bc4adce HTTP/1.1
Response: Receiving response 782100679: HTTP/1.1 404 Not Found
--------------------------------------------------------------------------------
Request: Sending request 1063600376: HEAD https://hcp-a0b4ab71-c807-48c5-85b7-797d57d9613e.s3-eu-central-1.amazonaws.com/12bbc932-79c5-4bed-bc9e-a82c6bc4adce
Response: Receiving response 1063600376: HTTP/1.1 404 Not Found{code}
I have used the following code in order to obtain the blob's metadata:
{code:java}
Iterable<Module> modules = ImmutableSet.<Module> of(new SLF4JLoggingModule());

// Init
BlobStoreContext context = ContextBuilder.newBuilder("aws-s3")
  .credentials("xxx", "xxxx")
  .modules(modules)
  .buildView(BlobStoreContext.class);

BlobStore blobStore = context.getBlobStore();
final String containerName = "xxxxxxxx";
PageSet<? extends StorageMetadata> list = blobStore.list(containerName, ListContainerOptions.Builder.withDetails());

for (StorageMetadata resourceMd : list) {
 System.out.println(resourceMd.getName());
 System.out.println(resourceMd.getUserMetadata());
}

context.close();

{code}
I need the 
{code:java}
ListContainerOptions.Builder.withDetails(){code}
because I need the userMetadata which contains some custom info about my blobs.

The NullPointerException is happening on this line:
{code:java}
System.out.println(resourceMd.getName());{code}
A little bit of explanation how the blobstore is being read and written:
 # we are uploading files in it
 # each file is being deleted after it is processed


was (Author: belezirev):
Hi,

 

I am attaching the wire logs. There is something interesting in it. For instance, there are more than one query for certain blob:
{code:java}
Request: Sending request 782100679: HEAD https://hcp-a0b4ab71-c807-48c5-85b7-797d57d9613e.s3-eu-central-1.amazonaws.com/12bbc932-79c5-4bed-bc9e-a82c6bc4adce HTTP/1.1
Response: Receiving response 782100679: HTTP/1.1 404 Not Found
-----
Request: Sending request 1063600376: HEAD https://hcp-a0b4ab71-c807-48c5-85b7-797d57d9613e.s3-eu-central-1.amazonaws.com/12bbc932-79c5-4bed-bc9e-a82c6bc4adce
Response: Receiving response 1063600376: HTTP/1.1 404 Not Found{code}
I have used the following code in order to obtain the blob's metadata:
{code:java}
Iterable<Module> modules = ImmutableSet.<Module> of(new SLF4JLoggingModule());

// Init
BlobStoreContext context = ContextBuilder.newBuilder("aws-s3")
  .credentials("xxx", "xxxx")
  .modules(modules)
  .buildView(BlobStoreContext.class);

BlobStore blobStore = context.getBlobStore();
final String containerName = "xxxxxxxx";
PageSet<? extends StorageMetadata> list = blobStore.list(containerName, ListContainerOptions.Builder.withDetails());

for (StorageMetadata resourceMd : list) {
 System.out.println(resourceMd.getName());
 System.out.println(resourceMd.getUserMetadata());
}

context.close();

{code}
I need the 
{code:java}
ListContainerOptions.Builder.withDetails(){code}
because I need the userMetadata which contains some custom info about my blobs.

The NullPointerException is happening on this line:
{code:java}
System.out.println(resourceMd.getName());{code}
A little bit of explanation how the blobstore is being read and written:
 # we are uploading files in it
 # each file is being deleted after it is processed

> BlobStore.list(container, ListContainerOptions) returns collection containing null elements 
> --------------------------------------------------------------------------------------------
>
>                 Key: JCLOUDS-1504
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1504
>             Project: jclouds
>          Issue Type: Bug
>            Reporter: Енчо Белезирев
>            Priority: Blocker
>         Attachments: jclouds-wire.log, jclouds.log
>
>
> Hello,
> I am an SAP developer and we are using jclouds for the communication with our blob stores.
> We have issue with the querying of the StorageMetadata for some blobs. We are using the method from the BlobStore - 
> {code:java}
> PageSet<? extends StorageMetadata> list(String container, ListContainerOptions options);{code}
> However, when we are using the method, we are providing ListContainerOptions.Builder.withDetails() because we want to take directly the userMetadata field from the StorageMetadata object and to use it later on. Here comes the problem, sometimes, when the method is being executed, the list that is being returned contains null objects. This is a problem because we want each StorageMetadata to be non-null.
> I have validated our entries in the container and it seemed that there are no suspicious entries in it(which might cause the issue). 
> Could you share some information what could have caused the issue and eventually, is there a way to workaround this?
>  
> We are using aws-s3 implementation of the blob store.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)