You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "BaurzhanSakhariev (via GitHub)" <gi...@apache.org> on 2024/09/10 08:49:03 UTC

Re: [D] What format should I use for the last-modified property in list API response when using scheme = azblob? [opendal]

GitHub user BaurzhanSakhariev closed a discussion: What format should I use for the last-modified property in list API response when using scheme = azblob?

Hi, thanks for providing a convenient way to access data! 

Have a question about specific API.

I have working implementation for read/write from Azure Blob Storage using java_binding. 

I tried to create a lightweight test by using [stub http server](https://github.com/crate/crate/blob/master/plugins/es-repository-azure/src/test/java/org/elasticsearch/repositories/azure/AzureHttpHandler.java). 

Read/write work fine, but when I try to use `list` API, i get an error `parse datetime from rfc2822 failed`

After checking code, I think I'm hitting last modified date extraction 
https://github.com/apache/opendal/blob/main/core/src/services/azblob/lister.rs#L99

I tried to add `<Last-Modified>some date in rfc2822 format</LastModified>` to the  `List Blobs` part of the stub server and I can confirm that it does react on changes in that property but I never managed to get it working.

Experiments so far:

- `input contains invalid characters` error if I provide dummy string (just to test that list API is sensitive to this property)
- `Connection reset by peer (os error 54)` if I provide some valid date, say,  `Mon, 12 Aug 2024 11:16:46 UT`
- OpenDAL passes `x-ms-date` header, I thought that maybe `list` expects `modified date` in the response to be the same with passed header? I tried using `exchange.getRequestHeaders().get("X-ms-date").get(0)` as LastModified but still getting `os error 54.`

It's probably similar to https://github.com/apache/opendal/pull/3478. 

What format should I use for Last-Modified property? Is there any specific requirement for the date other than being rfc2822 complaint?

Thanks!

GitHub link: https://github.com/apache/opendal/discussions/4998

----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org