You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "meteorgan (via GitHub)" <gi...@apache.org> on 2025/01/02 16:12:40 UTC
[PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
meteorgan opened a new pull request, #5500:
URL: https://github.com/apache/opendal/pull/5500
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
Part of #5486.
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
We'll add two methods `if_modified_since` and `if_unmodified_since` for `read_with` and `reader_with` API
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking-changes` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on PR #5500:
URL: https://github.com/apache/opendal/pull/5500#issuecomment-2569082031
> > minio should support those headers: [minio/minio#1098](https://github.com/minio/minio/issues/1098)
>
> `Minio` requires the time format to follow `RFC1123`(like: Fri, 01 Mar 2019 15:00:00 GMT) 😢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #5500:
URL: https://github.com/apache/opendal/pull/5500#issuecomment-2569152819
> `Minio` requires the time format to follow `RFC1123`(like: Fri, 01 Mar 2019 15:00:00 GMT) 😢
Ooops, it does the requirement for `If-Modified-Since` and `If-Unmodified-Since`: https://httpwg.org/specs/rfc9110.html#field.if-modified-since
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901807211
##########
core/src/raw/http_util/body.rs:
##########
@@ -39,7 +39,7 @@ pub struct HttpBody {
/// # Safety
///
-/// HttpBody is send on non wasm32 targets.
+/// HttpBody is sent on non wasm32 targets.
Review Comment:
emmm, you're right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901842368
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
It doesn't work for `S3`. It might be considered an invalid time by `S3` . (I haven't found any official resources about this)
<img width="996" alt="image" src="https://github.com/user-attachments/assets/88258f61-64a7-4c38-90b4-3be16936133c" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901842368
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
It doesn't work for `S3`. It might be considered an invalid time by `S3` . (but I haven't found any official resources about this yet)
<img width="996" alt="image" src="https://github.com/user-attachments/assets/88258f61-64a7-4c38-90b4-3be16936133c" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901927812
##########
core/src/raw/chrono_util.rs:
##########
@@ -62,3 +62,25 @@ pub fn parse_datetime_from_from_timestamp(s: i64) -> Result<DateTime<Utc>> {
Ok(st.into())
}
+
+/// format datetime into http date, this format is required by:
+/// https://httpwg.org/specs/rfc9110.html#field.if-modified-since
+///
Review Comment:
remove this empty comment line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #5500:
URL: https://github.com/apache/opendal/pull/5500#issuecomment-2568061713
minio should support those headers: https://github.com/minio/minio/issues/1098
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1902284752
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
Interesting. Ok, let's test in this way:
- Create a file.
- Retrieve its last modified time.
- Read it using `last_modified - 1s` (this should work).
- Wait for 1 second.
- Attempt to read it using `last_modified + 1s` (this should result in an error).
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901927005
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
It doesn't make sense for the server to reject a future time. I believe AWS S3 is likely to support it, so we can give it a try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901846406
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
Maybe We can just use `chrono::Utc::now()` here. but I'm not sure if this test will remain stable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901852685
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
By the way, Using a future time works for both `Minio` and `Ceph`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on PR #5500:
URL: https://github.com/apache/opendal/pull/5500#issuecomment-2569080429
> minio should support those headers: [minio/minio#1098](https://github.com/minio/minio/issues/1098)
`Minio` requires the time format to follow `RFC1123 `(like: Fri, 01 Mar 2019 15:00:00 GMT) 😢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1902815749
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
Ok, At least, this approach can help reduce the waiting time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan merged PR #5500:
URL: https://github.com/apache/opendal/pull/5500
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901948911
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
I've tested it in my local environment, and the result is the same as using the `AWS CLI`. When using a future time in `if-modified-since`, `S3` always return a `200` status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901753213
##########
core/src/raw/http_util/body.rs:
##########
@@ -39,7 +39,7 @@ pub struct HttpBody {
/// # Safety
///
-/// HttpBody is send on non wasm32 targets.
+/// HttpBody is sent on non wasm32 targets.
Review Comment:
"send" here is not a typo. You can change it to `Send` with "`" if you find it confusing.
##########
core/src/raw/chrono_util.rs:
##########
@@ -62,3 +62,10 @@ pub fn parse_datetime_from_from_timestamp(s: i64) -> Result<DateTime<Utc>> {
Ok(st.into())
}
+
+/// format datetime to rfc1123
+///
+/// for example: `Sun, 06 Nov 1994 08:49:37 GMT`
+pub fn to_rfc_1123(s: DateTime<Utc>) -> String {
Review Comment:
I suggest to use `format_datetime_into_http_date` as API name.
Besides, how about adding a test case for this?
##########
core/src/types/capability.rs:
##########
@@ -115,6 +115,10 @@ pub struct Capability {
pub read_with_override_content_type: bool,
/// Indicates if versions read operations are supported.
pub read_with_version: bool,
+ /// Indicates if conditional read operations using If-Modified-Since are supported.
+ pub read_with_if_modified_since: bool,
Review Comment:
The same.
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
Review Comment:
It's better to remove this sleep.
##########
core/src/services/s3/backend.rs:
##########
@@ -948,6 +948,8 @@ impl Access for S3Backend {
read_with_override_content_disposition: true,
read_with_override_content_type: true,
read_with_version: self.core.enable_versioning,
+ read_with_if_modified_since: true,
Review Comment:
Please arrange them in the correct order.
##########
core/src/raw/ops.rs:
##########
@@ -419,6 +421,28 @@ impl OpRead {
pub fn executor(&self) -> Option<&Executor> {
self.executor.as_ref()
}
+
+ /// Set the If-Modified-Since of the option
+ pub fn with_if_modified_since(mut self, if_modified_since: DateTime<Utc>) -> Self {
Review Comment:
The same, please place besides `if_none_match`
##########
core/src/raw/ops.rs:
##########
@@ -419,6 +421,28 @@ impl OpRead {
pub fn executor(&self) -> Option<&Executor> {
self.executor.as_ref()
}
+
+ /// Set the If-Modified-Since of the option
+ pub fn with_if_modified_since(mut self, if_modified_since: DateTime<Utc>) -> Self {
+ self.if_modified_since = Some(if_modified_since);
+ self
+ }
+
+ /// Get If-Modified-Since from option
+ pub fn if_modified_since(&self) -> Option<DateTime<Utc>> {
+ self.if_modified_since
+ }
+
+ /// Set the If-Unmodified-Since of the option
+ pub fn with_if_unmodified_since(mut self, if_unmodified_since: DateTime<Utc>) -> Self {
Review Comment:
Using `v: DateTime<Utc>` is clear enough; I don't think the full name is necessary here.
##########
core/src/raw/ops.rs:
##########
@@ -304,6 +304,8 @@ pub struct OpRead {
override_content_disposition: Option<String>,
version: Option<String>,
executor: Option<Executor>,
+ if_modified_since: Option<DateTime<Utc>>,
Review Comment:
Please include those fields under `if_none_match`.
##########
core/tests/behavior/async_read.rs:
##########
@@ -270,6 +274,74 @@ pub async fn test_reader_with_if_none_match(op: Operator) -> anyhow::Result<()>
Ok(())
}
+/// Reader with if_modified_since should match, otherwise, a ConditionNotMatch error will be returned.
+pub async fn test_reader_with_if_modified_since(op: Operator) -> anyhow::Result<()> {
+ if !op.info().full_capability().read_with_if_modified_since {
+ return Ok(());
+ }
+
+ let (path, content, _) = TEST_FIXTURE.new_file(op.clone());
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let one_hour_ago_check = chrono::Utc::now() - chrono::Duration::seconds(3600);
+ let reader = op
+ .reader_with(&path)
+ .if_modified_since(one_hour_ago_check)
+ .await?;
+ let bs = reader.read(..).await?.to_bytes();
+ assert_eq!(bs, content);
+
+ sleep(Duration::from_secs(3)).await;
+
+ let one_second_ago_check = chrono::Utc::now() - chrono::Duration::seconds(1);
Review Comment:
How about using `chrono::Utc::now() + chrono::Duration::seconds(10)`? Also, the name `one_second_ago_check` is incorrect. I recommend avoiding lengthy names; a simple `since` should suffice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #5500:
URL: https://github.com/apache/opendal/pull/5500#discussion_r1901751145
##########
core/src/raw/chrono_util.rs:
##########
@@ -62,3 +62,10 @@ pub fn parse_datetime_from_from_timestamp(s: i64) -> Result<DateTime<Utc>> {
Ok(st.into())
}
+
+/// format datetime to rfc1123
+///
+/// for example: `Sun, 06 Nov 1994 08:49:37 GMT`
+pub fn to_rfc_1123(s: DateTime<Utc>) -> String {
Review Comment:
I suggest to use `format_datetime_into_http_date` as API name.
Besides, how about adding a test case for this? We can use the time in the RFC: `Sat, 29 Oct 1994 19:43:31 GMT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(core): implement if_modified_since and if_unmodified_since for read_with and reader_with [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on PR #5500:
URL: https://github.com/apache/opendal/pull/5500#issuecomment-2568035787
Tests failed in `Minio`, I'm not sure if it supports these two options. I'll look into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org