You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "dqhl76 (via GitHub)" <gi...@apache.org> on 2023/07/14 09:43:42 UTC
[GitHub] [incubator-opendal] dqhl76 opened a new pull request, #2644: test: add a behaviour test for InvalidInput
dqhl76 opened a new pull request, #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263715012
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
```rs
let err = res.unwrap_err();
assert_eq!(_convert_to_error(err).kind(), ErrorKind::InvalidInput);
```
Do you mean a internal convert function like that?
```rs
fn _convert_to_error(err: std::io::Error) -> Error {
let kind = match err.kind() {
std::io::ErrorKind::Interrupted => ErrorKind::InvalidInput,
_ => ErrorKind::Unexpected
};
Error::new(kind, err.to_string().as_str())
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263693651
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
> It seems not a good idea to map them together.
I think we should implement an internal convert function that convert opendal::Error to io::Error (please don't implement `From<std::io::Error> for opendal::Error`, we don't want to leak this to user).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263578549
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
I found that
`let res = r.seek(std::io::SeekFrom::Current(-1024)).await;`
will return a std error `Interrupted` not `InvalidInput`. It seems not a good idea to map them together.
https://github.com/apache/incubator-opendal/blob/58226a619da06f9b018955840ff976377a4a86b1/core/src/raw/oio/read.rs#L172
I am not sure how to do that here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263535581
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
Oh, That why I felt strange all the time. Should I add a new PR or added in this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263645774
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
/cc @Xuanwo for ping
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263564849
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
We can add in this PR directly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#issuecomment-1636718036
@Xuanwo PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263715012
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
```rs
let err = res.unwrap_err();
assert_eq!(_convert_to_error(err).kind(), ErrorKind::InvalidInput);
```
Do you means a internal convert function like that?
```rs
fn _convert_to_error(err: std::io::Error) -> Error {
let kind = match err.kind() {
std::io::ErrorKind::Interrupted => ErrorKind::InvalidInput,
_ => ErrorKind::Unexpected
};
Error::new(kind, err.to_string().as_str())
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263578549
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
I found that
`let res = r.seek(std::io::SeekFrom::Current(-1024)).await;`
will return a std error `Interrupted` not `InvalidInput`. It seems not a good idea to map them together.
https://github.com/apache/incubator-opendal/blob/58226a619da06f9b018955840ff976377a4a86b1/core/src/raw/oio/read.rs#L172
I am not sure how to do that here
---
```rs
let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
assert!(res.is_err());
let err = res.unwrap_err().kind();
debug!("{}",err);
```
```
2023-07-14T12:20:38.665904Z DEBUG behavior::write: operation interrupted
at core/tests/behavior/write.rs:1253
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 closed pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 closed pull request #2644: test: add a behaviour test for InvalidInput
URL: https://github.com/apache/incubator-opendal/pull/2644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263530650
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
Review Comment:
Please check if returing's error kind is `InvalidInput`.
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
Oh, that's not I expected. Our users should don't depend on downcast to get the correct error kind. There are some extra work we need to:
https://github.com/apache/incubator-opendal/blob/58226a619da06f9b018955840ff976377a4a86b1/core/src/types/error.rs#L345-L355
We should mapping the opendal's error kind to io::ErrorKind correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263531096
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
+ let inner_error = res
+ .unwrap_err()
+ .into_inner()
+ .unwrap()
+ .downcast::<Error>()
+ .unwrap();
+ assert_eq!(inner_error.kind(), ErrorKind::InvalidInput);
Review Comment:
I know this is ugly but I can't find a simpler way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] dqhl76 commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "dqhl76 (via GitHub)" <gi...@apache.org>.
dqhl76 commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263670505
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
Review Comment:
https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263578549 Please check this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2644: test: add a behaviour test for InvalidInput
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644#discussion_r1263860968
##########
core/tests/behavior/write.rs:
##########
@@ -1233,3 +1234,31 @@ pub async fn test_fuzz_unsized_writer(op: Operator) -> Result<()> {
op.delete(&path).await.expect("delete must succeed");
Ok(())
}
+
+/// seeking a negative position should return a InvalidInput error
+pub async fn test_invalid_reader_seek(op: Operator) -> Result<()> {
+ let path = uuid::Uuid::new_v4().to_string();
+ debug!("Generate a random file: {}", &path);
+ let (content, _) = gen_bytes();
+
+ op.write(&path, content.clone())
+ .await
+ .expect("write must succeed");
+
+ let mut r = op.reader(&path).await?;
+ let res = r.seek(std::io::SeekFrom::Current(-1024)).await;
+
+ assert!(res.is_err());
+
+ // the res should be a std::io::Error, which contains a ErrorKind::InvalidInput
Review Comment:
The problem happens here:
https://github.com/apache/incubator-opendal/blob/11eb50fb67bb35c203a6d1e009429c6a7afd751f/core/src/raw/oio/read.rs#L164-L174
We convert all `opendal::Error` into `std::io::Error` with `std::io::ErrorKind::Interrupted` directly which is wrong. Instead, we should implement a `convert_to_io_error(err: opendal::Error) -> std::io::Error` so we can return the correct `std::io::Error` to our users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo merged pull request #2644: test: add a behaviour test for InvalidInput
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #2644:
URL: https://github.com/apache/incubator-opendal/pull/2644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org