You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "suyanhanx (via GitHub)" <gi...@apache.org> on 2024/05/31 14:44:18 UTC
[PR] ci: set behavior test ci for aliyun drive [opendal]
suyanhanx opened a new pull request, #4657:
URL: https://github.com/apache/opendal/pull/4657
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172710470
It's a bit strange. The env var is also behaving differently on my local machine.
I found this:
https://github.com/rust-lang/rust/issues/104053
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2147724447
It's superficial. Most of those errors reached the limit of read/write.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2147713140
I feel like the implemention has some bug to address. For example:
```
---- behavior::test_list_empty_dir ----
Unexpected (persistent) at List::next, context: { service: aliyun_drive, path: / } => The resource file cannot be found. file not exist
```
List empty dir should return `None` instead of `not found`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "yuchanns (via GitHub)" <gi...@apache.org>.
yuchanns commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2173400032
I rewrite the lister (without getting file details), and LGTM on green. The `test-check` will list items while other threads delete things. So the list behavior randomly gets 404 files not found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2147702638
The env var `RUST_TEST_THREADS` doesn't seem to be working properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "yuchanns (via GitHub)" <gi...@apache.org>.
yuchanns commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172040583
> List empty dir should return None instead of not found?
@Xuanwo It is nothing about List empty dir. I already return `None`. The truth is that AliyunDrive has some limitations with high-concurrency requests. Here I debugged with mimtproxy and showed you the capture:
- Pic 1-2: I requested a path and got a file_id (a folder).
- Pic 3-4: I requested the file_id and got 404.
- Pic 5: Error log showed `context: { service: aliyun_drive, path: / } => The resource file cannot be found. file not exist`.
As I mentioned, the tests go well when limited to 1 thread:
```bash
cargo test behavior --features tests,services-aliyun-drive -- --test-threads=1
```
Otherwise, you'll get random errors such as 404 and io-timeout, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "yuchanns (via GitHub)" <gi...@apache.org>.
yuchanns commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172028576
> List empty dir should return None instead of not found?
@Xuanwo It has nothing to do with List empty dir. I already return `None`. The truth is that AliyunDrive has some limitations with high concurrency requests.
I just debugged with `mimtproxy`. Here is the capture:
- Picture 1: I requested a path, and it returned with a file_id (which is a folder).
- Picture 2-3: I requested a file folder by the file_id, but it responded with 404.
- Picture 4: The error log shows `ERROR opendal::services: service=aliyun_drive operation=list path=/ -> Unexpected (permanent) at List::next => The resource file cannot be found. file not exist`.
As I mentioned, everything goes well when the tests are limited to 1 thread.
```bash
cargo test behavior --features tests,services-aliyun-drive -- --test-threads=1
```
Otherwise, you'll get random errors such as 404 and io timeout, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2173301494
<img width="838" alt="image" src="https://github.com/apache/opendal/assets/24221472/0ab68c70-ead2-4f54-90f7-f448b0a69085">
<img width="1028" alt="image" src="https://github.com/apache/opendal/assets/24221472/793f999f-4afd-4f16-9f71-afcfd8a7d38a">
<img width="1220" alt="image" src="https://github.com/apache/opendal/assets/24221472/189c6df9-2caa-45cc-9ca6-a31948ec82bf">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2142479583
> aliyun drive test failed for
Thanks. I'm handling this on my local machine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #4657:
URL: https://github.com/apache/opendal/pull/4657
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2142476870
aliyun drive test failed for
```
failures:
---- behavior::test_list_empty_dir ----
Unexpected (persistent) at List::next, context: { service: aliyun_drive, path: / } => The resource file cannot be found. file not exist
failures:
behavior::test_list_empty_dir
test result: FAILED. 113 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 291.46s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172384757
> Based on my testing on my host, `RUST_TEST_THREADS=1` does not behave the same as `--test-threads=1`.
But docs said
> This can also be specified with the `RUST_TEST_THREADS` environment variable.
https://doc.rust-lang.org/rustc/tests/index.html#--test-threads-num_threads
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172114850
> As I mentioned, the tests go well when limited to 1 thread:
Odd, we've already set `test_threads=1`.
https://github.com/apache/opendal/blob/f95cb4d06a615acd96f6f2e08dd3ee4b1d3c6ef3/.github/services/aliyun_drive/aliyun_drive/action.yml#L33-L38
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "yuchanns (via GitHub)" <gi...@apache.org>.
yuchanns commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2172369165
> Odd, we've already set test_threads=1.
Based on my testing on my host, `RUST_TEST_THREADS=1` does not behave the same as `--test-threads=1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: set behavior test ci for aliyun drive [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #4657:
URL: https://github.com/apache/opendal/pull/4657#issuecomment-2173470430
Wow, so cool! Is this PR ready to go?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org