You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/12/03 16:19:55 UTC
[GitHub] [ozone] xichen01 opened a new pull request, #4038: HDDS-7253. Fix exception when '/' in key name
xichen01 opened a new pull request, #4038:
URL: https://github.com/apache/ozone/pull/4038
This is a resubmitted MR, the old one is https://github.com/apache/ozone/pull/3774
This MR fixes an issue that may be making ITestOzoneContractMkdir flaky.
## What changes were proposed in this pull request?
Fix exception when '/' in key name
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7253
## How was this patch tested?
How to reproduce the bug
- First put a key that containing '/'
```bash
[root@Linux /root/ozone]% bin/ozone sh key put s3v/testbucket/dir1/dir2/key1 ~/testfile
[root@Linux /root/ozone]% bin/ozone sh key ls s3v/testbucket/ | grep name
"name" : "dir1/dir2/key1"
```
- Execute command
`ls` command for keys containing '/'
before this commit
```bash
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/
Found 1 items
drwxrwxrwx - root root 0 2022-09-19 14:11 ofs://localhost/s3v/testbucket/dir1
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/dir1
ls: `ofs://localhost/s3v/testbucket/dir1': No such file or directory
```
after this commit
```bash
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/
Found 1 items
drwxrwxrwx - root root 0 2022-09-19 14:37 ofs://localhost/s3v/testbucket/dir1
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/dir1
Found 1 items
drwxrwxrwx - root root 0 2022-09-19 14:37 ofs://localhost/s3v/testbucket/dir1/dir2
```
for `mv`, `mkdir`, `count` command for keys containing '/'
before this commit
```bash
[root@Linux /root/ozone]% bin/ozone fs -count ofs://localhost/s3v/testbucket/
count: dir1: No such file or directory!
[root@Linux /root/ozone]% bin/ozone fs -mv ofs://localhost/s3v/testbucket/dir1 ofs://localhost/s3v/testbucket/dir2
mv: `ofs://localhost/s3v/testbucket/dir1': No such file or directory
[root@Linux /root/ozone]% bin/ozone fs -mkdir ofs://localhost/s3v/testbucket/dir1/dir2/key2
mkdir: `ofs://localhost/s3v/testbucket/dir1/dir2': No such file or directory
```
after this commit
```bash
[root@Linux /root/ozone]% bin/ozone fs -count ofs://localhost/s3v/testbucket/
3 1 1048576 ofs://localhost/s3v/testbucket
[root@Linux /root/ozone]% bin/ozone fs -mv ofs://localhost/s3v/testbucket/dir1 ofs://localhost/s3v/testbucket/newdir1
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/
Found 1 items
drwxrwxrwx - root root 0 2022-09-19 14:39 ofs://localhost/s3v/testbucket/newdir1
[root@Linux /root/ozone]% bin/ozone fs -mkdir ofs://localhost/s3v/testbucket/newdir1/dir2/dir3
[root@Linux /root/ozone]% bin/ozone fs -ls ofs://localhost/s3v/testbucket/newdir1/dir2/
Found 2 items
drwxrwxrwx - root root 0 2022-09-19 14:43 ofs://localhost/s3v/testbucket/newdir1/dir2/dir3
-rw-rw-rw- 3 root root 1048576 2022-09-19 14:39 ofs://localhost/s3v/testbucket/newdir1/dir2/key1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #4038:
URL: https://github.com/apache/ozone/pull/4038#issuecomment-1336393844
Looks like `TestObjectStoreWithLegacyFS.testFlatKeyStructureWithOBS` is flaky:
https://github.com/apache/ozone/actions/runs/3612387380/jobs/6087464989#step:5:3182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #4038:
URL: https://github.com/apache/ozone/pull/4038#issuecomment-1336450007
Seems this issue exists before this PR, but with a smaller chance (1/300 vs 2/100).
https://github.com/kaijchen/ozone/actions/runs/3613290473/jobs/6089472749#step:5:3351
The problem is probably due to `write cache` then `read db without cache`.
I have created HDDS-7588 to track it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #4038:
URL: https://github.com/apache/ozone/pull/4038#issuecomment-1336195748
100x ITestOzoneContractMkdir: https://github.com/kaijchen/ozone/actions/runs/3609358975
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #4038:
URL: https://github.com/apache/ozone/pull/4038#issuecomment-1336397962
100x `TestObjectStoreWithLegacyFS `:
https://github.com/kaijchen/ozone/actions/runs/3613121247
@xichen01, sometimes after a rename, the old keyName is still in table:
```
2022-12-04 20:13:06,066 [Time-limited test] WARN om.TestObjectStoreWithLegacyFS (TestObjectStoreWithLegacyFS.java:assertKeyCount(160)) - keyName: /dnhdqlmxyt/iskcvwgnho/dir1/dir2/dir3/key-1, expected prefix: /dnhdqlmxyt/iskcvwgnho/dir, suffix: dir1/dir2/dir3/key-1
// renaming "dir1/dir2/dir3/key-1" to "dir1/NewKey-1"
2022-12-04 20:13:06,082 [Time-limited test] WARN om.TestObjectStoreWithLegacyFS (TestObjectStoreWithLegacyFS.java:assertKeyCount(160)) - keyName: /dnhdqlmxyt/iskcvwgnho/dir1/dir2/dir3/key-1, expected prefix: /dnhdqlmxyt/iskcvwgnho/dir, suffix: dir1/NewKey-1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #4038:
URL: https://github.com/apache/ozone/pull/4038#issuecomment-1336346832
Thanks @xichen01 for the update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen merged pull request #4038: HDDS-7253. Fix exception when '/' in key name
Posted by GitBox <gi...@apache.org>.
kaijchen merged PR #4038:
URL: https://github.com/apache/ozone/pull/4038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org