You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/11/15 02:57:34 UTC
[GitHub] [incubator-doris] wangshuo128 opened a new issue #7116: [Bug] Segment files in rowset got deleted before committing when doing stream load.
wangshuo128 opened a new issue #7116:
URL: https://github.com/apache/incubator-doris/issues/7116
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
0.13
### What's Wrong?
Sometimes uncommitted segment files would be deleted by the GC thread when doing stream load.
### What You Expected?
Segment files shouldn't be deleted by mistake.
### How to Reproduce?
Don't know yet.
### Anything Else?
BE logs:
```
I1109 06:12:36.945013 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_0.dat
I1109 06:12:36.945089 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_1.dat
I1109 06:12:36.945164 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_10.dat
I1109 06:12:36.951720 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_11.dat
I1109 06:12:36.951795 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_12.dat
I1109 06:12:36.951846 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_13.dat
I1109 06:12:36.951897 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_14.dat
I1109 06:12:36.951946 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_15.dat
I1109 06:12:36.952000 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_16.dat
I1109 06:12:36.952051 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_17.dat
I1109 06:12:36.952098 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_18.dat
I1109 06:12:36.952179 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_19.dat
I1109 06:12:36.952230 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_2.dat
I1109 06:12:36.952281 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_20.dat
I1109 06:12:36.952332 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_21.dat
I1109 06:12:36.952386 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_3.dat
I1109 06:12:36.952433 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_4.dat
I1109 06:12:36.952497 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_5.dat
I1109 06:12:36.952550 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_6.dat
I1109 06:12:36.952600 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_7.dat
I1109 06:12:36.952647 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_8.dat
I1109 06:12:36.952693 204030 data_dir.cpp:960] collect garbage dir path: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_9.dat
I1109 06:15:12.528883 204135 txn_manager.cpp:283] commit transaction to engine successfully. partition_id: 128312152, transaction_id: 123993241, tablet: 128312153.973533064.284deaddd2525d11-f9ae1941e5d035a5, rowsetid: 0200000000000be9274d994a6437af1e83d38b865617b9bf, version: 0
I1109 06:16:05.806581 204056 txn_manager.cpp:340] publish txn successfully. partition_id: 128312152, txn_id: 123993241, tablet: 128312153.973533064.284deaddd2525d11-f9ae1941e5d035a5, rowsetid: 0200000000000be9274d994a6437af1e83d38b865617b9bf, version: 29,29
W1109 06:17:13.120956 204006 beta_rowset.cpp:55] failed to open segment {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_0.dat under rowset {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf : Not found: {path_to_data}/data/873/128312153/973533064/0200000000000be9274d994a6437af1e83d38b865617b9bf_0.dat: No such file or directory (error 2)%
```
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org