You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/04/01 10:41:31 UTC
[GitHub] [incubator-doris] vagetablechicken opened a new issue #3243:
PTabletWriterOpenRequest should be independently configured
vagetablechicken opened a new issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243
We use `config::tablet_writer_rpc_timeout_sec` to set the rpc timeout in OlapTableSink. But sink node contains two rpc request types, `PTabletWriterOpenRequest` and `PTabletWriterAddBatchRequest`.
For example:
1. the default rpc is 10min. And the whole plan_fragment_executor timeout is 10min.
1. When I do sink->open(), if one be can't response to me, I will be blocked in NodeChannel::open_wait() for 10min.
1. I want to execute the plan(the loop of get_next() & sink->send()). But the plan fragment has timed out(10min).
It's only one channel open failed, won't break the plan execution. And we can finish the plan before the plan_fragment_executor timeout(10min). But we waste time in waiting`PTabletWriterOpenResult`.
So I think it's better to separate TabletWriterOpen timeout from TabletWriterAddBatch timeout.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
imay commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-608203924
> I think "open timeout" should be treated as a routine problem.
OK, looking forward to you PR to make it better.
> About the open timeout case we found, it's probably because the connected BE is overload. Actually the whole cluster is overload. ðŸ˜
> I will keep an eye on it.
Looking forward to your feedback
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] vagetablechicken edited a comment on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
vagetablechicken edited a comment on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-607719959
> I think you can use config:: tablet_writer_rpc_timeout_sec / 10 set PTabletWriterOpenRequest timeout.
It can be the suggestion value. But I think it's more flexible to add a `config::tablet_writer_open_rpc_timeout_sec`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] vagetablechicken commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
vagetablechicken commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-607719959
> I think you can use config:: tablet_writer_rpc_timeout_sec / 10 set PTabletWriterOpenRequest timeout.
It can be the suggestion value. But I think it's more flexible to add a `config::tablet_writer_open_rpc_timeout_sec`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] chaoyli commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
chaoyli commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-607194747
I think you can use config:: tablet_writer_rpc_timeout_sec / 10 set PTabletWriterOpenRequest timeout.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
imay commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-608198561
@vagetablechicken
I agree with you on the new config. However I think it will be better if we can find the root cause of open timeout.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay closed issue #3243: PTabletWriterOpenRequest
should be independently configured
Posted by GitBox <gi...@apache.org>.
imay closed issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] vagetablechicken commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
vagetablechicken commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-608202863
> @vagetablechicken
> I agree with you on the new config. However I think it will be better if we can find the root cause of open timeout.
I think "open timeout" should be treated as a routine problem. About the open timeout case we found, it's probably because the connected BE is overload. Actually the whole cluster is overload. :sob:
I will keep an eye on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] vagetablechicken commented on issue #3243:
PTabletWriterOpenRequest should be independently configured
Posted by GitBox <gi...@apache.org>.
vagetablechicken commented on issue #3243: PTabletWriterOpenRequest should be independently configured
URL: https://github.com/apache/incubator-doris/issues/3243#issuecomment-608236431
Oh, my bad. I debuged in old branch, so I missed this commit 8eb413fa699fe4c981e1e0e01be7eb781e2ed56f.
https://github.com/apache/incubator-doris/blob/3f247b0d2dc424335c0b81259733cc8cbc2fcc1f/be/src/exec/tablet_sink.cpp#L82
rpc_timeout_ms is set by query timeout, not the `config::tablet_writer_rpc_timeout_sec`.
But the solution can remain the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org